Repository navigation
fix: fetch only new Drupal core commits on top of the seed cache - #262
Merged
Merged
Conversation
## Short Summary (TL;DR) Drupal core workspaces downloaded all of core's history on every start, and a failed download left an empty checkout. This makes the seed cache's refs survive, so the fetch is a real delta and a failed fetch falls back to the cache. It also makes the hourly updater actually move the cache's branches. ## The Issue Drupal 11 and 12 integration jobs failed at "Verify workspace — web root exists", for example https://github.com/ddev/coder-ddev/actions/runs/44144337457. Each failure logged "Fetch from origin failed — continuing with cached refs" after 4 to 10 minutes. Drupal 12 then failed to install, and Drupal 11 failed to check out 11.x. Passing jobs also spent 80 to 514 seconds on that "delta" fetch. There were three causes: - The startup script fetched the cache into a `drupalcache` remote and then removed that remote. `git remote remove` also deletes the remote's refs, so the origin fetch had no refs to negotiate with and downloaded everything. - With no refs left, a failed origin fetch made `git checkout -b main --track origin/main` fail. Nothing checked its exit code, so setup reported "Drupal core cloned" with an empty working tree. - `git clone --bare` configures no fetch refspec. So `update-drupal-cache` ran `git fetch --all`, which only wrote `FETCH_HEAD` and left every branch where the first clone put it. ## How This PR Solves The Issue - `drupal-core/template.tf` fetches the cache's branches into `refs/remotes/origin/*` and its tags into `refs/tags/*`. The origin fetch now sends what the workspace already has. When the origin fetch fails, `main` and the version branches still check out from the cache. - A failed checkout of `main` now sets `SETUP_FAILED` with its own message. - `update-drupal-cache` fetches `+refs/heads/*:refs/heads/*` and `+refs/tags/*:refs/tags/*` into a bare cache. - `drupal-integration-test.yml` prints `SETUP_STATUS.txt` and the end of `/tmp/drupal-setup.log` when a job fails. Before this, these failures showed only "exit code 1". - The seed cache section of the server setup docs matches the new behavior. Its recovery steps now re-clone with `--bare`. ## Manual Testing Instructions On the Coder host, as the owner of the seed cache: ```bash git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main git ls-remote https://git.drupalcode.org/project/drupal.git refs/heads/main ``` Expect the cache to be behind upstream before the new updater runs. ```bash sudo install -m 755 drupal-core/scripts/update-drupal-cache /usr/local/bin/update-drupal-cache sudo systemctl start drupal-cache-updater.service git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main ``` Expect the cache's `main` to match upstream. Push this branch's drupal-core template, then create a Drupal 12 workspace. In `/tmp/drupal-setup.log`, expect "Fetched origin" to take seconds, not minutes, and `/home/coder/drupal-core/index.php` to exist. ## Automated Testing Overview Claude Code ran `terraform fmt -check -recursive`, `make validate`, and `make test-templates` on Linux, and all passed. Claude Code also checked the git commands against small local test repositories. The new fetch sends "have" lines, and with origin unreachable, `main` and `11.x` still check out from the cached refs. The real fetch time on staging needs CI to confirm. Nobody ran `make docs-check`. ## Release/Deployment Notes Reinstall `update-drupal-cache` to `/usr/local/bin` on every Coder host that has a seed cache, because the timer runs that installed copy. 🤖 Developed with assistance from [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T3Cc2G6867sD8ZdDCVZ7Em
PreviewsThe start.coder.ddev.com landing-page preview has been removed. Read the Docs keeps or removes its own docs preview. |
Documentation build overview
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Short Summary (TL;DR)
Drupal core workspaces downloaded all of core's history on every start, and a failed download left an empty checkout. This makes the seed cache's refs survive, so the fetch is a real delta and a failed fetch falls back to the cache. It also makes the hourly updater actually move the cache's branches.
The Issue
Drupal 11 and 12 integration jobs failed at "Verify workspace — web root exists", for example https://github.com/ddev/coder-ddev/actions/runs/44144337457. Each failure logged "Fetch from origin failed — continuing with cached refs" after 4 to 10 minutes. Drupal 12 then failed to install, and Drupal 11 failed to check out 11.x. Passing jobs also spent 80 to 514 seconds on that "delta" fetch.
There were three causes:
drupalcacheremote and then removed that remote.git remote removealso deletes the remote's refs, so the origin fetch had no refs to negotiate with and downloaded everything.git checkout -b main --track origin/mainfail. Nothing checked its exit code, so setup reported "Drupal core cloned" with an empty working tree.git clone --bareconfigures no fetch refspec. Soupdate-drupal-cacherangit fetch --all, which only wroteFETCH_HEADand left every branch where the first clone put it.How This PR Solves The Issue
drupal-core/template.tffetches the cache's branches intorefs/remotes/origin/*and its tags intorefs/tags/*. The origin fetch now sends what the workspace already has. When the origin fetch fails,mainand the version branches still check out from the cache.mainnow setsSETUP_FAILEDwith its own message.update-drupal-cachefetches+refs/heads/*:refs/heads/*and+refs/tags/*:refs/tags/*into a bare cache.drupal-integration-test.ymlprintsSETUP_STATUS.txtand the end of/tmp/drupal-setup.logwhen a job fails. Before this, these failures showed only "exit code 1".--bare.Manual Testing Instructions
On the Coder host, as the owner of the seed cache:
Expect the cache to be behind upstream before the new updater runs.
Expect the cache's
mainto match upstream.Push this branch's drupal-core template, then create a Drupal 12 workspace. In
/tmp/drupal-setup.log, expect "Fetched origin" to take seconds, not minutes, and/home/coder/drupal-core/index.phpto exist.Automated Testing Overview
Claude Code ran
terraform fmt -check -recursive,make validate, andmake test-templateson Linux, and all passed. Claude Code also checked the git commands against small local test repositories. The new fetch sends "have" lines, and with origin unreachable,mainand11.xstill check out from the cached refs. The real fetch time on staging needs CI to confirm. Nobody ranmake docs-check.Release/Deployment Notes
Reinstall
update-drupal-cacheto/usr/local/binon every Coder host that has a seed cache, because the timer runs that installed copy.🤖 Developed with assistance from Claude Code
https://claude.ai/code/session_01T3Cc2G6867sD8ZdDCVZ7Em