Skip to content

fix: fetch only new Drupal core commits on top of the seed cache - #262

Merged
rfay merged 1 commit into
mainfrom
20261117_rfay_drupal_core_seed_fetch
Nov 17, 2026
Merged

rfay merged 1 commit into
mainfrom
20261117_rfay_drupal_core_seed_fetch

Conversation

@rfay

@rfay rfay commented Nov 17, 2026

Copy link
Copy Markdown
Member

Short Summary (TL;DR)

Drupal core workspaces downloaded all of core's history on every start, and a failed download left an empty checkout. This makes the seed cache's refs survive, so the fetch is a real delta and a failed fetch falls back to the cache. It also makes the hourly updater actually move the cache's branches.

The Issue

Drupal 11 and 12 integration jobs failed at "Verify workspace — web root exists", for example https://github.com/ddev/coder-ddev/actions/runs/44144337457. Each failure logged "Fetch from origin failed — continuing with cached refs" after 4 to 10 minutes. Drupal 12 then failed to install, and Drupal 11 failed to check out 11.x. Passing jobs also spent 80 to 514 seconds on that "delta" fetch.

There were three causes:

  • The startup script fetched the cache into a drupalcache remote and then removed that remote. git remote remove also deletes the remote's refs, so the origin fetch had no refs to negotiate with and downloaded everything.
  • With no refs left, a failed origin fetch made git checkout -b main --track origin/main fail. Nothing checked its exit code, so setup reported "Drupal core cloned" with an empty working tree.
  • git clone --bare configures no fetch refspec. So update-drupal-cache ran git fetch --all, which only wrote FETCH_HEAD and left every branch where the first clone put it.

How This PR Solves The Issue

  • drupal-core/template.tf fetches the cache's branches into refs/remotes/origin/* and its tags into refs/tags/*. The origin fetch now sends what the workspace already has. When the origin fetch fails, main and the version branches still check out from the cache.
  • A failed checkout of main now sets SETUP_FAILED with its own message.
  • update-drupal-cache fetches +refs/heads/*:refs/heads/* and +refs/tags/*:refs/tags/* into a bare cache.
  • drupal-integration-test.yml prints SETUP_STATUS.txt and the end of /tmp/drupal-setup.log when a job fails. Before this, these failures showed only "exit code 1".
  • The seed cache section of the server setup docs matches the new behavior. Its recovery steps now re-clone with --bare.

Manual Testing Instructions

On the Coder host, as the owner of the seed cache:

git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main
git ls-remote https://git.drupalcode.org/project/drupal.git refs/heads/main

Expect the cache to be behind upstream before the new updater runs.

sudo install -m 755 drupal-core/scripts/update-drupal-cache /usr/local/bin/update-drupal-cache
sudo systemctl start drupal-cache-updater.service
git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main

Expect the cache's main to match upstream.

Push this branch's drupal-core template, then create a Drupal 12 workspace. In /tmp/drupal-setup.log, expect "Fetched origin" to take seconds, not minutes, and /home/coder/drupal-core/index.php to exist.

Automated Testing Overview

Claude Code ran terraform fmt -check -recursive, make validate, and make test-templates on Linux, and all passed. Claude Code also checked the git commands against small local test repositories. The new fetch sends "have" lines, and with origin unreachable, main and 11.x still check out from the cached refs. The real fetch time on staging needs CI to confirm. Nobody ran make docs-check.

Release/Deployment Notes

Reinstall update-drupal-cache to /usr/local/bin on every Coder host that has a seed cache, because the timer runs that installed copy.

🤖 Developed with assistance from Claude Code

https://claude.ai/code/session_01T3Cc2G6867sD8ZdDCVZ7Em

## Short Summary (TL;DR)

Drupal core workspaces downloaded all of core's history on every start, and a failed download left an empty checkout. This makes the seed cache's refs survive, so the fetch is a real delta and a failed fetch falls back to the cache. It also makes the hourly updater actually move the cache's branches.

## The Issue

Drupal 11 and 12 integration jobs failed at "Verify workspace — web root exists", for example https://github.com/ddev/coder-ddev/actions/runs/44144337457. Each failure logged "Fetch from origin failed — continuing with cached refs" after 4 to 10 minutes. Drupal 12 then failed to install, and Drupal 11 failed to check out 11.x. Passing jobs also spent 80 to 514 seconds on that "delta" fetch.

There were three causes:

- The startup script fetched the cache into a `drupalcache` remote and then removed that remote. `git remote remove` also deletes the remote's refs, so the origin fetch had no refs to negotiate with and downloaded everything.
- With no refs left, a failed origin fetch made `git checkout -b main --track origin/main` fail. Nothing checked its exit code, so setup reported "Drupal core cloned" with an empty working tree.
- `git clone --bare` configures no fetch refspec. So `update-drupal-cache` ran `git fetch --all`, which only wrote `FETCH_HEAD` and left every branch where the first clone put it.

## How This PR Solves The Issue

- `drupal-core/template.tf` fetches the cache's branches into `refs/remotes/origin/*` and its tags into `refs/tags/*`. The origin fetch now sends what the workspace already has. When the origin fetch fails, `main` and the version branches still check out from the cache.
- A failed checkout of `main` now sets `SETUP_FAILED` with its own message.
- `update-drupal-cache` fetches `+refs/heads/*:refs/heads/*` and `+refs/tags/*:refs/tags/*` into a bare cache.
- `drupal-integration-test.yml` prints `SETUP_STATUS.txt` and the end of `/tmp/drupal-setup.log` when a job fails. Before this, these failures showed only "exit code 1".
- The seed cache section of the server setup docs matches the new behavior. Its recovery steps now re-clone with `--bare`.

## Manual Testing Instructions

On the Coder host, as the owner of the seed cache:

```bash
git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main
git ls-remote https://git.drupalcode.org/project/drupal.git refs/heads/main
```

Expect the cache to be behind upstream before the new updater runs.

```bash
sudo install -m 755 drupal-core/scripts/update-drupal-cache /usr/local/bin/update-drupal-cache
sudo systemctl start drupal-cache-updater.service
git -C ~/cache/drupal-core-seed log -1 --format='%ci %h %s' main
```

Expect the cache's `main` to match upstream.

Push this branch's drupal-core template, then create a Drupal 12 workspace. In `/tmp/drupal-setup.log`, expect "Fetched origin" to take seconds, not minutes, and `/home/coder/drupal-core/index.php` to exist.

## Automated Testing Overview

Claude Code ran `terraform fmt -check -recursive`, `make validate`, and `make test-templates` on Linux, and all passed. Claude Code also checked the git commands against small local test repositories. The new fetch sends "have" lines, and with origin unreachable, `main` and `11.x` still check out from the cached refs. The real fetch time on staging needs CI to confirm. Nobody ran `make docs-check`.

## Release/Deployment Notes

Reinstall `update-drupal-cache` to `/usr/local/bin` on every Coder host that has a seed cache, because the timer runs that installed copy.

🤖 Developed with assistance from [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3Cc2G6867sD8ZdDCVZ7Em
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Previews

The start.coder.ddev.com landing-page preview has been removed. Read the Docs keeps or removes its own docs preview.

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 coder-ddev | 🛠️ Build #35969059 | 📁 Comparing a9ca682 against latest (b46b234)

  🔍 Preview build  

1 file changed
± self-hosting/server-setup/index.html

@rfay
rfay merged commit 7cdac4e into main Nov 17, 2026
21 checks passed
@rfay
rfay deleted the 20261117_rfay_drupal_core_seed_fetch branch November 17, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants