ci: rehearsable, promote-only release pipeline with build/publish job split - #487
Draft
fforootd wants to merge 10 commits into
Draft
ci: rehearsable, promote-only release pipeline with build/publish job split#487fforootd wants to merge 10 commits into
fforootd wants to merge 10 commits into
Conversation
Restructure the release pipeline so build and publish are separate jobs with an artifact handoff, keeping all orchestration in the Moon graph: - release:build produces every promotable artifact except container images; the release-publish workflow uploads dist/release and the new publish job downloads it and never rebuilds (check-release-graph now asserts the publish graph contains no build tasks). - release:gate evaluates publish preconditions once and records the decision in dist/release/publish-plan.json; every publish surface (verify-artifacts, publish-npm, publish-tags, publish-container, publish-github) reads the plan and no-ops green on a skip. - npm moves from 'changeset publish' to tarball promotion: the new scripts/release-npm.mjs publishes the prebuilt tarballs that are not yet on the registry (npm view existence check), with the dist-tag resolved from Changesets pre mode. prepack rebuilds now happen exactly once, at pack time in the build job. - git tags become real: publish-tags runs 'changeset tag' and pushes to origin. changeset publish previously created tags only on the runner, so remote package tags have been stale since alpha.1. - release:snapshot becomes release:build plus a loaded host-platform image; scripts/check.mjs and the runbook follow the new entrypoints. - verify-artifacts additionally recomputes archive checksums so the publish job proves it promotes exactly the bits the build job made.
Add a gate-free release rehearsal so the publish surfaces run (dry) in PR CI with the same environment shape as the publish job: - release:rehearse-plan writes an always-dry publish plan without the version-commit/changeset/main gates, which would fail any PR that adds a changeset. - release:rehearse (Moon task) runs release:build, the rehearsal plan, and the same verify-artifacts/publish-container/publish-github tasks the real publish uses; check-release-graph pins the dep order and asserts the gate, publish-npm, and publish-tags stay out. - --npm-rehearsal (version PRs and manual dry runs) publishes every built tarball to a throwaway local Verdaccio through the production publishNpmTarballs path - a fresh registry must accept all 19 packages or the rehearsal fails. Loopback-only registry guard. - ci.yml full mode replaces the plain artifact build with the rehearsal, invoked 'env -u CI' (GITHUB_ACTIONS kept) with GITHUB_TOKEN present and provenance off - the exact environment interactions behind the alpha.14 release failures (#477, #479) now execute on every PR. - ci.yml version-only mode rehearses with --npm-rehearsal, so the version PR - the last gate before the publishing commit - exercises version computation, prepack rebuilds, and real npm publishes.
Add a deployment smoke that tests what users actually deploy - the documented compose file - instead of the CLI's internal docker runtime: - scripts/deploy-smoke.mjs copies docs/operations/docker-compose.yaml verbatim into a scratch dir, generates the required .env, overlays the documented bootstrap-users mount (command: server --user-file), boots it with docker compose up --wait, and probes: /healthz within 90s, /livez + /readyz, services still running after bootstrap (the Postgres write-path proof - health endpoints are stubs), and a structured JSON 401 from unauthenticated /sessions/me (proves the generated API router is in the binary). Targets are a map so a helm chart target can be added without touching callers. Diagnostics (compose logs + ps) are captured on failure and uploaded in CI. - release:smoke-deploy task depends on verify-artifacts, not the build chain, so the publish job can smoke downloaded artifacts; without --image it builds a host-platform image from the release docker context via the existing buildContainerImage path. - release:rehearse gains --smoke; ci.yml runs it path-conditionally on full PRs (scripts/ci-mode.mjs now exports detectCiMode with a deploy_smoke output for Dockerfile/compose/release-script changes), always on version PRs, and release-publish.yml smokes on manual dry-run dispatches. - Fix the stale quickstart-smoke/GoReleaser comment in docs/operations/env.example; first unit coverage for ci-mode.mjs.
Release graphs promote; they never re-verify. The gate for shipping is green CI on the PRs that produced the commit - re-running tests inside the publish environment is how the alpha.14 release broke (#479: cli:test met a real GITHUB_TOKEN that no PR run ever had). - release:build drops its server:test dep; cli:build-release drops its cli:test dep. The publish/build jobs stop running Go and CLI tests (~4-6 min cold), and the env-sensitive-test failure class becomes structurally impossible. - check-release-graph replaces the removed edge with the inverse, stronger invariant: assertNoTestTasksInGraph fails if any :test task appears in the transitive graphs of release:build, release:pack, release:snapshot, release:publish, or release:rehearse (verified to fire by re-adding the dep locally). - The public-dist mutex on cli:test stays as dormant defense for explicit multi-target runs; a new TEST_MUTEX_REQUIREMENTS assertion keeps it from being silently dropped now that no release graph schedules the task. - PR CI coverage is unchanged: moon ci :test and the explicit server:test / postgres steps still run on every full PR.
Manual dispatch of release-publish becomes 'idempotently publish whatever is missing for the checked-out main version' - recovery stops being a special mode: - containerImageExists (docker buildx imagetools inspect; only a missing manifest counts as absent, auth/network errors surface) and ensureContainerTags (re-point missing secondary tags like :latest at the published manifest without rebuilding) land in release-artifacts.mjs. publish-container checks-and-skips; version tags are immutable, the corrupt-image escape hatch is deleting the GHCR tag (documented in the runbook). Dry runs report the registry state tolerantly for credential-less PR rehearsals. - The gate gains manual mode (--manual / ZITADEL_RELEASE_MANUAL): skip commit-message detection entirely, keep the semver check, assertNoUnrecordedPendingChangesets, and the main-branch gate. The automatic push path is unchanged. shouldFailManualPublishSkip (#480) is deleted - a manual no-op is now a green, meaningful result instead of a failure. - The recover_version workflow input is removed; --recover-version stays one release as a deprecated alias that asserts the checked-out version and warns. A fully-published re-run is a green no-op with per-surface skip logs - that no-op is the idempotency acceptance test. - Runbook: Manual controls/Recover collapse to dispatch-dry-run, read the per-surface summary, dispatch-real, verify.
oxlint no-empty-function rejects '() => {}'; follow the existing
'() => undefined' convention from local-registry.mjs.
The real compose smoke (first run against the actual image) showed unauthenticated GET /sessions/me returns 400 req.invalid - the session cookie is a required parameter, so the rejection comes from the decode layer through the centralized error handler, not a 401 from a security handler. That structured response is exactly what the probe wants to prove (generated router + error model present in the binary), so the probe now accepts 400/401/403 and additionally requires the error-model shape (object with a string code) instead of any JSON object.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
🦋 Changeset detectedLatest commit: 9929bee The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
fforootd
added a commit
that referenced
this pull request
Jul 22, 2026
## Summary <!-- Briefly describe what changed and why. --> - `buildServerBinaries` compiled the 5 server platforms strictly in sequence and `packPublicPackages` packed the 19 public packages one by one. Both loops are independent per item, so they now run through a new `mapWithConcurrency` helper in `scripts/dev-process.mjs` (order-preserving results, no new work started after a failure, in-flight children awaited). - All 5 `go build`s run at once — the Go build cache is concurrency-safe; the win comes from overlapping the serial link phases. Measured locally with a cold cache: 1m56 → 1m33 (~20%); warm-cache runs (the CI norm with the restored setup-go cache) save proportionally more. - `pnpm pack` runs with concurrency 8 into the shared tarballs dir (distinct filenames; the `@zitadel/cli` prepack rebuild keeps its production telemetry-channel stamp). - This shortens the release-artifact build step every full PR pays (and the publish/build job), independent of the pipeline rework in #487 — which merges over this cleanly since it doesn't touch these function bodies. ## Validation <!-- List exact commands run. If validation was not run, say so explicitly. --> - `moon run release:pack` then `node apps/cli-journey-e2e/scripts/verify-tarballs.mjs dist/release/<v>/npm` — 19 installable tarballs verified - Cold-cache timing A/B via direct `buildServerBinaries` invocation (`go clean -cache` before each): sequential 1m55.9s, parallel 1m32.8s - `vitest run tests/unit/scripts/dev-process.test.ts tests/unit/scripts/release-artifacts.test.ts` — 12 tests green (new coverage: order preservation, concurrency cap, fail-fast semantics, empty input) - `moon run cli:lint` — clean ## Release notes / changeset - No changeset required — no shipped behavior changed (release build scripts and tests only; no publishable package source is touched). ## Notes - Failure semantics are deliberately "finish in-flight, start nothing new": spawned `go`/`pnpm` children cannot be abandoned safely, so a failing platform/package surfaces after the current lane items settle, with the first error rethrown. - Log lines from concurrent packs interleave; Moon already interleaves parallel task output, so this matches the existing CI log shape.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mainmoved; historical recovery requires bothrelease_refandrecover_version.alphaorlatesttag; historical backfills use and remove a temporary recovery tag so the active channel never rolls backward.npm publishremains OIDC; standalone dist-tag repair uses a separately scoped token only for the exact tag command.Validation
moon run cli:test— 107 files and 829 tests passedmoon run release:check-graph— passedmoon run cli:typecheck— passedmoon run workspace:check -- --only release— passed, including artifact build and verificationcorepack pnpm exec changeset status --since origin/main— no packages to bumpgit diff --check— passedRelease notes / changeset
.changeset/rehearse-release-publishing.md. This changes release and CI wiring but does not change shipped package behavior.Notes
NPM_DIST_TAG_TOKENas a short-lived, package-scoped granular npm token. The secret is not currently configured. Normal first-time package publication does not use it, but stale-tag repair and historical temporary-tag cleanup fail clearly without it.zitadel/nextgenandrelease-publish.ymlon every public package.