ci: track root causes and restore CI on latest and release/v11

[reggi]

Purpose

This is the umbrella tracking issue for restoring CI and post-release integration on npm/cli's latest and release/v11 branches. It owns the overall failure inventory, coordinates fixes across npm/cli and upstream fixtures/actions, and tracks verification on both branches.

Track issues by root cause, not by error message, job, runtime, or branch. A child issue can contain multiple failures. Where the initiating cause is not yet established, retain an explicitly scoped investigation rather than presenting an infrastructure hypothesis as fact.

Investigated scope

The inventory below covers all 14 failed jobs in the four completed September 22, 2026 branch-head CI and Release runs:

Branch	Release	Commit	Workflow	Failed jobs
latest	npm 12.1.0	c039090578a5b21a1aa3aba9c96e199feaf1823a	Release	9
latest	npm 12.1.0	c039090578a5b21a1aa3aba9c96e199feaf1823a	CI	1
release/v11	npm 11.20.0	d12b9434dd010b5fb7044c3cc149cdda317813f8	Release	3
release/v11	npm 11.20.0	d12b9434dd010b5fb7044c3cc149cdda317813f8	CI	1

This is a complete inventory for those runs, not a claim that every historical failure has the same causes. The failed Release workflow status must not be equated with failed package publication: these failures occur in downstream integration.

Root-cause and incident owners

• ci: Node.js nightly integration fails while extracting source archive #10016
• ci: binary-split CITGM source download uses stale repository owner #10020
• ci: release integration selects Node 23 outside npm 12 supported engines #10024
• ci: CITGM Git-dependency fixtures lack npm 12 policy configuration #10026
• ci: CITGM relies on native-build option forwarding rejected by npm 12 #10023
• ci: live-registry smoke test fails on a socket reset with fetch retries disabled #10022
• ci: Cygwin provisioning fails before Windows shim tests run #10025

Owner issue	Group	latest Release	v11 Release	latest CI	v11 CI
#10016	Invalid Node nightly source response; publication-race hypothesis remains unproven	1	1	0	0
#10020	Stale binary-split source repository metadata	2	2	0	0
#10024	Stable integration matrix selects Node 23 outside npm 12's engines	2	0	0	0
#10026	CITGM Git dependency is blocked by npm 12's default policy	2	0	0	0
#10023	CITGM native-build option forwarding is rejected by npm 12	2	0	0	0
#10022	Live-registry smoke-test socket reset with retries disabled; initiating cause unresolved	0	0	1	0
#10025	Cygwin provisioning fails before shim tests; installer cause unresolved	0	0	0	1
Total		9	3	1	1

Complete failed-job inventory

Branch/workflow	Failed job	Owner
latest Release	Build nightly	#10016
v11 Release	Build nightly	#10016
latest Release	Node 22: binary-split	#10020
latest Release	Node 23: binary-split	#10020
v11 Release	Node 22: binary-split	#10020
v11 Release	Node 23: binary-split	#10020
latest Release	Node 23: test nodejs	#10024
latest Release	Node 23: test npm	#10024
latest Release	Node 22: JSONStream	#10026
latest Release	Node 23: JSONStream	#10026
latest Release	Node 22: thread-sleep	#10023
latest Release	Node 23: thread-sleep	#10023
latest CI	Smoke Tests	#10022
v11 CI	Windows Shims Tests	#10025

Grouping decisions and important qualifications

The Node 23 warning assertion, npm smoke stderr mismatch, EBADENGINE, subsequent unsatisfied registry mocks, and engine-dependent coverage failure belong under #10024. They should not produce separate tickets per assertion.

The two CITGM policy/configuration failures require different fixes. Allowing Git dependencies does not make --build-from-source a recognized npm CLI flag, and removing that flag does not permit Git dependencies. Both failures occur on supported Node 22, independently of #10024.

#10023 preserves the history from mapbox/node-pre-gyp#58: npm install --build-from-source was a documented and functional way to forward a native installer's configuration through npm. node-pre-gyp still implements source builds; npm 12 now rejects the unknown npm CLI flag before that installer can use it. The migration must verify the actual fixture's installer and preserve any intended source-build coverage, not simply remove the option and assume success.

#10016 and #10020 share the diagnostic weakness of streaming curl -sSL output into tar, but their upstream conditions are distinct. Better HTTP handling does not repair stale repository metadata or guarantee an archive exists. Both nightly jobs selected the same archive, but their logs do not preserve the HTTP response status/body, so a publication race is not a proven fact.

The socket reset in #10022 and the Cygwin setup failure in #10025 are separately owned investigations, not evidence of one shared outage. In particular, the Cygwin downgrade conflict and subsequent installer exit are both observed, but a causal relationship has not been established.

Cross-cutting follow-up owned by this tracker

• Preserve HTTP status and useful failure diagnostics for source downloads without feeding non-success responses directly into extraction.
• Correct FINALEXIT=STEPEXIT in the npm integration result aggregation and its template. It currently assigns a literal string and later emits exit: STEPEXIT: numeric argument required. This masks the intended exit status after earlier test failures; it is not another primary cause of the 14 failed jobs.
• Ensure upstream CITGM fixes reach the published version used by matrix generation. A change merged upstream but not consumed by npm/cli does not close the coverage gap.
• Apply branch-appropriate workflow/template changes to both release lines and document local equivalents for any new diagnostic or validation commands.
• Attach post-fix run evidence for both branches and confirm the affected tests actually execute rather than being skipped, suppressed, or bypassed.

Completion criteria

Close this tracker only when every child issue is resolved or has an explicitly justified disposition, the affected coverage is restored on both branches, and any remaining infrastructure uncertainty is documented with an owner and evidence. A rerun that happens to pass is not by itself proof of root cause, and making one error disappear is not proof that the rest of an integration job succeeded.

npm 12.2.0 follow-up inventory

The npm 12.2.0 Release workflow at c276cadf785c1018d82dd287fe7742567055efb9 reproduced the existing Node 23, JSONStream, binary-split, and thread-sleep failures and exposed two additional root causes:

• ci: nightly npm smoke test rejects expected engine warning #10064
• ci: nightly native CITGM fixtures ignore configured Node headers #10065

Workflow: https://github.com/npm/cli/actions/runs/36746332014

The ordinary branch CI workflow passed: https://github.com/npm/cli/actions/runs/36746330757

Owner issue	Group	Failed jobs
#10024	Stable Node 23 remains outside npm 12's supported engines	2
#10026	JSONStream Git dependency remains blocked by npm 12 policy	3
#10020	binary-split still consumes stale metadata because the merged CITGM change was not in the published version used by this run	3
#10023	thread-sleep still supplies the rejected --build-from-source npm CLI option	3
#10064	Nightly npm smoke test rejects the expected unsupported-engine warning	1
#10065	leveldown and microtime do not consume the configured Node source directory on nightly	2
Total		14

npm 12.2.0 failed-job additions

Branch/workflow	Failed job	Owner
latest Release	Nightly: test npm	#10064
latest Release	Nightly: leveldown	#10065
latest Release	Nightly: microtime	#10065

The release/v11 branch CI for npm 11.21.0 also passed. Its separate release-please failure occurred before downstream integration and is tracked in #10066.

npm 11.21.0 release automation failure

• release: GitHub 502 leaves npm 11.21.0 release records incomplete #10066

The npm 11.21.0 Release workflow failed in template-oss-release-please before downstream release integration could run:

Workflow: https://github.com/npm/cli/actions/runs/36756679730

Failing job: https://github.com/npm/cli/actions/runs/36756679730/job/110028585934

The corresponding branch CI workflow passed: https://github.com/npm/cli/actions/runs/36756678607

Both changed packages were published to the npm registry, but their GitHub release records remain incomplete:

Package	Registry publication	Missing Git tag	Missing GitHub Release
npm@11.21.0	Published	v11.21.0	v11.21.0
@npmcli/config@10.14.0	Published	config-v10.14.0	config-v10.14.0

The confirmed fatal error was an unhandled GitHub HTTP 502 Bad Gateway response during the release-please step. The 11-of-13 component discovery message and the parser warning for the old non-Conventional Commit Fix typos in some files were not fatal.

The first remediation is to rerun the complete failed workflow so skipped dependent jobs are reevaluated. A successful rerun should create both tags and GitHub Releases and then execute Release Integration. If the 502 recurs, capture the request endpoint class, response status, and retry behavior before manually creating release records.

Upstream CITGM changes

Four npm/cli integration failures have corresponding changes in nodejs/citgm:

npm/cli owner	CITGM pull request	State	Effect
#10020	nodejs/citgm#1142	Merged	Updates binary-split to use the current max-mapper/binary-split repository.
#10065	nodejs/citgm#1122	Merged	Removes the deprecated and archived leveldown fixture. This resolves only the leveldown half of #10065; microtime remains.
#10026	nodejs/citgm#1158	Open	Allows JSONStream's transitive Git test dependency in that fixture's npm 12 environment.
#10023	nodejs/citgm#1159	Open	Removes the obsolete --build-from-source argument and native tag from the now pure-JavaScript thread-sleep fixture.

The merged binary-split and leveldown changes are included in the pending CITGM 10.0.4 release pull request, nodejs/citgm#1147. None of these changes should be treated as consumed by npm/cli until a CITGM release containing them is published, the npm/cli matrix resolves that version, and the affected package tests execute successfully.