prompt: file the multi_dataset/jax_likelihood XLA compile stall - #266
Merged
Conversation
multi_dataset/jax_likelihood/mge.py was parked out of the PR smoke gate (autolens_workspace_test#262) to unblock the pynufft removal, after hanging to the 300s cap in 4/4 jobs on a PR whose diff that gate never executes. Parking removes the symptom, so this prompt is what keeps the cause visible. It is not a new bug: no_run.yaml already records the sibling delaunay.py as an "intermittent XLA compile stall, family-wide" (autolens_workspace_test#245), and mge.py's signature matches exactly — 32s when it passes, otherwise no progress at all. The real cost is the accumulation: the whole multi_dataset/jax_likelihood family is now quarantined somewhere, three times over, without anyone having diagnosed the stall. The prompt asks for the stall to be characterised rather than the scripts sped up, and for the runner to leave a stack trace on timeout — today a hang produces no evidence, which is why it keeps recurring.
Jammy2211
pushed a commit
that referenced
this pull request
Aug 23, 2026
…uard added Record start_workspace completion and the pushed implementation on both target repos. Entry drops its `worktree:` field: this is a web-github run with no local worktree, and a path there would read to a local session as an existing checkout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KMknTjH347FBZUE82nNEby
Jammy2211
pushed a commit
that referenced
this pull request
Aug 23, 2026
autolens_workspace_test#268 and autogalaxy_workspace_test#107, both labelled pending-release. Ship gate recorded as satisfied rather than skipped: Heart returned STALE with no red or yellow reasons, and the dev-ship gate treats an evidence gap as passing since it is organism-scope, not branch-scope. Not merged — merge stays a human act, and CI is still running. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KMknTjH347FBZUE82nNEby
Jammy2211
pushed a commit
that referenced
this pull request
Aug 23, 2026
…hang follow-up Both PRs merged: autolens_workspace_test#268 (9348e152) and autogalaxy_workspace_test#107 (0a91883d). Dated completion record written, the active.md entry pruned, index and dashboard regenerated. The record keeps the transferable lesson: the prompt offered "remove the pin or restate its real range" and quoted autonerves as jax<0.11.0 — a quote that was already stale, the cap having moved to <0.12.0. Restating would have held CI a minor version behind the supported range. CI confirmed it: with the pin gone the resolver reaches jax 0.11.1 and the suite passes there. Assert a range; do not own a second copy of it. Also files draft/bug/workspaces/intermittent_smoke_hang_jax_mge.md for the intermittent hang on imaging/jax_likelihood/mge.py that surfaced during the ship. Acquitted of being caused by the jax bump (the same script passed under 0.11.1 in autolens at 23/23, and a re-run of the identical commit did not reproduce it); it is a pre-existing pattern with four prior ~6h runs in that repo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KMknTjH347FBZUE82nNEby
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Files
draft/bug/autolens_workspace_test/multi_dataset_jax_likelihood_xla_stall.md.multi_dataset/jax_likelihood/mge.pywas parked out of the PR smoke gate in PyAutoLabs/autolens_workspace_test#262, to unblock the pynufft removal (#261) after it hung to the 300s cap in 4/4 jobs on a PR whose diff that gate never executes. Parking removes the symptom — this prompt is what keeps the cause visible.It is not a new bug.
config/build/no_run.yaml:87already records the siblingdelaunay.pyas an "intermittent XLA compile stall, family-wide" (autolens_workspace_test#245, which also hitag_test rectangular.py).mge.py's signature matches exactly: 32s standalone when it passes, otherwise no progress at all until the cap.The cost worth surfacing is the accumulation. The whole
multi_dataset/jax_likelihood/family is now quarantined somewhere —delaunay.pyanddelaunay_mgeandshared_preloads.pyinno_run.yaml,shared_preloads.pyand nowmge.pyinsmoke_tests.txt— across three separate incidents, and nobody has diagnosed the stall itself.So the prompt asks for two things a per-script speedup would not give:
faulthandleron SIGTERM at the cap) would settle it faster than more timing data.TIMEOUT, which is exactly why this has been quarantined three times without a diagnosis.Test Plan
python3 scripts/lifecycle.py check→lifecycle check: OKdashboard.md/dashboard.htmlregenerated viapyauto-brain intake --apply dashboard(149 prompts), not hand-editeddraft/bug/<target>/per the taxonomy, with the target repo as the second folderGenerated by Claude Code