W1 · Phase 4 Stage 1: PositionsLH plumbing in the searches framework + hazard transects (#159) - #168
Merged
Merged
Conversation
Phase 4 Stage 1 (issue #159). Adds SEARCHES_POSITIONS=off|on, SEARCHES_POSITIONS_THRESHOLD=<float>|auto, SEARCHES_POSITIONS_FACTOR=<float> env vars, threaded through _setup.build_for_cell/_build_analysis into al.AnalysisImaging/AnalysisInterferometer's positions_likelihood_list, for single-plane imaging/interferometer cells only — group/cluster/datacube/ point_source raise NotImplementedError rather than silently ignoring the request. Truth positions are loaded from dataset/<class>/<instrument>/positions.json when present, else (imaging only) derived once from tracer.json via the simulator's own PointSolver recipe and committed (dataset/imaging/hst/positions.json, 4 quad-image positions). The 'auto' threshold mode replicates SLaM's result.positions_threshold_from(factor=3.0, minimum_threshold=0.2) and empirically collapses to the 0.2 floor for this truth-positions-through-truth-tracer idealisation. Correctness guard: PyAutoFit's identifier hashes only [search, model, unique_tag] -- the Analysis object (and therefore whether a positions penalty is attached) is not hashed. positions_arm_tag() composes into multi_start_unique_tag and is passed as unique_tag to build_nautilus/ build_nss via the new arm_unique_tag() helper, and into run_search's config_name, so a positions-on run can never silently share an output directory/identifier with the positions-off run of the "same" cell. _sampler_config_dict/multi_start_settings/_build_summary always record a "positions" block ({"enabled": false} when off). Adds scripts/misc/test/test_searches_positions.py (16 tests) covering env parsing/validation, arm-tag composition, the auto-threshold floor collapse, the dataset_class support guard, and assert_disjoint_output_paths (new _samplers.py helper) proving positions-on/off and differing-factor arms never collide on output_path or identifier. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
…argmax-switch) Phase 4 Stage 1 (issue #159, [H4.2]). New likelihood-tier hazard check scripts/misc/hazards/checks/positions_penalty.py exercising the real autolens.analysis.positions.PositionsLH.log_likelihood_penalty_from through a cheap synthetic single-plane Isothermal tracer (the truth quad-image positions and mass/shear from dataset/imaging/hst/{positions,tracer}.json, no imaging dataset needed -- point-tracing only), producing three Findings: - likelihood.positions-penalty.threshold-hinge: C0-continuous value, discontinuous jax-autodiff gradient at each max_sep==threshold crossing (tight +/-1e-6 local probe, not the coarse sweep grid -- see _local_hinge_probe's docstring for why the coarse grid alone over/under-states the value-continuity gap at factor=1e8). - likelihood.positions-penalty.interior-plateau: exact-zero penalty value AND gradient throughout the region where max_sep < threshold. - likelihood.positions-penalty.argmax-switch: which pair of the 4 truth positions realises max_sep changes discontinuously as einstein_radius sweeps [0, 3] -- a second, threshold-independent gradient kink source. Registered in _registry.py (new likelihood:positions_penalty SubjectSpec) and checks/__init__.py. Ran scan.py --subject likelihood (--check then a real write); the unrelated prior_support/active_set/conditioning_floor records that the same command also touches were reverted to origin's committed values (float-noise / a pre-existing environment-flaky PriorExitCheck failure, both unrelated to this change -- see the commit history for the plumbing commit's verification that the PriorExitCheck failure reproduces identically on an unmodified checkout). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
…paths Found by the smoke on/off verification for issue #159. run_search's output/searches/... PyAutoFit search directory was already correctly disambiguated by positions_arm_tag() (see the earlier plumbing commit), but resolve_output_paths derives its basename from cli.config_name -- the raw --config-name CLI flag -- not the positions-tagged config_name local computed later in run_search. A positions-on and positions-off run of the same --config-name therefore silently overwrote the same results/searches/.../<name>.json summary artifact, even though their PyAutoFit output directories never collided. Fix: pass resolve_output_paths a dataclasses.replace(cli, config_name=config_name) carrying the arm-tagged name, so the results-JSON path picks it up too. Verified with a live smoke pair (multi_start_prodigy_ autoconv/imaging/mge, n_starts=4, n_steps=20, seed=0): before the fix both runs wrote results/searches/.../smoke_smoke.json; after, the positions-on run writes smoke_smoke_pos_t0.3_f1e8.json and positions-off keeps smoke_smoke.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
…SULTS.md Phase 4 Stage 1 (issue #159). New scripts/misc/searches/positions_transects.py: pure-evaluation eager jax.value_and_grad transects on imaging/mge/hst (positions off at construction, analysis.positions_likelihood_list swapped per arm), reusing build_for_cell + the model.instance_from_vector(..., xp=jnp) / analysis.log_likelihood_function idiom already used by clipper_campaign.py / multi_start_nan_accounting_overhead.py. - Transect A: einstein_radius in [0,3] (n=601), per arm logl/dlogl/penalty/ dpenalty, plus arm-independent max_sep(theta_E) + argmax-pair geometry. - Transect B: 1e-4-spacing fine windows around each arm's threshold crossing -- the one-sided gradient jump (C0 hinge), an interior-plateau check windowed inside each arm's OWN crossings (not a fixed range for every arm -- the tighter auto threshold's crossings sit right at a naive [1.5,1.7] window's edge), and a coarse+fine argmax-switch scan. - Transect C: mass ell_comps_0 / shear gamma_1 sweeps at fixed truth theta_E, plus a full 15-dim gradient-norm ratio (penalty vs likelihood) at 5 points, one representative arm per sweep. Two real bugs found and fixed during development (both would have silently zeroed every "on" arm's numbers): - jax.jit traces LAZILY on first call, not at wrap time: the original get_scalar_fns/get_full_grad_fns cache helpers set then immediately reset analysis.positions_likelihood_list around building the jit object, so the eventual first (real) call always traced with positions off. Fixed by setting it unconditionally right before every return, cache hit or miss. - Transect B derived the hinge probe location from the COARSE grid's interpolated crossing, imprecise enough (spacing ~0.075) that both probed fine-grid neighbours could land on the same (interior) side, silently reporting a zero jump. Fixed by re-locating the crossing from the fine grid's own precise max_sep values. A third issue is a memory/compute lesson, not a correctness bug: a single unchunked jax.vmap over Transect A's full n=601 grid OOMs (~30 GB requested) -- the real MGE+inversion likelihood over 15361 masked pixels. Added chunked_call, batching every vmap'd evaluation at 64 (matching vram.vmap_batch_for("imaging","mge","hst")'s A100-probed cap for this exact cell) with fixed padding, which also collapses compile count to one XLA compile per (idx, arm) regardless of how many distinct grid shapes Transects A/B/C throw at it. results/notes/inference/phase_04_positions/{RESULTS.md,transects/*} record the write-up and full numeric arrays from both a --quick (n=41) validation run and the full (n=601) run; the two runs' crossing locations and qualitative behaviour agree. README.md documents the SEARCHES_POSITIONS* env vars and this script's usage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
…list + log_det_method) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 4 Stage 1 of the inference programme (PROGRAMME §4 Phase 4). Closes #159. Unblocks Gate B pt 2 / W2 (#160) and Phase 5.
SEARCHES_POSITIONS=off|on,SEARCHES_POSITIONS_THRESHOLD=<float>|auto,SEARCHES_POSITIONS_FACTOR=<float>threadal.PositionsLHintoAnalysisImaging/AnalysisInterferometerfor single-plane imaging/interferometer cells; group/cluster/datacube/point_source raise when on. The pixelizedraise_inversion_positions_likelihood_exceptionshort-circuit now applies only with positions off.[search, model, unique_tag]only (theAnalysisis not hashed), so a positions arm tag (pos_t0.3_f1e8,pos_tauto0.2_f1e5) is composed intounique_tagfor every sampler builder (newarm_unique_tag) and intoconfig_name. Found + fixed a real collision on the way:resolve_output_pathsused the raw CLI name, so on/off pairs overwrote the same results JSON. 16 tests incl.assert_disjoint_output_paths.dataset/imaging/hst/had nopositions.json; derived once from the committedtracer.jsonwith the simulator's ownPointSolverrecipe (re-simulating would changedata.fits) and committed (4 quad positions). Idealisation recorded (target_class: 3).positions_transects.py: θ_E ∈ [0,3] × {0.3, auto} × {1e5, 1e8} + off; fine sweep across the threshold;ell_comps/shear sweeps. Results + PNG/JSON underresults/notes/inference/phase_04_positions/.likelihood/positions_penaltycheck withthreshold-hinge,interior-plateau,argmax-switchfindings.Key findings (RESULTS.md)
autothreshold =max(3 × 7.7e-4, 0.2)→ collapses to the 0.2 floor on truth positions; the "auto" arm is "0.2 fixed" here.factor.ell_comps_0 = 0(unrelated to positions).Verification
pytest scripts/misc/test: 108 passed, 1 failed —test_hazards_prior_exit.py::test__records_the_clipper_as_what_blocks_itpasses onmain; it fails in a task worktree only becausehazards/_anchor.pylooks for sibling repos under the worktree root (PyAutoLabs-wt/<task>/PyAutoFitdoes not exist), socode_existsresolves False — a layout artifact, not a regression; CI checks out the library mains as siblings.ruff check scriptsclean. On/off smoke pair onmulti_start_prodigy_autoconv/mgeconfirmed distinctoutput/dirs and results JSONs.build_readme --checkclean. Two OOM/laziness bugs in the transect harness found and fixed during the run (chunkedvmapat 64; jit trace ordering).Note: one merge commit (69bd8d2) was recorded with conflict markers by mistake and resolved in the immediately following commit — history left intact rather than rewritten.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EDABYoH6giHXhFJUks8yd6