Repository navigation
fix: Result.instance falls back to a valid sample; persist samples before materialisation #1535
Description
Overview
Six of seven finished Phase 8B MultiStartProdigy arms on the RAL A100 (autolens_profiling jobs 341874/341875, 1.8–4.0 h each) completed all 3000 steps and then died in analysis.save_results with SamplesException ← ModelParameterException: ell_comps must satisfy e0**2+e1**2 < 1. The best lane sits outside the ellipticity unit disk (box prior; validate_ell_comps is a no-op on JAX tracers), and the rejection surfaces only on host materialisation. Because updater._save_samples returns early on the exception, the run also loses samples.csv and .completed — the mechanism #1487 suspects. Companion PyAutoLens change widens the save_results catch.
Plan
Result.instance: onSamplesExceptionfrom the summary path, fall back toSamples.max_log_likelihood()(the recovering path added by fix(autofit): reconstructing a stored sample raises through ignore_assertions=True #1486); WARNING log.updater._save_samples: writesamples.csv/ summary before instance materialisation so a rejected best point never costs a run its data (also lets the weight-threshold prune run — Weight-threshold prune does not remove zero-weight samples with checks enabled #1487).- PyAutoLens
save_results/save_results_combined: catchaf.exc.SamplesExceptionas well asAttributeError. - Tests for both behaviours; no change to
validate_ell_compstracer behaviour (the physics gap is a separate follow-up). - After merge:
HPCPullPyAutoon RAL so the pending 8B arms write normally.
Detailed implementation plan
Work Classification
Library (PyAutoFit primary, PyAutoLens companion) — library-first gate.
Affected Repositories
- PyAutoFit (primary)
- PyAutoLens
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./PyAutoFit | main | clean |
| ./PyAutoLens | main | clean |
Suggested branch: feature/result-instance-fallback
Worktree root: ~/Code/PyAutoLabs-wt/result-instance-fallback/
Implementation Steps
autofit/non_linear/result.py:116-121—Result.instance:try: self.samples_summary.instance except exc.SamplesException: log + return self.samples.max_log_likelihood().autofit/non_linear/search/updater.py:211-224— reorder:save_samples(and a summary without the instance) before thesamples_summary.instanceattempt; keep the exception handling for the instance-dependent outputs only.autofit/non_linear/samples/summary.py:58— leave the raising policy (single sample); document why.autolens/analysis/analysis/dataset.py:186-191(+save_results_combined) —except (AttributeError, af.exc.SamplesException).- Tests: PyAutoFit — model whose
instance_from_vectorrejects the stored best vector →Result.instancereturns best valid sample;samples.csvexists;.completedwritten via a stub search. PyAutoLens —save_resultssurvives a raising result.
Key Files
autofit/non_linear/result.py,autofit/non_linear/search/updater.py,autofit/non_linear/samples/{summary,interface,samples}.pyautolens/analysis/analysis/dataset.py- Root-cause evidence: RAL logs
error.341875_*.err; audit https://claude.ai/code/artifact/d9f4b0f3-52a1-4830-a9ad-11a225b77507
Original Prompt
Click to expand starting prompt
@PyAutoFit Bug: a rejected best point kills the run at results-write and loses samples.csv
Type: bug
Target: autofit
Difficulty: small
Autonomy: supervised
Priority: high
Status: formalised
Issued: 2026-08-27
Original request (verbatim): "Look at the JAX profiling gradinet epic, download results
overnight from A100, and do a major assessment of the results so far are, maybe double
checking some conclusiojns." → audit → "yep do all that" (ranked action 1).
Problem
Six of seven finished Phase 8B MultiStartProdigy arms on the RAL A100 (autolens_profiling
jobs 341874/341875, 1.8–4.0 h each) completed 3000/3000 steps and then died inside
analysis.save_results with
autofit.exc.SamplesException: The stored parameters returned by max_log_likelihood
cannot be reconstructed as a model instance because the current model rejected them
← autogalaxy.exc.ModelParameterException: ell_comps must satisfy e0**2+e1**2 < 1
The best lane sits outside the ellipticity unit disk (the ell_comps prior is an
independent TruncatedGaussian(-1,1) box; validate_ell_comps is a no-op on JAX tracers,
so the jitted objective is finite there). The rejection only surfaces on host
materialisation, and then:
@PyAutoFit/autofit/non_linear/result.py:116-121Result.instancegoes through
SamplesSummary.max_log_likelihood(samples/summary.py:58→samples/interface.py:122,
raising policy). The summary holds ONE sample so it cannot "next_valid"; the recovery
added by fix(autofit): reconstructing a stored sample raises through ignore_assertions=True #1486 lives only onSamples.max_log_likelihood(samples/samples.py:378-413)
and never applies to the path that runs. (This is the open Weight-threshold prune does not remove zero-weight samples with checks enabled #1487 suspect.)@PyAutoFit/autofit/non_linear/search/updater.py:211-224catches the exception and
returns early —save_samples_summaryandsave_samplesare SKIPPED, so the run has
nosamples.csv, no.completed, and the only surviving artifact is
search_internal.dill(which the crash coincidentally spares from deletion).@PyAutoLens/autolens/analysis/analysis/dataset.py:186-191save_resultscatches
onlyAttributeError, so the exception escapesstart_resume_fit
(abstract_search.py:777) beforepaths.completed()at:781.
Fix (library-first; this prompt covers PyAutoFit + the PyAutoLens catch)
Result.instance: try the summary; onSamplesExceptionfall back to
self.samples.max_log_likelihood()(the recovering path). Log at WARNING.updater._save_samples: persistsamples(and a summary marked invalid / without
instance) BEFORE instance materialisation, so a rejected best point never costs the
run its data. This also removes the early return that Weight-threshold prune does not remove zero-weight samples with checks enabled #1487 identifies as the reason
the weight-threshold prune never runs.- PyAutoLens
save_results/save_results_combined: also catch
af.exc.SamplesException(belt-and-braces; never let a tracer/plot failure kill a fit). - Tests: a model whose stored best vector is rejected by
instance_from_vector→
Result.instancereturns the best VALID sample;samples.csvis written; no raise. - Do NOT make
validate_ell_compsfire on tracers. The physics gap (joint disk
constraint / reparameterisation) is a separate Phase-3 follow-up prompt.
After merge: HPCPullPyAuto on RAL so the ~22 pending 8B arms write normally.
Audit: https://claude.ai/code/artifact/d9f4b0f3-52a1-4830-a9ad-11a225b77507
Activity
Jammy2211 commentedon Aug 27, 2026
Library PRs Created — 2026-08-27
Result.instancefallback + samples written before materialisation; also closes the Weight-threshold prune does not remove zero-weight samples with checks enabled #1487 early-return)save_resultscatch)Shipped under the human-authorised Heart RED override for
release validation FAILED (stage integrate)(nightly 33073386315, unrelated scripts). Companion results PR: PyAutoLabs/autolens_profiling#182.API changes: none removed/renamed;
Result.instanceno longer raises on a rejected best point (WARNING + next-valid sample);_save_samplesalways writessamples.csv/samples_summary.json.Follow-up (not in this PR): PyAutoGalaxy
autogalaxy/analysis/analysis/dataset.py:178has the identicalexcept AttributeError-only catch (shadowed for lens fits).Next: merge (human) →
HPCPullPyAutoon RAL so pending Phase 8B arms 341874/341875 write normally.Merge pull request #1536 from PyAutoLabs/feature/result-instance-fall…
Jammy2211 commentedon Aug 28, 2026
Shipped — 2026-08-27
PRs
Summary
Merged after green CI on every leg (library-first). Human-authorised Heart RED override for
release validation FAILED (stage integrate)(unrelated nightly failures). Companion issue: PyAutoLabs/autolens_profiling#182.Session Notes
Audit: https://claude.ai/code/artifact/d9f4b0f3-52a1-4830-a9ad-11a225b77507. RAL mirror pulled after merge. Follow-ups: PyAutoGalaxy dataset.py:178 same unguarded catch; F2 reference ruling before any 8B verdict; ell_comps joint disk constraint (Mind draft feature/autogalaxy/ell_comps_joint_disk_constraint.md).