Skip to content

results: A100 A/B of the compacted ConstantSplit assembly (PyAutoArray#537, 12x) - #231

Merged
Jammy2211 merged 3 commits into
mainfrom
feature/delaunay-nn-constant-split-assembly
Sep 8, 2026
Merged

Jammy2211 merged 3 commits into
mainfrom
feature/delaunay-nn-constant-split-assembly

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

The autolens_profiling half of the DelaunayNN split-regularization assembly
compaction (PyAutoLabs/PyAutoArray#536): the bench that priced every candidate
assembly, the real HST stencil tables it runs on, the A100 A/B submits, and the
results note that records the verdict.

The bench is what chose the design rather than confirming it. On real HST tables
it priced the alternatives — dense GEMM 10.1 ms (a wash with today's scatter),
BCOO 21 ms, dedup/segment-sum 39 ms — and swept the compact width (12: 0.58 ms,
16: 1.47, 20: 2.78, 24: 4.53, 28: 6.71, 32: 9.31, 33 uncompacted: 10.03), which
is where SPLIT_REG_COMPACT_WIDTH = 12 comes from.

A100 A/B (jobs 342334-342338, same node, same window)

measurement before after speedup
assembly row (ms/call) 10.03 0.84 12.0x
params -> H prefix, vmap 16 (ms/call) 16.423 7.260 2.26x
params -> H prefix, unbatched (ms) 24.341 15.189 1.60x
full likelihood, single JIT (ms) 75.6 66.1 1.14x
full likelihood, vmap 16 (ms/call) 50.04 40.86 1.22x

Pinned likelihood 29144.581944 bit-identical on both legs, pinned_drift: [].
The pre-registered witness (assembly < 3 ms, params -> H < 11 ms) is met.

Scripts Changed

  • scripts/misc/delaunay_nn/assembly_bench.py — new: V0..V6 assembly variants
    (uncompacted scatter, compact scatter + wide-row supplement, dense GEMM, BCOO,
    dedup/segment-sum), compact-width sweep, run against real tables.
  • scripts/misc/delaunay_nn/build_tables.py — new: rebuilds the frozen stencil
    tables the bench consumes.
  • scripts/misc/delaunay_nn/extract_mesh_points.py — new: extracts the HST mesh
    points the tables are built from.
  • scripts/misc/delaunay_nn/README.md — new: how to rebuild and re-run.
  • hpc/batch_gpu/submit_delaunay_nn_assembly_bench_a100 and the four A/B
    breakdown/likelihood submits — new A100 submits (assembly + control legs).
  • results/delaunay_nn/{tables_hst_1500.npz, mesh_points_hst.npy, assembly_bench_a100_fp64.json} — frozen inputs and bench results.
  • results/{breakdown,jit}/...delaunay_nn_hpc_a100_fp64_assembly{,_control}.{json,png}
    — the A/B results.
  • results/notes/delaunay_nn_constant_split_assembly.md — the results note:
    design, the priced alternatives, the width sweep, the A/B verdict, the cap
    audit that sizes the wide-row budget.

Results and notes only apart from the new bench scripts; no existing profiling
script or result is modified.

Upstream PR

PyAutoLabs/PyAutoArray#537

Library-first gate: merge after PyAutoLabs/PyAutoArray#537. The bench's V1
variant mirrors the compacted library path, and the note's verdict describes
behaviour that only exists once the library PR lands.

Test Plan

  • ruff check scripts/misc/delaunay_nn/ — all checks passed.
  • ruff format --check scripts/misc/delaunay_nn/ — 4 files already
    formatted.
  • assembly_bench.py executed on the A100 (job 342338) and on CPU; results
    committed under results/delaunay_nn/.
  • A/B breakdown + likelihood submits ran on the same A100 node and window
    (jobs 342334-342338), pinned likelihood bit-identical.

Links

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw

Jammy2211 and others added 3 commits September 8, 2026 09:01
…les, rebuild scripts (PyAutoArray#536)

`assembly_bench.py` times alternative formulations of the DelaunayNN
`ConstantSplit` regularization-matrix assembly (`reg_split_from` +
`pixel_splitted_regularization_matrix_from`) on the real (4P, K) split stencil
tables from the HST cell, unbatched and at vmap 16, fp64. `V0` is the pre-#536
full padded (4P, K, K) scatter and the reference every variant is checked
against; `V6_library` is the shipped library path with its default compaction,
so the bench measures what production actually runs.

`extract_mesh_points.py` dumps the HST DelaunayNN source-plane mesh points and
`build_tables.py` turns them into the committed
`results/delaunay_nn/{mesh_points_hst.npy,tables_hst_1500.npz}`, so the bench
runs anywhere without a GPU or the imaging dataset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
…tion (PyAutoArray#536)

Five submits, mirroring the #532/#533 launch-latency A/B: the breakdown and
runtime cells of imaging/delaunay_nn x hst x fp64 on the control (merge base
180c8a40) and feature (04152bb7) private PyAutoArray checkouts under
/mnt/ral/jnightin/PyAuto_wt/delaunay-nn-constant-split-assembly/, plus a direct
assembly bench that times the assembly function itself on the committed HST
split tables (V0 vs V6_library) with no prefix-attribution caveat.

Same flags (--split-setup --vmap-batch 16), same env, PYAUTO_SIBSON_QUERY_CHUNK
left unset on every leg, shared /mnt/ral/jnightin/PyAuto install untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
…y#536)

Five jobs (342334-342338) back-to-back on euclid-ral-gpu-2 inside one 8.4-minute
window, control 180c8a40 vs feature 04152bb7 on private PyAutoArray checkouts,
shared /mnt/ral/jnightin/PyAuto install untouched.

params->H prefix 24.34 -> 15.19 ms unbatched (1.60x) and 16.42 -> 7.26 ms per
call at vmap 16 (2.26x). The assembly measured directly on the real split
tables: 10.03 -> 0.84 ms per call at vmap 16 (12.0x), corroborated to 5% by the
breakdown's own H step row (10.05 -> 0.80). Whole likelihood at vmap 16
50.04 -> 40.86 ms per call (1.22x). The 29144.581944 pin is bit-identical on
both legs, pinned_drift [] on both. Both halves of the witness are met.

The note records honestly that the --split-setup differenced "H, ConstantSplit
assembly" cell goes negative (-1.60 ms) on the feature leg once the interval is
smaller than the compile-boundary materialisation slack, and re-costs Phase B
against the new numbers: the assembly is now 11% of the params->H prefix and
Sibson 89%, reversing #533's ordering.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsh8KqgiHWTfPGgWEYMQw
@Jammy2211 Jammy2211 added the pending-release Queued for the next release label Sep 8, 2026
@Jammy2211
Jammy2211 merged commit c7c07ae into main Sep 8, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/delaunay-nn-constant-split-assembly branch September 8, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release Queued for the next release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant