Skip to content

Reuse ordinary Lua function artifacts after fresh compile-time execution - #1585

Merged
Frotty merged 2 commits into
masterfrom
codex/incremental-symbol-bodies
Oct 27, 2026
Merged

Frotty merged 2 commits into
masterfrom
codex/incremental-symbol-bodies

Conversation

@Frotty

@Frotty Frotty commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Ordinary map function edits previously rebuilt the entire Lua backend even when retained frontend IM was available. This adds a post-compile-time artifact path for same-shape primitive function edits with ordinary calls, globals/arrays, real/boolean control flow and preserved names. Fresh compile-time execution and object injection still run; changed owners alone are materialised, lowered and emitted, and unchanged callers keep their bindings.

Generated NodeGraphs/NodeCodec contracts own body snapshots and symbolic operands. Cold admission requires equality with both batch-lowered IM and the batch-emitted definition. Request-local semantic receipts guard compiler facts, non-admitted bodies and admitted shapes; frontend rebuilds invalidate the snapshot. Entry/init/replay products retain batch manifest ownership. Unsupported structural, generic/class, vararg, helper-dependent arithmetic, stack-trace and optimisation cases remain explicitly observable batch paths.

Also repairs Flatten's initializer metadata to follow the replacement assignment, marks IM source fields in the schema, tracks owned function flags, and distinguishes compile-time/runtime lowering in work receipts. No emitted runtime representation or language semantics change.

Validation:

  • 232 focused integration tests passed (symbol/scalar backend, retained IM, model reuse, deterministic compilation and compile-time tests).
  • Final symbol and metadata tests: 16 passed with generated ownership checks enabled, including 1,000 unrelated live functions, unchanged callers applying a callee payload in stock Lua, multiple edits, no-ops, annotation/name bindings, real/array bodies, fresh compile-time effects and cancellation/retry. Initial caller, initializer ownership and flag-tracking repros failed before their fixes.
  • Actual existing-function probes in Castle Fight and Zombie Defense already match independent fresh Lua/manifests and publish one-function payloads with exactly one runtime function lowered/emitted. All twelve final A/B trials passed. The full 24-commit replay has completed: every retained build matches independent fresh Lua and manifests. Payload outcomes remain unchanged: CF 8 accepted / 4 restart, ZD 12 accepted. Of the accepted commits, CF 5 and ZD 10 have state/init limitations; only CF 3 and ZD 2 are warning-free offline candidates. This compiler PR does not improve runtime commit-acceptance coverage.
  • Windows x64 / Java 27 locally; Ubuntu full suite via PR CI. macOS and Warcraft client application remain unverified.

This remains an incremental rollout: fresh compile-time work still copies the whole IM graph, semantic validation scans global state, and script assembly/ABI hashing/payload comparison process full output. Cold snapshot admission adds startup work. The docs and backlog record these remaining costs and producer migrations; this PR does not claim a fully incremental compiler or subsecond reloads.

Controlled warm-edit timing: Windows x64, Java 27, 6 GiB heap, ParallelGC. Three separate JVM trials per map/variant in baseline/symbol/symbol/baseline/baseline/symbol order, exact previous master 1c23fb384c0f6c96da55218a996703378014ee51 versus reviewed head ef9e5cb3d87d37920f22aa37290c3c6139ccad19; identical current harness, dependencies, exported fixture and flags.

Existing map function edit Previous median, reconcile to payload New median Reduction Compilation alone, old → new
Castle Fight sync threshold 9.71 s 6.37 s 34.4% 7.44 → 4.33 s
Zombie Defense rock probability 5.82 s 4.89 s 15.9% 3.99 → 3.05 s

Total ranges: Castle Fight old 9.29–9.77 s / new 6.35–6.44 s; Zombie Defense old 5.39–6.06 s / new 4.66–6.32 s. Zombie Defense ranges overlap and one new trial was slower than the old median. Three trials show a local median gain, not a p95 guarantee. Editor queueing and game pickup/application are excluded. All twelve trials matched independent fresh Lua and full manifests and produced one-function payloads. New backend runtime lowering/emission: 1/1 functions; baseline Castle Fight: 36,757/35,723; baseline Zombie Defense: 30,624/29,761. Each new request also lowered one compile-time owner freshly. Reproduce with the checked-in SymbolHotReloadProfile driver and edit specs; full local receipts: de.peeeq.wurstscript/test-output/symbol-final-benchmarks.json.

Cold admission adds work: independent fresh-model compilation oracles took median 8.11 → 14.08 s on Castle Fight and 4.94 → 9.29 s on Zombie Defense. These are compilation-stage observations after model loading, not end-to-end startup measurements. Warm results above use completed admission snapshots. Full private IM copying remains 60,018 functions / 2,339,305 nodes on Castle Fight and 54,551 / 1,519,344 on Zombie Defense. Removing that compile-time bridge and global receipt scans remains necessary for the subsecond target.

Runtime support assessment (separate from compiler speed): all four CF rejections involve closure lineage/identity ambiguity or reuse of closure storage that existing objects may still read. Partial updates commonly retain removed callbacks, skip changed main/package initialization, preserve old initial values, or cannot reconstruct newly added captures on already-created closures. Warning-free receipt is not an in-game whole-commit proof, and the sample includes test-only changes. This is not yet broad everyday-commit coverage. Next coverage work should minimize actual callback/initialization commits, distinguish compiler identity churn from genuine migration requirements, and prove changed behavior in stock Lua and the game rather than relaxing rejection guards. The history campaign used the full backend for all 24 pairs; the speedup claim comes from the separately identified existing-function body probes.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T08:56:04.103954Z a8a1459 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Frotty

Frotty commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: a8a1459d75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Frotty
Frotty merged commit 168b9a9 into master Oct 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant