Skip to content

Add opt-in generator compile cache - #9266

Open
alexreinking wants to merge 7 commits into
mainfrom
alexreinking/generator-cache
Open

Add opt-in generator compile cache#9266
alexreinking wants to merge 7 commits into
mainfrom
alexreinking/generator-cache

Conversation

@alexreinking

@alexreinking alexreinking commented Jul 31, 2026

Copy link
Copy Markdown
Member

When HL_CACHE_DIR is set, execute_generator caches the artifacts a generator emits and restores them instead of recompiling when the inputs are unchanged. The cache key combines:

  • a compiler-identity fingerprint: the linker-assigned build id, read directly from the already-loaded libHalide image (Mach-O LC_UUID on macOS, ELF .note.gnu.build-id via dl_iterate_phdr on Linux), so it changes on any relink without hashing the ~32 MB binary; falls back to a full-binary hash when no build id is available (e.g. --build-id=none, or Windows, not yet implemented)
  • the target(s), output types, and generator-param settings
  • the serialized pipeline for each target, which is what makes the key sound: it captures the actual algorithm and schedule, so editing a generator's source invalidates the cache even when its command-line params are unchanged
  • the contents of any loaded -p plugins (e.g. autoschedulers), since they affect codegen but live outside libHalide

Both the generator (compile_multitarget) and standalone-runtime (compile_standalone_runtime) paths are cached. Entries are installed atomically (temp dir + rename) so concurrent builds never see a partial entry. The cache is pruned to honor HL_CACHE_MAX_SIZE (default 1 GiB) and optional HL_CACHE_MAX_AGE, debounced to at most once per 60s via a .last_prune stamp so a large parallel build's many generator processes don't each walk the whole cache; the size limit is thus a soft cap. The whole feature is gated on WITH_SERIALIZATION and is completely inert when HL_CACHE_DIR is unset.

The shipped CMake helpers gain a Halide_CACHE_DIR cache variable (defaulting to $ENV{HL_CACHE_DIR}) and wrap every generator/GenRT invocation so the value reaches the generator process. When it is empty the emitted commands are unchanged.

Measured on the app suite (~309 generator processes): using the build id instead of a full-binary hash halves the cache-hit generator time (~46s -> ~22s of summed edge time), taking the generator-phase speedup from ~2.2x to ~5.3x versus a full recompile.

Tests: test/correctness/generator_cache.cpp re-execs itself per generation and tampers a cached blob to prove artifacts are restored from the cache (a real hit only happens across separate processes, since Halide's global unique-name counter advances within a process). test/integration/cache verifies the shipped helpers route HL_CACHE_DIR to the generator and that a build sharing the cache still links and runs.

Breaking changes

None.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking
alexreinking requested a review from abadams July 31, 2026 21:53
@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 56.63265% with 170 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.90%. Comparing base (aa169dc) to head (01f97a4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/GeneratorCache.cpp 54.04% 107 Missing and 35 partials ⚠️
src/Generator.cpp 66.26% 19 Missing and 9 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9266      +/-   ##
==========================================
- Coverage   70.00%   69.90%   -0.10%     
==========================================
  Files         258      259       +1     
  Lines       77353    77743     +390     
  Branches    18837    18927      +90     
==========================================
+ Hits        54151    54347     +196     
- Misses      17626    17753     +127     
- Partials     5576     5643      +67     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking alexreinking added the release_notes For changes that may warrant a note in README for official releases. label Aug 5, 2026
alexreinking and others added 5 commits August 7, 2026 09:31
When HL_CACHE_DIR is set, execute_generator caches the artifacts a
generator emits and restores them instead of recompiling when the
inputs are unchanged. The cache key combines:

- a compiler-identity fingerprint: the linker-assigned build id, read
  directly from the already-loaded libHalide image (Mach-O LC_UUID on
  macOS, ELF .note.gnu.build-id via dl_iterate_phdr on Linux), so it
  changes on any relink without hashing the ~32 MB binary; falls back
  to a full-binary hash when no build id is available (e.g.
  --build-id=none, or Windows, not yet implemented)
- the target(s), output types, and generator-param settings
- the serialized pipeline for each target, which is what makes the
  key sound: it captures the actual algorithm and schedule, so editing
  a generator's source invalidates the cache even when its
  command-line params are unchanged
- the contents of any loaded -p plugins (e.g. autoschedulers), since
  they affect codegen but live outside libHalide

Both the generator (compile_multitarget) and standalone-runtime
(compile_standalone_runtime) paths are cached. Entries are installed
atomically (temp dir + rename) so concurrent builds never see a
partial entry. The cache is pruned to honor HL_CACHE_MAX_SIZE (default
1 GiB) and optional HL_CACHE_MAX_AGE, debounced to at most once per
60s via a .last_prune stamp so a large parallel build's many generator
processes don't each walk the whole cache; the size limit is thus a
soft cap. The whole feature is gated on WITH_SERIALIZATION and is
completely inert when HL_CACHE_DIR is unset.

The shipped CMake helpers gain a Halide_CACHE_DIR cache variable
(defaulting to $ENV{HL_CACHE_DIR}) and wrap every generator/GenRT
invocation so the value reaches the generator process. When it is
empty the emitted commands are unchanged.

Measured on the app suite (~309 generator processes): using the build
id instead of a full-binary hash halves the cache-hit generator time
(~46s -> ~22s of summed edge time), taking the generator-phase speedup
from ~2.2x to ~5.3x versus a full recompile.

Tests: test/correctness/generator_cache.cpp re-execs itself per
generation and tampers a cached blob to prove artifacts are restored
from the cache (a real hit only happens across separate processes,
since Halide's global unique-name counter advances within a process).
test/integration/cache verifies the shipped helpers route HL_CACHE_DIR
to the generator and that a build sharing the cache still links and
runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add doc/GeneratorCache.md covering the cache key composition, CLI and
CMake usage, and pruning/maintenance behavior, and link it from
README.md, HalideCMakePackage.md, and BuildingHalideWithCMake.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
get_output_info() names the compiled object cache_add.obj on Windows
(is_windows_coff) rather than cache_add.o, so the test's hardcoded .o
path never existed and every check after the first compile failed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Read the PE debug directory's CodeView (RSDS) entry -- the
linker-assigned PDB GUID + age -- directly from the already-loaded
image, mirroring the Mach-O LC_UUID and ELF .note.gnu.build-id paths
used on macOS and Linux. This lets Windows skip hashing the whole
binary on every generator invocation, same as the other platforms.
Falls back to a full-binary hash when no CodeView entry is present
(e.g. a release build with debug info fully stripped).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alexreinking
alexreinking force-pushed the alexreinking/generator-cache branch from 286006f to 598c5f0 Compare August 7, 2026 13:40

@abadams abadams left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend also caching JIT-compiled runtimes as a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release_notes For changes that may warrant a note in README for official releases.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants