Skip to content

Cache the transform output per file - #55

Merged
johanrd merged 2 commits into
mainfrom
perf-transform-cache
Aug 29, 2026
Merged

Cache the transform output per file#55
johanrd merged 2 commits into
mainfrom
perf-transform-cache

Conversation

@johanrd

@johanrd johanrd commented Aug 29, 2026

Copy link
Copy Markdown
Owner

The blanked passes of a .gts/.gjs file are a pure function of the file's content, the tsconfig (through Glint) and the environment switches (HVE_GLINT, HVE_TS_BACKEND, HVE_MAX_CONDITIONAL_BRANCHES), so they are now cached on disk next to the Glint result, under node_modules/.cache/html-validate-ember/transform/. On a hit the transformer replays the stored passes and their hook inputs (dynamic-content offsets, attribute injections, per-element disables): no content-tag parse, no Glint extraction, no Glimmer parse, no blanking. HVE_NO_CACHE=1 bypasses it like the Glint cache.

transformGlimmer is split: computeTemplates produces serialisable template entries; the generator yields from those entries whether they came from the cache or were just computed. Output is identical (both lanes: 291 + 1 expected fail).

Also fixes a latent invalidation gap: the plugin-source hash covered only lib/, but blank.ts/transform.ts live at the package root, so an edit to the blanker did not invalidate cached Glint results and would not have invalidated cached transform output. The hash now covers root sources plus lib/.

Same cross-file caveat as the Glint cache: the key is this file's content, so a change in an imported component's template does not invalidate a consumer's cached output until the consumer changes (or the plugin version does). That trade-off already existed for the resolver's results inside the Glint cache; this PR extends it to the blanked output. If that becomes a problem the key should include the resolved import set.

Measured (pnpm bench:compare, base without the large template): warm run −13 %, one cached file −18 %. On the console app (838 files) the warm run goes from 5.8 s (with #54) to the number in the follow-up comment.

Stacks cleanly on top of #54 (one import line touches the same spot).

Cowritten by Claude

The blanked passes of a `.gts`/`.gjs` file are a function of its content,
the tsconfig and the environment switches, so they are now cached on
disk next to the Glint result (`.../html-validate-ember/transform/`). On
a hit the transformer replays the passes: no content-tag parse, no Glint
extraction, no blanking. `transformGlimmer` now computes the serialisable
template entries in `computeTemplates` and yields from them either way.

The plugin-source hash that invalidates both caches covered only `lib/`;
it now covers the root sources (`blank`, `transform`, …) as well, so an
edit to the blanker invalidates cached output.

Cowritten by Claude
@johanrd johanrd added enhancement New feature or request run-bench Run the benchmark comparison on this pull request labels Aug 29, 2026
@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

🏎️ Benchmark Comparison

Benchmark Control (p50) Experiment (p50) Δ
🔴 extract small template 562.58 µs 626.33 µs +11.3%
🔴 extract medium template 2.36 ms 2.65 ms +12.5%
🔴 extract cross-file resolution 1.96 ms 3.09 ms +57.9%
cold run (cache off) 1456.79 ms 1468.07 ms +0.8%
🟡 warm run (all cached) 583.35 ms 528.01 ms -9.5%
🟡 one cached file 384.88 ms 347.40 ms -9.7%
no glint 773.51 ms 795.33 ms +2.8%

🟢 faster · 🔴 slower · 🟡 5–10 %, possibly noise · ⚪ within 5 %

Full output
clk: ~3.56 GHz
cpu: AMD EPYC 9V74 80-Core Processor
runtime: node 24.19.0 (x64-linux)

benchmark                    avg (min … max) p75 / p99    (min … top 1%)
-------------------------------------------- -------------------------------
extract small template        785.68 µs/iter 736.68 µs █▂                   
                       (471.18 µs … 6.03 ms)   3.40 ms ██                   
                     ( 25.32 kb …   3.65 mb) 278.81 kb ██▆▄▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁

extract medium template         2.85 ms/iter   2.89 ms ▅█                   
                        (1.98 ms … 11.93 ms)   8.43 ms ██▇                  
                     (167.18 kb …   4.18 mb) 749.82 kb ███▇▅▃▂▃▂▁▂▁▁▂▁▂▁▁▁▁▁

extract cross-file resolution   2.30 ms/iter   2.24 ms  █                   
                        (1.70 ms … 11.32 ms)   7.38 ms ▅█                   
                     (237.13 kb …   1.47 mb) 415.49 kb ██▆▃▄▂▂▁▂▂▁▂▁▁▁▁▁▁▁▁▁

whole process (min / p50 of 3 runs, 20 files)
  cold run (cache off)           1444 ms /   1457 ms
  warm run (all cached)           583 ms /    583 ms
  one cached file                 382 ms /    385 ms
  no glint                        770 ms /    774 ms
clk: ~3.54 GHz
cpu: AMD EPYC 9V74 80-Core Processor
runtime: node 24.19.0 (x64-linux)

benchmark                    avg (min … max) p75 / p99    (min … top 1%)
-------------------------------------------- -------------------------------
extract small template        893.04 µs/iter 812.70 µs ██                   
                       (499.39 µs … 6.81 ms)   3.92 ms ██▂                  
                     ( 14.38 kb …   3.26 mb) 285.26 kb ███▄▂▂▁▂▂▁▁▂▂▁▁▂▂▁▁▁▁

extract medium template         3.26 ms/iter   3.46 ms  █                   
                         (2.31 ms … 9.10 ms)   8.18 ms ██                   
                     (112.10 kb …   4.85 mb)   0.99 mb ███▇▃▃▃▂▃▃▂▂▂▂▁▂▁▁▁▁▁

extract cross-file resolution   3.55 ms/iter   3.70 ms  █                   
                         (2.63 ms … 8.90 ms)   7.27 ms  █▃                  
                     (276.56 kb …   2.31 mb)   1.28 mb ▅██▇▅▃▃▄▂▂▁▃▂▂▃▁▁▁▂▁▁

whole process (min / p50 of 3 runs, 20 files)
  cold run (cache off)           1460 ms /   1468 ms
  warm run (all cached)           527 ms /    528 ms
  one cached file                 345 ms /    347 ms
  no glint                        790 ms /    795 ms

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new caching layer introduces subtle invalidation and serialization behavior (including a backend-detection keying gap versus the sibling Glint cache) that warrants human verification, and the new cache functions lack the unit coverage the existing cache has.

Pull request overview

This PR adds a second on-disk cache layer to the html-validate-ember plugin: it stores the blanked <template> passes of a .gts/.gjs file (with their hook inputs) under node_modules/.cache/html-validate-ember/transform/, so repeat runs replay the stored passes and skip content-tag parsing, Glint extraction, Glimmer parsing, and blanking entirely. It fits alongside the existing Glint extraction cache and builds on top of the parse-memoisation work in #54.

Changes:

  • Split transformGlimmer into a pure computeTemplates (produces serialisable CachedTemplate[]) plus a generator that replays those entries whether they came from cache or were freshly computed.
  • Added transformCacheKey/readTransformCache/writeTransformCache and the CachedPass/CachedTemplate shapes in lib/cache.ts, keyed on file content + tsconfig SHA + env switches, bypassed by HVE_NO_CACHE=1.
  • Widened the plugin-source SHA to cover the package root sources (blank.ts/transform.ts) plus lib/, fixing a latent invalidation gap where blanker edits didn't invalidate cached results.
File summaries
File Description
transform.ts Splits the .gts/.gjs transform into computeTemplates + a cache-aware replay generator; wires in the transform cache.
lib/cache.ts Adds the transform-output cache (key, read, write, serializable types) and broadens the plugin-source SHA walk to the package root plus lib/.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/cache.ts
Comment thread lib/cache.ts Outdated
Comment thread transform.ts
The key now carries the backend `backendFor` would select (forced kind,
or the TypeScript 7 package name and version when the tsconfig declares
`contentMappers`), resolved without loading TypeScript. A stored glimmer
parse error is re-emitted on replay, so the diagnostic no longer depends
on cache state. Round-trip and key-mismatch tests.

Cowritten by Claude
@johanrd
johanrd merged commit a755505 into main Aug 29, 2026
4 checks passed
@github-actions github-actions Bot mentioned this pull request Aug 29, 2026
johanrd added a commit that referenced this pull request Aug 29, 2026
The key carries the backend kind from `backendKindFor` (as the transform
cache does since #55) instead of the raw `HVE_TS_BACKEND`. Round-trip,
miss-on-any-input-change and one-entry-per-path tests.

Cowritten by Claude
johanrd added a commit that referenced this pull request Aug 29, 2026
* CLI: replay the report of unchanged files

`validate-gts` caches each file's deduplicated html-validate report,
keyed by the file's content, the resolved configuration, html-validate's
version, the tsconfig and the environment switches (under
`.../html-validate-ember/report/`). An unchanged file is not validated
again; its report is replayed into the same counters and output.

Console app (838 files), warm run: 3.3 s -> 0.3 s. Output identical to
an uncached run; `HVE_NO_CACHE=1` bypasses it like the other caches.

Cowritten by Claude

* Report cache: key on the resolved backend, tests

The key carries the backend kind from `backendKindFor` (as the transform
cache does since #55) instead of the raw `HVE_TS_BACKEND`. Round-trip,
miss-on-any-input-change and one-entry-per-path tests.

Cowritten by Claude
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request run-bench Run the benchmark comparison on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants