Harden and optimize RegExp execution#543
Open
zzjjob wants to merge 4 commits into
Open
Conversation
Validate compiled RegExp programs before execution and reject malformed opcode streams, register access, jump targets, metadata, and forged atom flags. Verify RegExp constants while reading serialized function bytecode, rebuild trusted programs from their source, and bump BC_VERSION for the changed format. Emit versioned optimization metadata for decoded literals, required prefixes, and multi-position quick checks. Use direct string searches for literal expressions, prefilter generic expressions before entering the backtracking interpreter, execute verified candidate offsets with lre_exec_at(), and avoid heap allocation for common capture vectors. Add C coverage for the validator and metadata derivation, mutate QBC fixtures to exercise malformed RegExp and function bytecode rejection, extend source/QBC built-in tests, and add focused microbench cases for escaped literals, misses, prefixes, and quick checks. Verification: make test Benchmark on macOS arm64: the complete microbench total decreases by 35.79% for source and 35.60% for QBC; the 82-case geometric mean improves 1.175x and 1.168x, while the RegExp subset improves 2.765x and 2.792x. Non-RegExp results are broadly flat, with a reproducible 3-4.5% function-call microbenchmark regression retained for follow-up.
Move the standard unanchored search loop out of the backtracking bytecode so failed candidates can reuse the interpreter stack without repeatedly constructing the synthetic split/any/goto states. Restore capture slots when a top-level candidate fails, increase the inline backtracking stack from 32 to 128 entries to avoid repeated reallocations, and recognize safe (^|X)literal prefixes. The prefix filter scans for up to four required literal characters while preserving the original interpreter path for ignore-case, multiline, sticky, and Unicode modes. Add regression coverage for captures, lookarounds, Unicode advancement, global replacement, sticky matching, bounded leading alternations, wide strings, escaped candidates, and fallback flag combinations. Benchmark: ahaoboy/js-engine-benchmark RegExp median improves from 862 to 1771 (+105.5%); the re17 hotspot drops from about 325 ms to 17 ms, and the full-suite score improves from 1938 to 2153 (+11.1%). Verified with make test, source and checked-QBC execution, the RegExp bytecode validator, ASan, and UBSan.
Make the RegExp fast paths introduced by the previous optimization commits safe for production use. Refresh the borrowed metadata view whenever compiled RegExp data is replaced, so RegExp.prototype.compile() cannot retain metadata owned by freed bytecode. Route all construction and recompilation paths through the same installer. Preserve interrupt semantics in atom, prefix, quick-check, and leading-character scans by polling at bounded intervals and propagating timeouts through exec and global replace. Replace runtime opcode-shape probing with a compiler-produced metadata v3 execution descriptor, bound duplicated payloads, validate descriptor fields, and reject stale or malformed serialized bytecode. Keep interpreter and capture stacks configurable for constrained arm32/arm64 targets, while retaining measured defaults. Centralize optimized exec/replace dispatch without adding shared mutable state. Add compile transition and GC stress coverage, metadata differential and mutation tests, checked-QBC rejection cases, embedding-level interrupt tests, and independent-runtime pthread stress tests. Wire the new tests into make test and bump the QuickJS bytecode version for the metadata format change. Verified with make test, Werror, ASan, UBSan, TSan, macOS leak checks, relevant Test262 coverage, and Android/iOS arm compilation. Benchmark medians remain within the release thresholds while long-literal QBC size and RSS are reduced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #541. This branch starts at the exact head commit of #541 (
a2c571acad9a4079fc0e29aacc4da7bbef94d4ff). Once #541 is merged, this PR reduces to three RegExp-only commits and 11 changed files. No Date optimization is included.Summary
RegExp.prototype.compile()transitionDesign and compatibility
Optimization metadata is treated as untrusted when it comes from serialized bytecode. Valid source-compiled and checked-QBC patterns use the same validated descriptor; patterns that cannot be proven eligible continue through
lre_exec(). The implementation adds no JIT, executable memory, architecture-specific assembly, or shared mutable global state.The binary-object version is bumped because the metadata layout changes. Old or malformed QBC is rejected cleanly instead of being interpreted with incompatible optimization state.
Performance
Fresh seven-run medians on macOS 15.7.4 arm64, Apple clang 17.0.0,
-O2. The before build is the exact #541 head and the after build is this PR head. Times are ns/op; lower is better.regexp_asciiregexp_utf16regexp_replaceThe prefix and quick-check tiers were also measured against otherwise identical builds with their metadata emission disabled:
ahaoboy/js-engine-benchmark
Benchmark commit:
81415304391884c41601d8e7fd4ebf1cbc6385e6. Scores are higher-is-better.The non-RegExp subtests remained broadly flat; the full-score increase is driven by the RegExp workload.
Serialized metadata size
A fixture with 1,000 raw literals of 256 code units measured:
Correctness and hardening
RegExp.prototype.compile()transition matrix covers metadata-to-generic and metadata-to-metadata replacements, flags,lastIndex, captures, indices, replacement, failures, and GC stressVerification
make teston the exact official stacked branchThis PR contains the RegExp changes derived from
a6088bb2ad78365e84fd6955dc6fe14d523925cd,63df7cc81b5136455971d9705aea94553c4d6fc4, andc2ca48b03498d3cf28b21d7a32a5d1b856b1dec0.