ReviewOS

zig-utils/zig-js

Frontend: reclaim parse-only RegExp validation state

#560
Closed chris opened this 9 days ago ·
9 days ago

Parent: #496 Related: #473, #480, #493

Problem

Every RegExp literal is correctly compiled during parsing so invalid flags/patterns and zig-regex's security analysis remain early errors. The validation result is then discarded, but today it is allocated in the long-lived parser/Context arena and never deinitialized. Non-Unicode literals also call normalizeAnnexBClassRanges, which builds a complete growth-list copy even when no Annex B rewrite is needed and then returns the original slice.

Regex-heavy source therefore retains parse-only AST/NFA/optimizer/named-capture state until Context teardown and performs an avoidable full-pattern copy on the common unchanged path. Runtime evaluation compiles the literal again because the parser-side object is not an executable artifact.

Scope

  • Freeze parser-only 1K/2K/4K RegExp-literal growth rows before changing the implementation, with unchanged non-Unicode patterns plus Unicode and Annex B rewrite controls.
  • Make unchanged Annex B normalization allocation-free and changed normalization one exact bounded allocation.
  • Compile validation state through the parser's invocation-local scratch allocator and deinitialize every successful validation object before returning.
  • Preserve exact pattern/flag source slices in the AST; this issue does not introduce a compiled-artifact cache or move compilation outside the parse boundary.
  • Keep the runtime compile and security analyzer authoritative for the executable RegExp object.

No-workaround rules

  • No delayed or skipped syntax validation, disabled ReDoS/security analysis, benchmark/source recognition, hash-only cache key, permanent compiled-object leak, reduced pattern/flag coverage, or arena-reset trick that invalidates AST/source slices.
  • No change to JavaScript-visible RegExp identity or creation timing.

Acceptance

  • Benchmark source, 1K/2K/4K identities, controls, exact checksums, allocation accounting, warmup, and timed boundary land before the implementation.
  • Unchanged non-Unicode normalization performs zero allocation; rewritten Annex B input performs one exact allocation and releases it after compile.
  • Successful parse validation releases all zig-regex compile state through freeable scratch; every failure path remains leak-free and reports the same parse-phase error.
  • Valid/invalid flags, Unicode u/v, Annex B class ranges, escapes, named captures, lookarounds, backreferences, and security-rejected patterns retain exact semantics.
  • OOM/fault injection, bounded Context teardown, focused TSan/frontend tests, full units, and exact-parent RegExp/test262 comparisons pass with zero regressions.
  • Exact-parent evidence reports wall/instructions/allocation/RSS changes without hiding validation work or runtime compilation.

Security boundary

Pattern bytes and source size are attacker-controlled. Validation must remain eager, bounded by the existing zig-regex security policy, failure-atomic under OOM, and free of proportional retained parse-only state.

9 days ago

Benchmark-first boundary landed on main in 354c7171: bench(frontend): add RegExp validation rows (#560).

Frozen parser-only rows:

  • unchanged non-Unicode patterns at 1,024 / 2,048 / 4,096 literals;
  • a 4,096-literal u control where Annex B normalization is inapplicable;
  • a 4,096-literal Annex B class-range control that must be rewritten exactly.

Every parse validates the complete AST pattern/flag slices and index sequence. Exact checksums are 557482, 2164650, 8524714, 8471466, and 8463274. The current 4K base allocation replay records 536,269 requests and 96,011,248 cumulative bytes. This commit contains no runtime optimization; normalization and validation lifetime changes will be measured against this pushed parent.

9 days ago

The scratch-lifetime work exposed and fixed two real OOM rollback defects in the local regex dependency, tracked as zig-regex#20: e9ea8b5 covers normalized group names and predefined byte-class ranges; cfb24da makes nested/Unicode/class-set item, set, and AST-node transfers failure-atomic. Dependency suite: 610/610, zero leaks. The zig-js parse-validation sweep now passes all 101 injected allocation failures.

9 days ago

Completed on main:

  • 354c7171 adds exact-checksum RegExp literal benchmark rows.
  • 9932fd7e mirrors the production AST/scratch allocator topology.
  • 8dc1c182 validates literals in one per-parse resettable arena, resets after every literal, borrows unchanged patterns, and performs Annex B rewrites with one exact allocation.
  • zig-regex#20 fixed the OOM rollback defects exposed by the exhaustive validation path in e9ea8b5 and cfb24da.

Security/semantics remain intact: validation and ReDoS analysis are still eager, source-backed AST pattern/flag slices are unchanged, invalid syntax remains a parse error, and OOM propagates with failure-atomic ownership.

Exact-parent results (seven alternating pairs): 1K/2K/4K medians improve 10.3%/6.9%/7.8%; Unicode and Annex B 4K controls improve 5.0%/8.1%. At 4K ordinary literals, allocation requests fall 536,269 → 4,148 and cumulative bytes 96,011,248 → 2,549,128 with checksum 8,524,714 unchanged.

Validation: full units 1,636 pass / 1 skip / 0 fail / 0 leak; focused TSan 20/20; exact-parent test262 remains 1,879/1,879 built-ins/RegExp and 238/238 language/literals/regexp (0 flips, 0 regressions).

Sign in to comment on this issue.