ReviewOS

zig-utils/zig-js

Frontend: borrow and batch-decode template quasis

#537
Closed chris opened this 9 days ago ·
9 days ago

Parent: #496 Related: #493, #473, #460, #461, #536

Problem

Parser.parseTemplate and Parser.parseTaggedTemplate append every ordinary byte of every cooked template quasi into arena-owned ArrayList storage. Untagged templates then duplicate the completed buffer again in concatStr. Wide generated templates, bundles, JSX-style transpiler output, and tagged DSL inputs therefore pay avoidable per-byte growth, copying, and retained parse-arena bytes even when a quasi contains no escape and is already a stable slice of the Context-owned source.

Scope

  • Add parser-only growth rows for 1K/2K/4K ordinary untagged templates, plus escaped and tagged 4K controls with frozen source construction, structural checks, warmup, and timed parse boundary.
  • Borrow cooked untagged and tagged quasi slices when no normalization or escape decoding is required.
  • On the first escape, materialize owned cooked storage and append ordinary spans in batches.
  • Preserve raw tagged-template slices, one-more-quasi-than-expression shape, substitution parsing context, CR/CRLF normalization, line continuations, WTF-8/lone-surrogate behavior, invalid tagged escapes (cooked === undefined), early errors for invalid untagged escapes, and exact source lifetime.

Security and correctness

  • No borrowed slice may outlive the Context-owned source or normalized arena backing.
  • Escaped decoding and normalization must remain OOM-propagating and failure-atomic.
  • Attacker-width template text must not create hidden quadratic copying or unbounded global caches.
  • Do not special-case benchmark text, skip tagged/raw semantics, weaken escape validation, or defer syntax errors.

Acceptance

  • Benchmark source and structural checks land before parser changes.
  • AST pointer/lifetime witnesses prove ordinary cooked quasis borrow stable backing; escaped/normalized quasis own exact decoded bytes.
  • 1K/2K/4K ordinary rows show bounded near-linear work and materially lower allocation/instruction cost versus the exact parent; escaped/tagged controls remain semantically exact.
  • Lexer/parser/frontend suites, untagged/tagged/invalid-escape/line-normalization tests, OOM diagnostics, forced execution paths, full units, and affected test262 template-literal subtrees pass with no regressions.
9 days ago

Benchmark-first commit landed on main as 09b525bc: bench(frontend): add template quasi growth rows (#537).

The parser-only runner now freezes 1,024/2,048/4,096 ordinary untagged-template programs plus escaped and tagged 4,096 controls. Source construction is outside warmup and timing. Each parse validates the top-level declaration and array width; untagged rows require exact StringLiteral nodes, while tagged rows require one cooked quasi, one raw quasi, and zero expressions before summing every decoded byte.

Frozen one-job checksums:

  • ordinary 1,024: 50,090
  • ordinary 2,048: 101,290
  • ordinary 4,096: 203,690
  • escaped 4,096: 203,690
  • tagged 4,096: 203,690

No parser behavior or performance claim is included in this commit. The implementation will be compared against this exact parent.

9 days ago

Completed by 718cb122: perf(frontend): borrow plain template quasis (#537).

The parser now retains exact source-backed cooked slices for ordinary untagged and tagged quasis, switches to arena-owned storage only at the first escape, appends ordinary spans in batches, and avoids the former second copy while constructing the untagged concatenation AST. CR/CRLF normalization remains arena-backed; tagged raw slices, invalid-escape undefined, substitution parsing context, validation, and OOM propagation are unchanged.

Coverage proves:

  • plain untagged and both plain tagged quasis have exact source-pointer identity;
  • escaped cooked text owns distinct decoded bytes while raw remains source-backed;
  • normalized CRLF cooked/raw text shares normalized arena backing, not original source;
  • tagged cooked/raw/expression cardinality remains 2/2/1 for a one-substitution template;
  • deterministic allocation failures propagate OutOfMemory and the same parse eventually succeeds.

Seven order-balanced exact-parent diagnostic pairs, 100 parses per sample, identical frozen checksums:

  • ordinary 1,024: 700,939,124 -> 571,631,034 retired instructions (0.816x); median peak RSS 3,063,808 -> 2,916,352 bytes (0.952x);
  • ordinary 2,048: 1,355,145,003 -> 1,087,574,171 instructions (0.803x); RSS 3,801,088 -> 3,440,640 (0.905x);
  • ordinary 4,096: 2,802,784,279 -> 2,241,895,402 instructions (0.800x); RSS 5,832,704 -> 5,079,040 (0.871x);
  • escaped 4,096: 3,086,177,163 -> 2,737,183,599 instructions (0.887x), same 20,369,000 checksum;
  • tagged 4,096: 4,205,480,424 -> 3,885,876,827 instructions (0.924x), same checksum.

The ordinary candidate remains near-linear: normalized instructions per literal at 4,096 are 0.980x the 1,024 row. Wall rows are diagnostic and not claimed where noise exceeded the repository threshold.

Validation:

  • focused template units: 5/5
  • parser units: 30/30
  • focused frontend: 12/12
  • parser TSan: 30/30
  • full sharded units: 1,596 passed, 1 expected skip, 0 failed/leaked
  • exact-parent test262 template-literal and tagged-template: 84 -> 84; flips 0

The benchmark-first requirement is 09b525bc. All acceptance items are satisfied.

Sign in to comment on this issue.