ReviewOS

zig-utils/zig-js

Frontend: normalize template raw text with one exact allocation

#550
Closed chris opened this 9 days ago ·
9 days ago

Parent: #496 Related: #537, #549

Problem

normalizeTemplateRaw correctly maps CRLF and lone CR to LF, but the first CR switches the complete template token to byte-at-a-time ArrayListUnmanaged growth in the parser arena. Allocation request count and retained capacity therefore depend on the raw length even though the normalized output size is knowable before allocation. Both untagged cooked values and tagged template raw/cooked arrays consume this path.

Scope

  • Add 1K/2K/4K mixed CRLF/lone-CR template parser rows plus a tagged 4K control, validating every complete normalized value.
  • Count the exact normalized length before allocation, allocate once, and fill once.
  • Preserve zero-allocation source borrowing when no CR is present.
  • Preserve TRV rules, line continuations, substitution boundaries, tagged raw/cooked values, LS/PS bytes, invalid escapes, source lifetimes, and diagnostic behavior.

No-workaround rules

  • No worst-case allocation, hidden scratch buffer, source rewriting before the timed parse boundary, CR-only special-case that mishandles CRLF, reduced tagged-template validation, or deferred syntax errors.

Acceptance

  • Any raw template token requiring CR normalization uses one exact output allocation with no growth/transfer; CR-free templates remain source-backed.
  • Mixed CRLF/lone-CR, leading/trailing terminators, escaped line continuations, substitutions, tagged raw/cooked, LS/PS, and deterministic OOM paths have direct tests.
  • Benchmark validation independently checks every normalized element and records 1K/2K/4K allocation growth plus a tagged control.
  • Focused frontend, parser/lexer, TSan, full units, and exact template/tagged-template test262 comparisons pass.
  • Exact-parent evidence records wall, hardware counters, allocation requests/bytes, RSS, checksums, and dispersion without moving source construction or parsing outside the timed boundary.
9 days ago

Implemented, verified, and published on main:

normalizeTemplateRaw now sizes the TRV result before allocation, performs one exact allocation, and bulk-copies unchanged spans while mapping lone CR and CRLF to LF. CR-free input still borrows the immutable source slice. Direct tests prove exact requested bytes with no resize, zero-memory borrowing, deterministic OOM followed by successful retry, mixed leading/trailing terminators, line continuations, substitutions, tagged raw/cooked semantics, and LS/PS preservation.

Verification:

  • focused frontend Debug: 12/12
  • focused frontend TSan: 12/12
  • full unit suite: 1,619 passed, 1 skipped, 0 failed, 0 leaked
  • exact-parent test262: template literals 57 -> 57; tagged templates 27 -> 27

Across seven order-balanced exact-parent pairs, allocation requests remain exact while bytes fall 9.6%, 12.6%, and 10.6% at 1K/2K/4K. Stable retired instructions fall 7.9%, 8.1%, and 8.3%. The tagged 4K row reduces bytes 8.0% and instructions 5.7%. Stable 1K, 2K, and tagged wall rows improve 7.6-10.1%; the noisy 4K untagged wall median remains explicitly diagnostic.

Every acceptance item is complete. This advances parent #496.

Sign in to comment on this issue.