ReviewOS

zig-utils/zig-js

Parser: resolve statement locations in source order

#610
Closed chris opened this 9 days ago ·
9 days ago

Parent: #496 Related: #554, #607, #608, #609

Problem

Parser.parseStatement resolved source coordinates after a complete statement parsed. Nested statements publish in completion order (inner before outer), so #554 bounded prefix rescans with a line-start index and a binary search per lookup. A current ReleaseFast profile resolved the dominant Unicode-identifier parser samples to this lookup: a large single-line function crossed the same attacker-controlled source prefix four times solely for debugger metadata.

Classic for cover-grammar parsing exposed a second correctness boundary: a discarded speculative function body could publish a ghost debugger location before the parser rewound to parse the real classic-for initializer.

Resolution

Landed on main as 288136ba: perf(parser): advance statement locations once (#610).

  • Capture each statement coordinate at recursive-descent entry, when statement starts advance in source order, and retain it until the node publishes in its unchanged completion order.
  • Maintain one exact forward cursor instead of an attacker-proportional line-start table and per-statement binary searches.
  • Reject non-monotonic entry and token starts inside multi-byte line terminators in release builds.
  • Checkpoint and restore the cursor plus published-location length when classic-for cover parsing rewinds, removing ghost entries for discarded speculative function bodies.

Exact-parent evidence

Fifteen order-balanced pairs per row, all 210 fresh processes, no discarded samples. The host was nominal but classified diagnostic, so wall results are supporting observations; retired instructions and exact allocation accounting are the stable claims.

ReleaseFast rowwallinstructionsallocationsallocated bytes
Unicode identifiers 4K0.842x0.7327x69 -> 682,813,232 -> 2,813,224
sequential locations 1K0.848x0.9107x2,087 -> 2,086925,384 -> 917,184
sequential locations 2K0.831x0.9058x4,140 -> 4,1391,451,192 -> 1,434,800
sequential locations 4K0.805x0.9029x8,241 -> 8,2403,471,632 -> 3,438,856
mixed terminators 4K0.881x0.8835x8,241 -> 8,2403,471,632 -> 3,438,856
nested completion order 1K0.864x0.9070x6,189 -> 6,1883,066,880 -> 3,042,296
one-statement 4K control0.978x0.9949x4,133 -> 4,1332,470,848 -> 2,470,848

Peak-RSS medians are 0.9952x to 0.9983x on affected rows and 0.9982x on the control. Every frozen checksum and input identity matched. Energy was below counter resolution, so no energy claim is made.

Correctness

  • test262: line terminators 41 -> 41, source text 1 -> 1, debugger 2 -> 2, modules 599 -> 599, classic for 385 -> 385: 1,028 -> 1,028, zero failures or flips.
  • Four real-corpus witnesses pass under both forced tree-walker and required-bytecode execution.
  • Frontend gate: 12/12.
  • Focused Debug and TSan: all three new location/rewind invariants pass.
  • Full units: 1,726 passed, one expected skip, zero failures, zero leaks.

Acceptance

  • Prove/test the monotonic statement-entry invariant, including nested blocks, function bodies, module/export declarations, and parser speculative rewinds.
  • Preserve registry order, node identity, debugger-statement marking, exact byte offsets, and 1-based line/column values for LF, CRLF, lone CR, U+2028, and U+2029.
  • Keep one-statement parses allocation-free for source-location metadata.
  • Remove the line-start table and its arena allocation without adding token/AST fields or attacker-proportional side indexes.
  • Exact-parent evidence covers Unicode identifiers plus sequential, mixed-terminator, nested-completion-order, and one-statement control rows.
  • Focused frontend Debug/TSan, full units, debugger/source-location test262, and tree-walker/VM probes remain exact.

Security and no-workaround ruling

Source length, statement nesting, comments, line-break encoding, and speculative grammar shapes remain attacker-controlled. Total metadata scanning is bounded to one forward pass over the reached prefix; CRLF is atomic; speculative state rolls back transactionally; invariant violations reject instead of underflowing or rescanning prefixes. No locations are omitted or coarsened, debugger registration remains enabled, tokens/AST nodes are not widened, and no job, Promise/microtask, Worker, stack, or queue lifecycle code changed.

9 days ago

Completed and pushed to main as 288136ba: perf(parser): advance statement locations once (#610).

The issue body now records the exact-parent matrix, transactional classic-for rollback, all correctness gates, and the completed acceptance checklist. The stable efficiency result is 0.7327x instructions on the causal Unicode row and 0.8835x–0.9107x across repeated/mixed/nested location rows; the wide one-statement control is 0.9949x with allocations and bytes exact. Test262 is 1,028 -> 1,028, full units are 1,726 passed plus one expected skip, and four real-corpus witnesses pass in both forced tree-walker and required-bytecode modes.

Sign in to comment on this issue.