ReviewOS

zig-utils/zig-js

Frontend: decode escaped private names into one owned token

#551
Closed chris opened this 9 days ago ·
9 days ago

Parent: #496 Related: #538, #539, #540

Problem

An escaped private name currently materializes its decoded identifier in lexEscapedIdentName, then nextRaw uses allocPrint to allocate a second #-prefixed token. The parser retains both arena buffers even though only the complete private-name token remains reachable. Generated classes with many escaped private fields therefore pay two owned buffers per key.

Scope

  • Extend the existing escaped-private-class parser workload to 1K/2K/4K rows with exact per-member decoded-key validation.
  • Let the escaped-identifier decoder seed the final owned spelling with # for private names, eliminating the intermediate decoded-only buffer and allocPrint copy.
  • Keep ordinary private names as complete source-backed slices and ordinary identifiers unchanged.
  • Preserve Unicode IdentifierStart/IdentifierPart validation, \\uXXXX/\\u{X} decoding, escaped-keyword behavior, token positions/ranges, private-name identity, and deterministic OOM propagation.

No-workaround rules

  • No source rewriting, worst-case allocation, hash-only identity, interning cache, benchmark-specific spelling, dropped #, relaxed Unicode validation, or deferred syntax errors.

Acceptance

  • Benchmark-first 1K/2K/4K rows validate every exact decoded private key and establish allocation growth before lexer changes.
  • Each escaped private token owns one complete #-prefixed decoded buffer; no decoded-only intermediate survives. Ordinary private names remain source-backed with zero decoded-byte allocation.
  • Tests cover multiple escapes, raw Unicode, \\uXXXX/\\u{X}, invalid start/continue scalars, exact offsets, allocation counts/bytes, and deterministic OOM.
  • Lexer/parser/frontend, TSan, full units, forced private-name execution, and exact private-name/class test262 comparisons pass without regressions.
  • Exact-parent evidence records wall, hardware counters, allocation requests/bytes, RSS, checksums, and 1K/2K/4K growth without moving lexing/parsing outside the timed boundary.
9 days ago

Implemented, verified, and published on main:

Escaped private names now seed identifier decoding with #, producing the complete token in one owned list and removing the second formatted token buffer. Ordinary identifiers are unchanged; unescaped private names retain complete source-slice borrowing. Tests cover multiple Unicode escape forms mixed with raw Unicode, exact offsets, one-allocation ownership plus exact shrink, zero-memory borrowing, deterministic OOM/retry, and invalid escaped start/continue scalars.

Verification:

  • focused frontend Debug and TSan: 12/12 each
  • full units: 1,619 passed, 1 skipped, 0 failed, 0 leaked
  • exact-parent test262: class-expression elements 1,428 -> 1,428; class-declaration elements 1,534 -> 1,534
  • forced escaped-private-name execution: 17 -> 17

Across seven order-balanced pairs, parser allocation requests fall 59.5-59.8%, allocated bytes 13.0-15.5%, and stable instructions 10.0-10.1%. Stable 1K/4K wall improves 6.7%/8.7%; noisy rows remain diagnostic. Every acceptance item is complete.

Sign in to comment on this issue.