ReviewOS

zig-utils/zig-js

RegExp: avoid retained no-op UTF-16 search-input copies

#561
Closed chris opened this 9 days ago ·
9 days ago

Parent: #473 Related: #560

Problem

Interpreter.regexpSearchInput appends the complete input to an arena before it knows whether any astral code point requires conversion to WTF-8 surrogate code units. When no conversion is needed it returns the original slice, leaving the full constructed copy unused but retained by the arena.

The hottest matching callers avoid this for ASCII, but compileRegex invokes it for every non-Unicode pattern, and BMP-only non-ASCII match inputs still take the allocate-then-discard path.

Required behavior

  • return the original slice without allocating when the input contains no astral code point;
  • when conversion is required, allocate/fill exact output or otherwise prove a lower-cost bounded construction;
  • preserve non-u UTF-16 code-unit matching, lone-surrogate behavior, byte↔code-unit span mapping, captures, lastIndex, and /u/v behavior exactly;
  • propagate OOM without publishing partial state or retaining abandoned conversion buffers.

No-workaround rules

  • no ASCII-only semantic shortcut at the public boundary;
  • no skipped conversion for astral input;
  • no changed RegExp indices/captures or invalid Unicode normalization;
  • no cache with unbounded, realm-unsafe, or unsynchronized lifetime.

Acceptance

  • allocation tests prove the no-op path performs zero conversion allocations and the changed path is failure-atomic;
  • exact-parent rows cover ASCII, BMP non-ASCII, astral, lone-surrogate/WTF-8, match/no-match, and global/sticky index progression with exact checksums;
  • focused RegExp tests, test262 RegExp/literal subtrees, OOM, TSan, and full units pass.
9 days ago

Completed on main:

  • d51b6a7f adds exact-checksum ASCII, BMP, astral, and lone-surrogate global-match controls.
  • e96196c8 borrows every no-op search input with zero conversion allocations and expands astral suffixes through one overflow-checked arena allocation bounded to ≤1.5× input.

Seven alternating exact-parent pairs improve ASCII 3.3%, BMP 9.4%, astral 2.1%, and lone-surrogate 9.4%, with checksums unchanged at 298,992 / 298,992 / 349,040 / 324,016. The focused allocator test proves the no-op path succeeds with a zero-byte allocator, astral conversion uses one backing allocation, and zero-capacity changed input propagates OOM.

Validation: focused RegExp + TSan 21/21; full units 1,637 pass / 1 skip / 0 fail / 0 leak; exact-parent test262 remains 1,879/1,879 built-ins/RegExp and 238/238 language/literals/regexp (0 flips, 0 regressions).

Sign in to comment on this issue.