ReviewOS

zig-utils/zig-js

Strings: make indexOf positions UTF-16 exact

#623
Closed chris opened this 9 days ago ·
9 days ago

Parent: #473 Depends on: #622 (complete) Related: #493

Problem

String.prototype.indexOf and lastIndexOf clamped and reported UTF-16 positions, but their actual match was a raw WTF-8 byte substring search. A high or low surrogate needle therefore could not match its corresponding code unit inside an astral scalar, and a start/limit inside a surrogate pair was mapped to a whole-sequence byte boundary that could admit or exclude the wrong candidate. #622 established the shared exact forward matcher for the predicate methods; this child owns the numeric-returning methods and reverse/limited search.

Scope

  • Reuse the exact UTF-16 matcher boundary from #622 for indexOf.
  • Add a bounded-linear last-match search whose candidate start is at or before the specified UTF-16 limit.
  • Preserve ToString/ToNumber order, NaN/default handling, empty-needle results, ASCII allocation-free matching, lone surrogates, and OOM propagation.
  • Keep byte-offset conversion only where representation invariants prove it equivalent to code-unit indexing.

No-workaround rules

  • No code-point comparison, normalization, replacement characters, repeated random-access decoding, ASCII-only acceptance, or raw-byte result correction after an inexact search.

Acceptance

  • Unit fixtures cover BMP, astral halves, lone surrogates, empty needles, start/limit boundaries, overlap, and coercion order.
  • indexOf and lastIndexOf return exact UTF-16 code-unit indices with bounded-linear non-ASCII work.
  • Relevant Test262 subtrees have nonzero exact before → after accounting with no regressions.
  • Focused/full units and applicable sanitizer gates pass; #473 and #622 are cross-linked.
9 days ago

Dependency #622 is complete in 287550f8. #623 can now reuse Interpreter.stringIndexOfUtf16, including its ASCII fast path, streaming UTF-16 KMP path, freeable scratch lifetime, and OOM guard; only numeric result plumbing and bounded last-match classification remain.

9 days ago

Completed on main in 9a167534 (fix(strings): return UTF-16 search indexes (#623)).

indexOf now returns the first exact UTF-16 code-unit match from its clamped start. lastIndexOf performs a bounded forward scan only through maxStart + needleLength, retaining the greatest admissible start and the KMP suffix-prefix state needed for overlapping matches. Both share #622’s u16 pattern/u32 prefix table, free it on every path, preserve the allocation-free ASCII path, and propagate OOM without an inexact fallback.

Exact evidence:

  • frozen parent 287550f8c2abd016c6979a58242f02aef696e489 runner SHA-256: f7a1109694c25eb3e4d0b0d08a49a238fbc5f6c6a1700f33326175458166831a;
  • candidate runner SHA-256: cc08f845daa6f5877a27afff5fd558f5f8dc80c4c732694a4e2b18bae1531fe3;
  • Test262: indexOf 47→47 and lastIndexOf 25→25, for 72→72, zero failures and zero flips;
  • focused UTF-16 runtime: 8/8 normal and 8/8 TSan (five predicate regressions plus three numeric-search cases);
  • filtered inline unit: 1/1, zero leaks;
  • full units: 1,736 passed, 1 platform skip, 0 failed, 0 leaked across 10 shards;
  • dependency audit: 15 classified edges, 0 migration targets, 60 tools.

The new fixtures cover numeric BMP positions, astral high/low halves, lone surrogates, empty needles, start/limit boundaries, coercion order, long repeated prefixes, and overlapping matches. Flips 0 Test262 cases.

9 days ago

Follow-up #624 extends the shared exact UTF-16 matcher from this issue into plain-string replace, replaceAll, and split, including astral surrogate-half boundaries and non-overlapping scans.

Sign in to comment on this issue.