ReviewOS

zig-utils/zig-js

Strings: search predicates by UTF-16 code unit

#622
Closed chris opened this 9 days ago ·
9 days ago

Parent: #473 Related: #493, #495, #623

Problem

String.prototype.includes, startsWith, and endsWith clamped their positions against WTF-8 byte length and sliced the backing bytes at that numeric offset. ECMAScript indexes Strings by UTF-16 code unit. BMP/astral/lone-surrogate input could therefore search from the wrong position, slice through an encoded sequence, or fail to match a surrogate half inside an astral scalar. The existing private-ABI stringIncludesUtf16 helper was exact but repeatedly rescanned both strings from the beginning, giving adversarial inputs a superlinear implementation path.

Scope

  • Preserve the specified RequireObjectCoercible, IsRegExp, ToString, and ToIntegerOrInfinity ordering.
  • Clamp positions against UTF-16 length and compare exact UTF-16 code-unit sequences for includes/startsWith/endsWith.
  • Preserve the allocation-free byte-search fast path when receiver/search representations prove byte offsets equal code-unit offsets.
  • Give non-ASCII search a bounded linear-time implementation, including astral halves and lone surrogates.
  • Share one helper boundary so #623 can adopt the same exact matcher for indexOf/lastIndexOf.
  • Preserve the private JSC__JSValue__stringIncludes contract and pending-exception behavior.

No-workaround rules

  • No normalization, replacement-character substitution, code-point comparison, ASCII-only acceptance, test exclusion, or byte-position reinterpretation.
  • No repeated random-access decoding that multiplies source length by search length.
  • OOM must propagate; no silent fallback to the known-inexact byte matcher.

Acceptance

  • Exact unit fixtures cover ASCII, BMP, astral pairs, high/low surrogate needles, lone surrogates, empty needles, boundary positions, and coercion order.
  • Search predicates use UTF-16 lengths and exact code-unit equality while retaining the proven ASCII fast path.
  • Non-ASCII includes is bounded linear in receiver plus needle length, with freeable invocation scratch.
  • Relevant Test262 subtrees have nonzero exact before → after accounting with no regressions.
  • Focused/full units, private ABI, and applicable sanitizer gates pass; parents and follow-up are cross-linked.
9 days ago

The same storage/index mismatch also affects the numeric-returning methods. I split that follow-up into #623 so this change can land the shared forward matcher and predicate semantics as one reviewable correctness unit; #623 will reuse it for indexOf and add the limited last-match path for lastIndexOf.

9 days ago

Completed on main in 287550f8 (fix(strings): search predicates by UTF-16 units (#622)).

The predicate methods now clamp and compare in UTF-16 code-unit space. ASCII cells retain the allocation-free byte path; non-ASCII includes streams through KMP in O(receiver + needle) work. Its u16 pattern and u32 prefix tables use the context-owned freeable scratch allocator, are released on every return/error path, and explicitly fail with OOM before a prefix-width narrowing could trap. startsWith/endsWith stream exact code-unit equality without materializing substrings. The private JSC__JSValue__stringIncludes shim uses the same matcher and maps OOM into the existing pending-exception channel.

Exact evidence:

  • frozen parent runner SHA-256: c278ec5a517b44df353f50b59e52d7c52ddfc968fabbcaf94f98ceb2f7154366 at 84efdae532e0bc9a198b3764dc9a20645bea2ed0;
  • final candidate runner SHA-256: f7a1109694c25eb3e4d0b0d08a49a238fbc5f6c6a1700f33326175458166831a;
  • Test262: includes 27→27, startsWith 21→21, endsWith 27→27: 75→75, zero failures and zero flips;
  • focused runtime: 5/5 normal and 5/5 TSan; filtered inline unit: 1/1, zero leaks;
  • full units: 1,735 passed, 1 platform skip, 0 failed, 0 leaked across 10 shards;
  • Home private ABI: audit 447/447 implemented private symbols, 0 pending/unclassified; runtime 392/392 normal and TSan;
  • dependency audit: 15 classified edges, 0 migration targets, 60 tools.

The corpus has no surrogate-position cases, so the focused and inline fixtures supply the missing BMP, astral-half, lone-surrogate, empty-boundary, coercion-order, and repeated-prefix coverage. #623 owns the already-cross-linked numeric indexOf/lastIndexOf follow-up. Flips 0 Test262 cases.

9 days ago

The planned numeric-search follow-up #623 is complete via 9a167534. It reuses #622’s exact forward matcher and factors the shared freeable KMP pattern while adding bounded overlapping last-match scanning; all eight predicate/index focused cases pass normal and TSan.

Sign in to comment on this issue.