Strings: bound non-ASCII UTF-16 indexed access

Parents: #473, #493 Related: #623, #665, #766

Problem

#766 makes cached-ASCII UTF-16 length and indexed reads constant time. Every non-ASCII canonical WTF-8 cell still resolves an abstract UTF-16 index by constructing a code-unit iterator at byte zero. A sequential reader such as for (let i = 0; i < s.length; i++) s.charCodeAt(i) therefore performs quadratic decoding work on BMP, astral, and lone-surrogate inputs. Primitive and boxed string exotic index reads share the same underlying scan.

This is an attacker-controlled complexity surface and a material gap from a production JavaScript engine string model.

Scope

• Freeze benchmark-first sequential and high-index witnesses for BMP, astral, lone-surrogate, mixed, Latin-1, and ASCII controls through both String builtins and primitive/boxed exotic properties.
• Measure representation-native flat storage versus a bounded immutable UTF-16 offset index before choosing the runtime shape.
• Make repeated non-ASCII UTF-16 index resolution bounded: O(1) for a flat representation or O(stride) after one O(n) index construction.
• Preserve exact UTF-16 length, surrogate-half visibility, codePointAt pairing, canonical WTF-8 equality/hashing/interning, property descriptors, and ABI string egress.
• Keep metadata proportional to immutable string storage, lazily absent on strings that are never indexed, and released exactly once across arena, interned, external, and GC-managed cell lifetimes.
• Publish any lazy metadata race-free under no-GIL; moving GC must preserve or relocate its identity without stale cell-pointer keys.
• Make allocation and entropy failure atomic and catchable where the public operation can allocate; allocation-free length/has/descriptor probes must keep an exact non-allocating path.

No-workaround rules

No source/workload recognizer, ASCII-only claim presented as completion, process-global cache, unbounded per-index memoization, eager index allocation for every string, UTF-8 byte indexing, skipped surrogate halves, disabled moving GC, leaked side table, or OOM fallback that silently restores attacker-sized repeated rescans.

No stacks, queues, jobs-runtime, Promise, microtask, Worker-lifecycle, or Map/Set iterator changes.

Acceptance

• Benchmark-first rows freeze equal logical work, checksums, representations, allocation replay, and 1K/2K/4K growth before the runtime change.
• BMP, astral, lone-surrogate, and mixed sequential reads demonstrate bounded candidate growth; ASCII and Latin-1 controls do not regress materially.
• charAt, charCodeAt, at, codePointAt, primitive indices, boxed exotic indices, length, descriptors, enumeration, and delete/set/has boundaries remain semantically identical.
• Metadata construction/publication is bounded, race-free, failure-atomic, moving-GC safe, and released under managed, arena, interned, and external string lifetimes.
• Focused units, OOM/fault injection, moving GC, no-GIL/TSan, full units, and exact String Test262 accounting pass with before -> after results.
• Exact-parent raw evidence and methodology are checked in without converting noisy wall time into a claim.

Benchmark-first evidence landed in bench(strings): add UTF-16 indexing growth witnesses (#767).

Frozen evidence:

• 18 required-bytecode/no-native rows: six representations × 1K/2K/4K UTF-16 widths
• all 18 quick and full checksums reproduced exactly
• two independent full attribution replays per row matched admissions, VM dispatches, Context backing-allocation counts, and backing-allocation bytes
• benchmark harness self-tests 18/18, representative matrix structural tests 52/52, dependency audit, 277-file docs link check, and 50-page BunPress manifest build all passed

No runtime behavior or Test262 result changes are claimed by this benchmark-only commit. The issue remains open for the representation decision, bounded implementation, semantic/security tests, and exact-parent evidence.

Foundation landed in perf(strings): cache UTF-16 lengths (#767).

This packs exact ordinary-string UTF-16 lengths into existing StringCell hash-state bits, keeps the cell at 32 bytes, retains exact byte equality and independent keyed intern placement, and makes primitive/boxed length plus allocation-free bounds/has probes O(1) without adding an allocation or an OOM fallback.

Validation: 17/17 StringCell tests, 17/17 UTF-16 cross-layer tests, and the focused primitive/boxed length unit passed. No additional acceptance checkbox is closed yet: non-ASCII character reads still need the bounded lazy index, lifetime/fault/no-GIL/moving-GC proof, corpus accounting, and exact-parent evidence.

Runtime implementation landed in perf(strings): bound UTF-16 indexed access (#767).

The 32-byte StringCell now publishes one immutable 64-code-unit stride index after the first valid long non-ASCII indexed read. Publication is CAS-based and failure-atomic; losers free complete candidates. The index retains exact unit count for oversized embedding inputs, preserves external/intern ownership, survives moving GC without cell-key side tables, and is finalized by the owning managed/intern/arena lifetime. The shared iterator also removes the remaining quadratic empty-separator split path.

Validation:

• full sharded units: 1,935 passed, 1 expected skip, 0 failed, 0 leaked
• focused UTF-16 cross-layer suite: 24/24
• direct eight-thread no-GIL publication witness under TSan: 1/1, no sanitizer findings
• indexed external callback lifecycle: exact-once and post-sweep
• exact-parent Test262: built-ins/String 1,223 -> 1,223; language/types/string 24 -> 24; Object 3,411 -> 3,411; Reflect 153 -> 153; total 4,811 -> 4,811, flips 0

The issue remains open for the checked-in exact-parent instruction/allocation growth artifact. No wall-time or throughput claim is made by this commit.

Evidence-pipeline unblock landed in a5975dbb / #768.

The rejected ASCII-1024 attempt exposed that the old collector gated /usr/bin/time occupancy over startup, warmup, teardown, thermal queries, and replay rather than the scored invocation. V24 now records native getrusage CPU deltas over the exact boundary and applies the same 60% threshold there; no threshold, logical job count, checksum, workload, stability rule, or non-throughput boundary changed.

A final candidate probe preserves checksum 23,880,132 and reports ~99.8% measured-boundary occupancy despite ~9.7% complete-process occupancy. Exact parent/candidate growth collection resumes with the frozen V23 rows through attribution/algorithmic-growth v2. No performance claim is made from this single diagnostic probe.

The exact-parent evidence pipeline now records and validates the shared measurement-overlay provenance in 7831d3df, completing the tooling side of the #768 unblock. Collection will compare logical 5bca24a3 → c65729e5 using the byte-identical V24 measurement overlay at parent binary 7d82c96e and candidate binary a5975dbb; those identities and every overlay path will be embedded in each v3 raw artifact.

The first valid exact-parent control is checked in at 4e4531ec: ASCII-1024, two order-balanced pairs, exact checksum 23,880,132, exact allocation replay 176,365 requests / 24,257,955 bytes for both variants, stable native instructions, and measured-boundary occupancy above the 60% guard. The artifact is explicitly diagnostic_only; it is not a public wall-time claim. The remaining 17-row batch was attempted again after a clear preflight and failed closed before its first invocation when four unrelated Home builds started during collector startup. No partial rows were written.

Completed by 33734e97 — bench(strings): publish guarded UTF-16 growth (#767).

Final checked evidence:

• 18 ordinary exact-parent rows cover ASCII, Latin-1, BMP, astral, lone-surrogate, and mixed strings at 1K/2K/4K, with two alternating samples and two exact allocation replays per variant.
• 1K → 4K retired-instruction growth is parent/candidate: ASCII 4.053×/4.048×, Latin-1 15.244×/4.037×, BMP 15.295×/4.035×, astral 14.945×/4.037×, lone-surrogate 14.546×/4.043×, mixed 15.000×/4.037×.
• The ASCII control stays linear and its candidate overhead is 1.89–2.03%; Latin-1 improves rather than regresses.
• Minimum measured-boundary occupancy is 99.67%, every thermal transition is nominal, every wall row is stable, and all wall/CPU/RSS/cycle/energy/thermal observations remain explicitly diagnostic and unscored.
• All six v3 aggregate artifacts pass the official algorithmic-growth.ts --artifact validator.

This evidence-only commit flips 0 Test262 cases.

Follow-up CI repair landed in 57dfae74: the dependency audit now classifies the generated algorithmic-growth and exact-parent evidence families through narrow, fail-closed docs/.data/ prefixes, while continuing to reject broad and stale prefixes. zig build dependency-audit passes with 63 classified tools; flips 0 Test262 cases.