ReviewOS

zig-utils/zig-js

JSON: truncate stringify gap by UTF-16 code units

#664
Closed chris opened this 9 days ago ·
9 days ago

Parents: #473, #493

Problem

JSON.stringify(value, replacer, space) truncated a string space value with wtf8_bytes[0..10]. ECMA-262 requires the first 10 UTF-16 code units. Non-ASCII storage therefore under-counted multibyte BMP characters, and a cutoff inside an astral scalar could publish invalid backing bytes instead of the required lone surrogate code unit.

The upstream Test262 range case is ASCII-only, so it does not cover this representation boundary.

Scope

  • Truncate the gap by exactly 10 UTF-16 code units over canonical WTF-8.
  • Preserve an astral scalar when both surrogate code units fit; materialize the exact lone high surrogate when the tenth code unit bisects a pair.
  • Keep the ASCII common path allocation-free and stop non-ASCII prefix scanning after the required boundary.
  • Apply the same rule after observable String-wrapper coercion.
  • Add integration coverage for multibyte BMP, exact astral pairs, split astral pairs, lone surrogates, and boxed strings.

No-workaround rules

  • No invalid-byte slicing, replacement characters, code-point truncation, source rewriting, upstream-test exclusion, or third-party string conversion.
  • No stacks, queues, jobs, Promise, Worker, or Map/Set iterator changes.

Acceptance

  • Gap output contains exactly the first min(10, length) UTF-16 code units for ASCII, BMP, astral, and lone-surrogate inputs.
  • Cutoff inside an astral scalar preserves the exact high surrogate as canonical WTF-8.
  • Existing JSON stringify ordering, replacer/toJSON observability, OOM behavior, and ASCII fast path remain unchanged.
  • Focused JSON units, relevant nonzero Test262 JSON.stringify accounting, and the full unit gate pass with zero regressions.
9 days ago

Implemented and pushed to main as 0052b2aa.

The gap now truncates by UTF-16 code units across ASCII, flat Latin-1, canonical WTF-8, astral, lone-surrogate, and boxed-string inputs. ASCII remains borrowed/allocation-free; flat Latin-1 transcodes at most ten source units; an astral split at unit ten materializes the exact lone high surrogate.

Validation:

  • full 8-shard unit gate: 1,754 passed, 1 platform skip, 0 failed, 0 leaked
  • Test262 test/built-ins/JSON/stringify: 66 -> 66 passing, 0 failures, nonzero denominator
  • focused JSON coverage is included in the full gate; no regressions

This does not touch stacks, queues, jobs, Promise, Worker, or Map/Set iterator surfaces.

9 days ago

Remote CI follow-up for 0052b2aa: workflow 32423835718 completed with 49 successful jobs, 1 skipped job, and 2 TSan failures. Both failures reproduce the separately owned #643 Map iterator cursor race (collectionIteratorState / collectionCursorIterNext): 2 warnings in tsan-parallel-js and 3 warnings in TSan unit shard 1. No JSON.stringify test, unit, Test262, or non-TSan gate failed.

Sign in to comment on this issue.