ReviewOS

zig-utils/zig-js

JSON: emit flat Latin-1 strings without transcode storage

#778
Closed chris opened this 9 days ago ·
9 days ago

Parents: #473, #480 Related: #675, #777

Problem

With production flat Latin-1 StringCell storage enabled, JSON.stringify currently calls Value.asWtf8 for every string value. Eligible strings are first transcoded into temporary arena storage and then copied again into the authoritative JSON output buffer. The property-key writer already writes canonical storage directly, and structured clone already has a representation-aware final-buffer writer.

Scope

  • Add a representation-aware JSON string writer for String Values.
  • Borrow ASCII and canonical WTF-8 storage directly.
  • Encode physical flat Latin-1 directly into the authoritative output buffer while applying QuoteJSONString escapes.
  • Preserve lone-surrogate escaping, control/quote/backslash escapes, astral and BMP content, replacer/toJSON ordering, omission rollback, output limits, and OOM atomicity.
  • Prove a capacity-prepared flat-string write succeeds with an allocator that fails its first allocation.

No-workaround rules

  • No permanent canonical shadow copy, eager global canonicalization, lossy replacement, invalid-byte acceptance, alternate JSON semantics, hidden second pass, or benchmark/input special case.
  • No stack, queue, job, Promise, Worker, or Map/Set iterator implementation changes.

Acceptance

  • Flat Latin-1 string values emit exact canonical JSON directly into final output storage.
  • Canonical/ASCII/BMP/astral/lone-surrogate and all JSON escape classes remain exact.
  • A fail-first allocation witness proves a pre-capacitated flat write allocates no transcode buffer.
  • JSON-focused normal/TSan/OOM tests, exact-parent Test262 JSON accounting, full units, and deterministic tier fuzzing pass with zero regressions.
  • Exact allocation evidence is recorded without an unmeasured throughput claim.
9 days ago

Implementation is active locally and focused evidence is green.

  • Flat Latin-1 string values now expand directly into the authoritative JSON output buffer; ASCII/canonical storage remains borrowed.
  • A pre-capacitated output plus fail-first allocator proves the direct flat writer performs no hidden transcode allocation while preserving every JSON escape class and canonical éÿ output.
  • JSON-focused units: 17/17 normal and 17/17 TSan, including heap-limit OOM atomicity, GC callback survival, replacer/toJSON/key/gap/raw/cycle behavior, and C/private API boundaries; zero leaks/races.
  • Exact-parent test/built-ins/JSON: 165 -> 165 pass, 0 fail, 0 flips.

Full units and deterministic tier fuzzing remain before landing. No throughput claim is being made.

9 days ago

Implementation landed on main in perf(json): stream flat Latin-1 strings (#778).\n\nThe direct writer and all deterministic gates are complete: fail-first zero-intermediate-allocation witness, 17/17 JSON units, 17/17 JSON TSan, exact-parent Test262 JSON 165 -> 165 with zero flips, and 400/400 seeded threadfuzz.\n\nI am deliberately leaving #778 open. Two full-suite attempts under concurrent external Home/Bun Zig work each completed 1,967 pass / 1 expected skip / 0 leaks but recorded one timeout in parallel_js: cooperative shared nursery rendezvous bounds object churn; that exact witness passes 1/1 in isolation. No concurrency threshold or reserved stack/queue/job code was changed. Closure waits for a clean-host full gate.

9 days ago

Follow-up landed in a44cf2b1: boxed String values and boxed indentation now retain the exact primitive StringData returned by ToPrimitive/ToString instead of canonicalizing into scratch WTF-8 and allocating a replacement StringCell.

Evidence on the final source:

  • JSON units: 17/17 normal, 17/17 TSan; zero failures, leaks, or sanitizer reports.
  • Test262 test/built-ins/JSON: 165 pass, 0 fail; issue-wide exact-parent result remains 165 -> 165 with zero flips.
  • The regression witness covers flat Latin-1 boxed serialization plus boxed indentation.

The issue remains open pending a clean-host full unit gate; unrelated Home/Bun Zig builds are still active, so another contaminated full run would not add reliable evidence.

9 days ago

Acceptance is complete. Clean main CI run 33071676798 passed all four ordinary unit shards at 872dbbb5; the issue-specific focused normal/TSan/OOM, exact-parent Test262, allocation, and deterministic fuzz evidence is recorded above. Closing with no semantic regressions.

Sign in to comment on this issue.