ReviewOS

zig-utils/zig-js

Private ABI: canonicalize decoded String value egress

#672
Closed chris opened this 8 days ago ·
8 days ago

Parents: #473, #495 Related: #480, #493

Problem

Private ZigString and BunString ingress correctly decodes external Latin-1/UTF-16/UTF-8 into a String Value, but several downstream consumers immediately expose Value.asStr() as if it were canonical WTF-8. Once flat-Latin-1 storage is enabled, that leaks the physical one-byte image into canonical property-key interning, URL and form parsing, error construction, host-function/structure names, atom interning, time-zone validation, and string property-path parsing. Equal non-ASCII inputs can then corrupt bytes, miss keys, or trip parser/debug invariants.

Confirmed boundaries include privateZigStringPropertyKey, privateBunStringPropertyKey, URLSearchParams__create, ZigString__toAtomicValue, JSGlobalObject__setTimeZone, private Error/SystemError/DOMException constructors, JSFunction__createFromZig, JSC__createStructure, and JSC__JSValue__getIfPropertyExistsFromPath. Existing serialized output views and true representation-native predicates are separate and must remain allocation-aware.

Module loading and every Promise, stack, queue, job, Worker, and Map/Set iterator surface are explicitly excluded and remain user-owned.

Scope

  • Inventory every in-scope private ABI path that converts an externally decoded String Value back to semantic bytes.
  • Introduce one ownership- and OOM-correct canonical WTF-8 borrow/conversion boundary.
  • Route semantic byte consumers through it while preserving representation-native is8Bit and outward Latin-1 view behavior.
  • Keep ASCII and canonical storage borrowed with zero conversion allocation; physical flat Latin-1 uses at most one bounded conversion per consumed value.
  • Preserve exact exception publication, VM/realm ownership, atom identity, URL/form semantics, error fields, and ABI layouts/symbols.

No-workaround rules

  • No global flat-storage disable, lossy UTF-8 replacement, invalid-byte acceptance, duplicate hidden copy, system transcoder, ABI fork, or input-specific special case.
  • No module, stack, queue, job, Promise, Worker, or Map/Set iterator changes.

Acceptance

  • Property keys, atoms, host-function names, structure names, and string property paths preserve exact non-ASCII identity across canonical and physical flat-Latin-1 layouts.
  • URLSearchParams, FormData URL query, DOMURL, time-zone, Error, SystemError, AggregateError, and DOMException ingress consume exact canonical bytes.
  • ASCII/canonical inputs borrow storage; physical flat Latin-1 performs one bounded conversion with exact OOM publication.
  • Public/private ABI audits and fixtures, focused/full units, OOM, GC, no-GIL/TSan where shared, and relevant Test262/Web Platform controls remain exact.
  • No public/private symbol, enum, or layout changes.
8 days ago

Landed on main in df57cd7e.

One representation-aware WTF-8 boundary now covers the in-scope private ABI semantic byte consumers: host-function and SQL structure names, atom and property-key interning, Error/SystemError/AggregateError/DOMException text, string property paths, URLSearchParams, FormData URL queries, DOMURL input, and time-zone validation. ASCII and canonical cells borrow; future physical flat-Latin-1 cells perform one bounded conversion. Outward Latin-1 views, ABI layouts/symbols, module loading, and every user-owned Promise/job/Worker/iterator surface are unchanged.

Verification:

  • focused canonical-ingress witness: pass, 0 leaks
  • focused TSan witness: pass, 0 leaks
  • Unicode and Latin-1 property-path witness: pass
  • full units: 1767 passed, 1 skipped, 0 failed, 0 leaked
  • Home public ABI: 50/50 exports
  • Home private ABI: 447/447 implemented, 0 pending/unclassified
  • Bun private ABI: 437/437 implemented, 0 pending/unclassified
  • Home value fixture: 392/392 symbols and runtime matrix
  • Bun SQL structure: 4/4 symbols and runtime matrix
  • Bun DOMFormData: 10/10 symbols and runtime matrix
  • private ABI scanner self-test and affected C-API extension fixture: pass
  • Test262: 0 relevant private-host-boundary cases, so no flip measurement applies
8 days ago

The public C counterpart to this private egress fix is complete as #674 via d56dbb80. All four JSObject*PropertyForKey entry points now consume representation-aware canonical String bytes with unchanged symbols, layouts, coercion, and exception behavior.

8 days ago

Follow-on #696 landed in 459af38b, routing remaining private-ABI equality and substring consumers away from physical string bytes. Forced-flat, sanitizer, ABI, and full-unit gates passed.

8 days ago

Follow-on #697 landed in f9ca94d6, closing the standalone NaN-box bridge reader missed by the earlier string-cell migration. Canonical and forced-flat intern identities now converge exactly.

8 days ago

Follow-on reader #698 landed in f843f8d8, routing zipKeyed padding/property-name consumers through canonical WTF-8 rather than StringCell physical bytes.

8 days ago

Follow-up #699 found and closed the remaining FormData callback filename egress missed by this umbrella. ad6c363c replaces the physical StringCell byte read with representation-aware WTF-8 projection; the exact old path fails under forced-flat storage and the repaired path passes Debug/ReleaseSafe/TSan plus all ABI inventories and full units.

Sign in to comment on this issue.