ReviewOS

zig-utils/zig-js

Intl: bound CanonicalizeLocaleList deduplication and index keys

#528
Closed chris opened this 9 days ago ·
9 days ago

Parent: #475 Related: #460, #461, #480, #493

Problem

CanonicalizeLocaleList currently linearly scans every previously accepted locale for each candidate and formats each array index into realm-owned arena storage before HasProperty and Get. A large valid locale list therefore has quadratic duplicate-check cost and retains avoidable decimal-key backing even though each key is used only synchronously. Every Intl service constructor and supportedLocalesOf shares this path.

Scope

  • Preserve ECMA-402 CanonicalizeLocaleList ordering exactly: length, then per-index HasProperty, Get, element conversion/canonicalization, and first-occurrence output.
  • Replace quadratic duplicate membership with an invocation-local, securely seeded native content index whose exact byte equality remains authoritative on collisions.
  • Format decimal property keys into bounded native storage; no borrowed key may escape the synchronous property operation.
  • Release all index scratch at invocation exit and keep output strings/array under normal realm/GC ownership.
  • Cover unique, duplicate-heavy, canonical-equivalent, sparse, Proxy-observable, invalid, and adversarial inputs.

No-workaround rules

  • No locale-count cap, skipped validation, sorting/reordering, deterministic attacker-known hash placement, Object/Map-visible cache, unbounded process/realm cache, or bypass of Proxy traps and conversion hooks.
  • Index allocation failure must produce the engine's normal OOM outcome without a partial observable result or leak.

Acceptance

  • Deterministic 1K/2K/4K unique and duplicate-heavy rows plus a small-list control preserve frozen checksums.
  • Exact-parent wall/instruction/allocation/RSS evidence attributes the retained change and rejects regressions on the control.
  • Getter/Proxy order, first-occurrence ordering, canonical-equivalent deduplication, and invalid-tag exceptions are unchanged.
  • OOM at index growth and output publication is rollback-safe and leak-free.
  • Moving GC, bounded-heap reclamation, no-GIL, focused TSan, full units, and test/intl402 pass with a nonzero denominator.

Security boundary

Hash placement must be unpredictable per interpreter/invocation from the existing secure engine entropy stream, while exact canonical-locale byte equality decides membership. This is an algorithmic-complexity boundary, not a best-effort micro-optimization.

9 days ago

Implemented and verified on main.

Commits:

  • 77b93913 adds frozen 8/1K/2K/4K/duplicate-heavy rows.
  • 9cc8d496 adds the secure invocation-local exact-content index, bounded decimal keys, OOM propagation, and GC/thread regressions.
  • c4662097 retains one bounded per-invocation canonicalization capacity and resets it per element.
  • b9e44874 checks in the raw exact-parent evidence and attribution report.

Evidence: exact-parent-locale-list-allocation-2026-08-10.md.

Measured result (candidate / exact parent; seven order-balanced fresh-process pairs per edge):

  • final 8-item control: ~0.969x wall / 0.954x instructions across the two exact-parent edges;
  • 1K unique: 0.252x wall / 0.219x instructions;
  • 2K unique: 0.156x / 0.141x;
  • final 4K unique: ~0.069x / 0.056x across both edges;
  • duplicate-heavy 4K: 0.862x / 0.921x.

Allocation attribution for the scored 4K invocation: backing calls 164,272 -> 82,450, requested bytes 61,486,226 -> 13,048,926, retained-backing increase 60,710,606 -> 4,191,682, and GC-cell calls 163,922 -> 82,002. Exact binary/dependency hashes, cycles, energy, thermal state, RSS, and all raw samples are checked in.

Correctness/security:

  • HasProperty/Get and conversion order, sparse Proxy behavior, first occurrence, and canonical-equivalent dedup are regression-tested;
  • exact bytes remain authoritative under securely seeded Wyhash placement; entropy failure propagates instead of using a predictable fallback;
  • deterministic allocation-failure sweep covers canonicalization, index growth, duplicate insertion, and StringCell publication;
  • moving nursery, bounded 4 MiB heap, and four-lane no-GIL ownership pass;
  • focused TSan: 1/1, 0 races, 0 leaks;
  • full units: 1,574 pass, 1 skip, 0 fail, 0 leak;
  • test/intl402: 3,341/3,341 (0 failures, nonzero denominator).

All acceptance boxes are complete.

9 days ago

Closing as completed. The implementation, refinement, exact-parent evidence, allocation attribution, and all acceptance gates are linked above.

Sign in to comment on this issue.