ReviewOS

zig-utils/zig-js

Intl: generate complete locale-aware DisplayNames data

#582
Closed chris opened this 9 days ago ·
9 days ago

Parent: #475 Related: #460, #473

Problem

A public Intl.DisplayNames differential against the host JavaScriptCore found that current generated/owned data and composition rules are not exact outside the already-covered English lookup subset:

  • new Intl.DisplayNames("en", {type:"language"}).of("zh-Hant") returns Traditional Chinese; JSC returns Chinese, Traditional.
  • English short/standard language composition uses long region wording (English (United States) versus JSC English (US)).
  • Non-English resolved locales currently fall back to codes even for known currencies: French USD/EUR/JPY returns the codes rather than localized names.

The existing structural Intl402 suite remains green, so this needs an explicit data/algorithm child rather than being hidden inside a performance change.

Requirements

  • Generate locale-indexed DisplayNames language, region, script, currency, calendar, and date-time-field data from pinned owned CLDR inputs; edit the generator, never generated tables by hand.
  • Implement exact dialect/standard and long/short/narrow pattern composition, aliases, canonicalization, fallback, and missing-data behavior.
  • Preserve ECMA-402 validation and observable coercion order.
  • Keep tables immutable, bounded, shared safely across realms, and independent of ICU/system JavaScriptCore.
  • Add exact public differential controls across representative locales/scripts/regions/currencies and all supported types/styles.

Acceptance

  • Frozen public controls match system JSC exactly for text and resolvedOptions(), including the repros above.
  • Full DisplayNames + Intl402 test262 has no regressions.
  • Generator determinism, pinned-input provenance, bounded table footprint, OOM, moving-GC, no-GIL, TSan, and full units pass.
  • Any unsupported locale/data family remains explicitly visible rather than silently reported as complete.
9 days ago

The byte-identical English language/region/script/currency subset is now frozen as the performance boundary for #583 in 6f1e25f8. The excluded public differentials remain owned here: zh-Hant language-pattern composition, short standard region wording, and non-English localized currency names. #583 will not claim or conceal those data/semantic gaps.

9 days ago

#583 is complete in 925075c8. It changes only immutable option-state ownership and bounded canonicalization scratch for the byte-identical English subset. The zh-Hant pattern, English short-standard wording, and non-English localized-name gaps documented here are unchanged and remain the next correctness/data boundary.

9 days ago

Pinned-source inspection corrected one frozen expectation before implementation: CLDR-JSON a79b499916d486dca4b0f74fe423ea457705fdd9 defines English zh-Hant as Traditional Chinese, while this macOS system JSC returns Chinese, Traditional. That is a host locale-data-version delta, not a composition defect; the implementation will keep the pinned CLDR result rather than hardcoding Apple output.

The real gaps remain in scope: locale-indexed data for all six types, short/narrow fallback chains, standard-language composition (including English (US)), localized non-English currencies/calendars/date fields, aliases/canonicalization, and explicit code/undefined fallback for locales outside the owned bundle set. Exact JSC controls will be classified against the pinned source so version drift stays visible.

9 days ago

Completed in 2b4449f6.

Implemented:

  • revision-pinned, deterministic CLDR-JSON generation for 51 owned locale bundles and 969 immutable long/short/narrow table arrays across language, region, script, variant, currency, calendar, and date-time-field data;
  • fail-closed source revision and locale-pattern validation;
  • construction-time locale inheritance/indexing, service-specific DisplayNames locale availability, ECMA-402 region/script/currency/calendar/language canonicalization, alias handling, dialect/standard variant composition, localized patterns, and explicit fallback semantics;
  • zig build intl-displaynames-jsc-diff, covering all six types plus styles, canonicalization, dialect/standard composition, and resolvedOptions(). The final public control is 9 byte-exact JSC rows, 11/11 exact resolvedOptions() records, and the two previously documented pinned CLDR/ICU-version deltas frozen exactly on both sides.

Verification:

  • DisplayNames test262: 57 -> 57; Intl402: 3341 -> 3341; 0 regressions / 0 flips.
  • focused normal: 7/7; focused TSan: 7/7, including OOM, bounded-heap GC, and native concurrent reads.
  • full units: 1,683 passed, 1 skipped, 0 failed, 0 leaked across 10 shards.
  • generator regeneration is byte-identical after zig fmt; wrong revision is rejected before input reads.
  • exact-parent ReleaseFast steady checksum remains 139703394; property-lock, backing-allocation/byte, GC-cell, and collection counters are identical.
  • bounded footprint: generated source is 4,093,642 bytes; the ReleaseFast comparison executable is 13,112,040 -> 17,968,360 bytes (+4,856,320, +37.04%).

The original zh-Hant Apple-JSC wording is intentionally not hardcoded: pinned CLDR at a79b4999 owns Traditional Chinese; the differential gate makes that version delta auditable.

Sign in to comment on this issue.