ReviewOS

zig-utils/zig-js

Intl: reclaim DateTimeFormat structural part scratch after materialization

#641
Closed chris opened this 9 days ago ·
9 days ago

Problem

DtfOutput appends formatToParts metadata, translated digits, and copies of stack-formatted numeric fields directly to the Context arena. Callers copy those bytes into managed JavaScript strings, but the intermediate structural graph remains retained for the Context lifetime. NumberFormat now has explicit invocation-local ownership in #640; DateTimeFormat still uses the old implicit lifetime.

This is a bounded-memory and allocator-failure issue, not a semantic shortcut. Part values must remain alive through range comparison, source classification, and managed result construction, then be reclaimed on every success/error path.

Required implementation

  • Give DateTimeFormat structural output one explicit invocation-local owner.
  • Route part arrays, translated digits, and copied stack fields through that owner.
  • Keep text output ownership unchanged unless separately measured.
  • Preserve range shared-prefix/suffix classification, calendar/time-zone output, Temporal conversion ordering, numbering systems, and OOM propagation.
  • Do not raise heap limits, retain a global formatter cache, or weaken the bounded structural-result witness.

Acceptance

  • Allocation-failure coverage exercises Latin and translated numeric parts and copied stack fields with exact teardown.
  • DateTimeFormat unit/range/Temporal and bounded-heap structural-result tests pass normally and under TSan.
  • test/intl402/DateTimeFormat and relevant Temporal formatting subtrees have nonzero exact before→after accounting with zero regressions.
  • Full exact unit shard remains leak-free.

Parents: #475, #480 Related: #638, #639, #640.

9 days ago

Implemented and pushed in d6997880. DateTimeFormat structural part vectors and every dynamic part-byte source (Latin stack copies, translated digits, Chinese year names, and GMT offset names) now use one invocation-local arena retained through range comparison/source classification and managed JS result publication, then deinitialized on every return/error path. Text sink ownership is unchanged.

Evidence on exact Zig dev.1818:

  • DateTimeFormat units: 9/9 normal and 9/9 TSan, zero leaks/races.
  • Allocation-failure witness covers Latin copied fields, Devanagari expansion, Chinese year names, and dynamic GMT offsets.
  • Bounded 4 MiB precise-heap witness performs 512 formatToParts + formatRangeToParts materializations with Chinese calendar and Devanagari digits.
  • test/intl402/DateTimeFormat: 244 -> 244 pass.
  • Seven relevant Temporal toLocaleString subtrees: 95 -> 95 pass.
  • Zero conformance flips and zero regressions.

The full sharded artifact reported zero leaks; its single unrelated, focus-pass-only recovery assertion is tracked with fresh evidence on #628.

9 days ago

Reopened: fresh exact dev.1818 Linux unit shard 1/4 fails the unchanged 4 MiB acceptance witness. The complete stack now reaches DtfOutput.push -> pushCopied -> pushInt -> dtfAppendYearPart -> dtfBuildOutput -> dtfBuildParts, with OOM growing the invocation-local parts list at src/interpreter.zig:27346.

This does not show retained scratch after return—the owner still deinitializes correctly. It shows that the structural scratch graph and managed result graph can exceed the bound while both are simultaneously live during materialization on Linux. The next correction must reduce that exact peak (for example, by proving a streaming/ownership-transfer boundary), while preserving range prefix/suffix classification and without moving input-sized bytes outside heap accounting, raising 4 MiB, or reducing 512 materializations. #648 removed the preceding generic-bind allocation but did not cause this DtfOutput failure.

9 days ago

Follow-up landed in 05bbcaf4. The closed DateTimeFormat emitter now keeps its at-most-24 structural records in caller-owned native storage, with an explicit fail-closed overflow error if a future emitter expands the bound. Immutable CLDR names, punctuation, range separators, era/day-period labels, and fixed zone labels publish through static string cells; translated digits, numeric fields, Chinese year names, and dynamic offsets remain fallible and invocation-owned. The text-only formatter does not carry the fixed structural buffer.\n\nExact dev.1818 evidence:\n- DateTimeFormat units: 9/9 normal and 9/9 TSan, zero leaks/races;\n- unchanged 4 MiB / 512-materialization witness passes normally and under TSan;\n- full test-parallel: 1,745 pass / 1 skip / 0 fail / 0 leaks;\n- test/intl402/DateTimeFormat: 244 -> 244 pass;\n- seven DateTimeFormat-backed Temporal toLocaleString directories: 49 -> 49 pass;\n- zero conformance flips and zero regressions.\n\nKeeping #641 open until fresh isolated Linux unit and TSan shard 1 confirm the host-sensitive cap that reopened it.

9 days ago

Closed by 05bbcaf4.

Fresh isolated Linux confirmation on exact Zig 0.17.0-dev.1818+7051f8e73 is green at the final documentation-only descendant db5d0e5a:

Those are the two shards that previously failed the unchanged 4 MiB DateTimeFormat witness. Local final-candidate evidence remains 9/9 focused units normally and under TSan, the bounded witness in both modes, 1,745 pass / 1 platform skip / 0 fail / 0 leak across full units, and exact DateTimeFormat plus seven Temporal locale subtrees with 293 pass / 0 fail before and after (0 flips, 0 regressions). No public timing claim is made.

Sign in to comment on this issue.