ReviewOS

zig-utils/zig-js

WeakMap/WeakSet: preserve proportional identity lookup across moving GC

#523
Closed chris opened this 9 days ago ·
9 days ago

Parent: #475 Related: #493, #480, #477, #513

Defect

WeakMap/WeakSet keep an auxiliary pointer -> entry position table. Moving collection clears that table because every live key address changes, but ordinary get/has do not rebuild it. Every subsequent lookup therefore linearly scans the complete live entry array until a mutation happens to repopulate an individual key. A wide live weak collection becomes quadratic under a full pass after compaction.

Index insertion also currently swallows allocation failure after publishing the entry. Correctness falls back to the same linear scan, but the collection can remain partially indexed indefinitely. Dead-key pruning swap-removes entries while retaining historical list and hash capacity, so a long-lived empty WeakMap can retain its peak backing.

Required change

  • Add deterministic 1K/2K/4K post-moving-GC WeakMap/WeakSet lookup witnesses with exact object/Symbol, hit/miss, value, and survivor checksums.
  • Use relocation-stable exact key identity for an authoritative lookup index; preserve exact identity on hash collision and use a securely seeded hash policy rather than an attacker-predictable collision surface.
  • Reserve entry and index storage before publication so OOM leaves the collection exact, unchanged, and still usable; no partially authoritative index.
  • Keep GC ephemeron semantics exact: weak keys never become strong, dead keys prune before relocation, live keys/values relocate, and the index remains valid after movement without pointer repair or a full lookup scan.
  • Bound historical backing after GC removes most or all keys, while preserving the old exact state if an optional shrink allocation fails.
  • Cover WeakMap and WeakSet object/non-registered-Symbol keys, duplicate/update/delete, moving nursery/full collection, OOM, GC pruning, no-GIL, and TSan.
  • Record exact-parent CPU, allocation, and peak/retained-memory evidence with an ordinary no-movement weak-collection control.

No-workaround rules

No disabled moving GC, forced strong retention, address pinning, property-count limit, lookup-time full rebuild, swallowed semantic OOM, global key table, predictable collision shortcut, stale pointer key, or benchmark-specific engine path.

9 days ago

Implemented and published on main.

Commits:

  • bfc795fd — deterministic 1K/2K/4K mixed object/non-registered-Symbol post-moving-GC witnesses plus the ordinary 4K no-movement control.
  • 6989099b — relocation-stable authoritative WeakMap/WeakSet indexes with secure per-collection hashing, failure-atomic list/index reservation, exact relocation/pruning, and failure-safe backing shrink.
  • 39eeb362 — exact-parent CPU/allocation/peak/retained evidence.

Correctness and safety:

  • zig build test-parallel: 1,563 passed, 1 skipped, 0 failed, 0 leaked.
  • WeakMap test262: 141 → 141; WeakSet: 85 → 85; zero flips/regressions.
  • Focused TSan weak suite: 15 passed; separate no-GIL disjoint mutation and moving-nursery mixed-key witnesses passed under TSan.
  • OOM witnesses prove entry/index insertion is failure-atomic and failed prune compaction retains the exact old index.
  • GC pruning releases an empty historical list/index to capacity 0 and shrinks a 512-entry high-water mark to bounded survivor capacity.

Exact-parent evidence (7 order-balanced pairs per row, no discarded samples):

  • post-move retired instructions candidate/parent: 0.726× at 1K, 0.565× at 2K, 0.395× at 4K;
  • 4K post-move wall median: 12.694 ms → 5.316 ms (0.419×);
  • ordinary 4K no-movement control: 1.009× instructions, 1.012× wall, isolating the gain to removal of the old post-relocation linear fallback;
  • observed 4K post-move retained RSS: 1.009×; peak RSS: 1.009×;
  • fixture backing: bounded +86,312 bytes for 8,192 total weak entries, with 0 additional allocation calls and 0 additional GC cells;
  • scored lookup allocation deltas are identical parent/candidate, so lookup neither rebuilds nor allocates proportionally.

Evidence reports are under docs/.data/exact-parent-weak-*. The implementation does not disable GC, retain or pin weak keys, impose count limits, rebuild on lookup, swallow OOM, use a global key table, expose predictable bucket placement, or keep stale moving pointers.

Sign in to comment on this issue.