ReviewOS

zig-utils/zig-js

GC: make mark-work publication OOM-atomic

#888
Closed chris opened this 11 days ago ·
11 days ago

Parent: #476 Related: #480, #493, #884

Problem

The owned zig-gc collector has several mark-work publication paths that set a cell's marked bit before a fallible worklist append and then swallow allocation failure:

  • incrementalBarrierHeader wins claimMark(h), then uses barrier_buf.append(...) catch {} or mark_stack.append(...) catch {};
  • publishCellLocked publishes a born-marked cell, then uses born_concurrent.append(...) catch {};
  • concurrent finish paths use fallible appendSlice/append operations and clear the source worklist even when the destination append failed.

Once claimMark succeeds, a terminal root rescan cannot repair a lost work item: visiting the same cell observes it already marked and does not enqueue it for tracing. The cell itself survives, but white children reachable only through it may be swept. A newly born marked cell whose born_concurrent publication is lost has the same untraced-child gap.

This surfaced while auditing #884's rejection-root insertion barrier. The intrusive rejection FIFO remains a durable Context root, but its Promise payload can hold a wrapper, rejection reason, reactions, and async links that still require tracing. A swallowed barrier-work OOM is therefore not an acceptable publication proof.

Required implementation

  • Make every transition from white/newly-born to trace-pending atomic with durable work ownership: either the trace item is infallibly published, the mark claim is safely rolled back with a concurrency proof, or the collection aborts before any sweep can occur.
  • Make concurrent handoff-to-marker and finish-time worklist transfer preserve the source until destination publication succeeds.
  • Treat worklist reservation failure as an explicit fail-closed collection outcome; never continue to sweep a graph whose marking closure is incomplete.
  • Cover single-mutator concurrent marking, parallel mid-script marking, ordinary stop-the-world marking, nursery remembering, born-cell publication, deferred tracing, and terminal root rescans.
  • Keep barrier fast paths bounded and allocation-free after cycle setup where a live-cell bound proves sufficient; do not add an unbounded emergency pool or a global mutator lock.
  • Land the collector fix in the owned zig-gc dependency and pin/integrate it in zig-js with exact regression evidence.

No-workaround rules

  • No swallowed OOM, leaked marked-but-untraced state, unconditional process abort for recoverable bounded-heap pressure, disabling concurrent/moving GC, retaining all cells, skipping sweep without resetting cycle state, or relying on a later root scan after the mark bit already suppresses tracing.
  • A mark bit alone is not durable ownership of the cell's outgoing graph.

Acceptance

  • Deterministic failing-allocator witnesses hit each barrier, born-cell, worklist-transfer, and finish failure boundary and prove no reachable child is reclaimed or relocated stale.
  • Collector state returns to an exact reusable idle/aborted state after every recoverable failure; a later full collection succeeds.
  • Concurrent and parallel stress force root insertion, black-to-white stores, new-cell graphs, weak/ephemeron work, and queue handoff at capacity boundaries.
  • Suppression-free TSan, zig-gc units, zig-js moving/concurrent GC units, mid-GC/threadfuzz profiles, full units, and leak checks pass.
  • Exact Test262/PR-249/Wasm coverage has zero regressions; performance evidence shows no new per-store allocation or global contention.

Security boundary

Heap pressure and graph shape are script-controlled. A worklist OOM must fail closed without turning a reachable JavaScript object graph into marked shells with reclaimed children, stale pointers, or relocation corruption.

11 days ago

Collector/integration checkpoint:

  • owned zig-gc fix: 0285ef80 aborts incomplete mark-work cycles instead of sweeping marked shells with untraced children
  • zig-js integration: 4a1e5d82 records abort attribution, preserves retry pressure, and makes ordinary/concurrent/parallel/nursery/compaction paths honor the fail-closed outcome

Evidence green so far:

  • full zig-gc units in Debug, ReleaseSafe, and suppression-free TSan
  • focused zig-js normal + suppression-free TSan witnesses for Context retry pressure, nursery scratch OOM, concurrent finish, and parallel M3 abort handling
  • fresh zig-js full normal units after integration: 2,311 passed, 1 skipped, 0 failed, 0 leaked

Keeping #888 open for the required mid-GC/threadfuzz, full TSan/leak, exact Test262/PR-249/Wasm, and performance acceptance evidence.

11 days ago

CI integration fix landed: 1f00f7e5 — fix(ci): pin the mark-work GC contract used by the engine (#888).

Run 34272540372, unit shard 0 failed before testing because CI still selected zig-gc 98edfaa, which lacks the Accounting.aborted_collections, markWorkPublicationFailed and minor_abort contracts now used by zig-js. All nine CI/performance GC checkouts now pin the already-pushed zig-gc 0285ef8, the same revision used by the completed local 2,380-test Debug run and Debug/ReleaseSafe/unsuppressed TSan ABI fixtures.

The dependency audit's stale checkout/action occurrence counts are reconciled with the actual workflow set while retaining exact immutable pins and every classification check. Audit passes: 16 classified edges, zero migration targets, 63 classified tools. Docs-link-check resolves 328 files.

Fresh Linux confirmation is running at 34287554681. This removes the dependency-contract build blocker; it does not claim all #888 acceptance or full CI success yet.

11 days ago

Linux verification after 1f00f7e5: run 34287554681 completed with 52 passing jobs and two intentional skips. Full Debug and TSan units, all no-GIL Debug/ReleaseSafe/TSan corpus shards, normal/TSan mid-GC fuzz, private ABI, documentation, and the dependency audit passed. The only failing job is the reference terminal-disposition audit: local reproduction identifies the stale detached-buffer error message changed by #903, rather than a GC compilation/runtime failure. That exact expectation is being reconciled separately.

11 days ago

Closing as implemented and accepted.

The owned collector fix is zig-gc 0285ef8, integrated in zig-js by 4a1e5d82 and pinned across CI by 1f00f7e5.

Acceptance evidence:

  • Deterministic failing-allocator units cover full/minor collection, concurrent and parallel marking, barriers, born cells, weak/ephemeron work, deferred tracing, finish transfer, relocation, abort, and a successful later collection.
  • Failed mark-work publication is sticky for the attempt and aborts before weak clearing, relocation, or sweep; source worklists remain owned until destination publication succeeds.
  • Concurrent barriers reserve the exact pre-existing live-cell bound before mutators resume. The store fast path adds no per-store allocation and no global mutator lock; its additional state check is an acquire load that lowers to a plain load on supported x86-64/arm64 targets.
  • Linux run 34287554681 passed full Debug and TSan unit shards, Debug/ReleaseSafe/TSan no-GIL corpus shards, Test262 parallel, all configured Wasm profiles, private ABI, mid-GC/lifecycle/default/broad/amplified/verify fuzz, TSan fuzz, and the benchmark harness.
  • That run's only failure was threads-reference-audit, caused by the separately changed detached-buffer diagnostic from #903; no GC build, runtime, sanitizer, corpus, Wasm, or performance gate failed.

The implementation therefore satisfies the fail-closed security boundary without retaining marked shells, disabling moving/concurrent GC, or adding an emergency pool.

Sign in to comment on this issue.