ReviewOS

zig-utils/zig-js

GC: parallelize and shorten shared precise collection pauses

#476
Open chris opened this 9 days ago ·
9 days ago

Parent: #459 Depends on: #461

Goal

Reduce shared precise-GC pause time and collector CPU by making sweep/reclamation and safe phases parallel/incremental where provably correct.

Measured problem

The July 29 moving shared row reaches 110.56 ms maximum pause, 0.80x non-moving-parent throughput, and bounded rendezvous retries. The earlier object-churn profile attributes 87.4% of its eight-lane cooperative pause to nursery sweep, while rendezvous and trace are negligible. A whole-run backing-reclamation prototype reduced sweep only 0.7% and regressed wall time 0.9%, so that rejected design must not be repeated without new evidence.

Scope

  • Re-profile the current precise nursery after 14d94ec0 and separate park, root publication, prepare, trace, copy/rewrite, sweep, finalization, backing reclaim, and resume.
  • Partition sweep/reclamation by independently owned slab/chunk/card ranges with deterministic merge and no shared hot cursor.
  • Move safe bookkeeping outside stop-the-world windows or make it concurrent/incremental with explicit phase/state barriers.
  • Bound helper count, work stealing, scratch, retry, and tail latency; avoid parallel overhead on small heaps.
  • Coordinate all algorithm changes with the owned ../zig-gc sibling and keep stable-ID/relocation contracts exact.

No-workaround rules

  • No longer allocation tranche presented as a collector fix, skipped sweep/finalization, hidden whole-run retention, disabled movement, larger unbounded reserve, or lower collection frequency without equivalent memory evidence.
  • Do not reintroduce the rejected #427 whole-run scheme unchanged.

Pre-registered objective gate

Before the first production optimization lands, run the current clean exact-parent phase/matrix harness and check in or attach a frozen objective artifact naming the targeted rows, pause p50/p95/max, wall/CPU, phase times, helper/scratch/RSS, retry/timeout, and required non-regression rows. Success thresholds and allowed noise are fixed before candidate results are inspected; a methodology correction versions the baseline and preserves the original.

Acceptance

  • Current exact-parent profiles show the post-change phase responsible for the gain; checksums and byte conservation match.
  • The candidate satisfies the pre-registered objective artifact without moving thresholds, deleting rows, or changing work.
  • Shared pause p50/p95/max and wall throughput improve materially with bounded scratch/RSS and no single-mutator regression.
  • Ephemeral, mixed, high-survival, shared, compaction, weak/finalization, fault/OOM, and movement inventories pass.
  • Mid-GC fuzz, lifecycle fuzz, no-GIL corpus, full unit, and suppression-free TSan pass at production time budgets.
  • Raw dated evidence is published only after a clean full matrix.

Foundation

#426, #427, #446, #447; current generation evidence; precise nursery batching at 325a12ba.

9 days ago

Landed f5363e46 — fix(gc): trace suspended activations during recovery (#476).

This slice removes conservative allocation-recovery aborts for safely suspended ordinary generators, async activations, and iterator helpers by tracing under their existing resume/request/state ownership. Running and async-generator activations remain fail-closed on the deferred finish path. It also fixes the precise-root gap for a successful evaluate() result while the host drains microtasks and waits for child Threads.

Evidence:

  • clean origin/main: focused tlsasyncholdcleanup failed 1/1 because the top-level Promise wrapper was collected;
  • candidate: 20/20 focused seeds passed;
  • lifecycle profile: 1,120/1,120 programs passed;
  • mid-GC profile: 450/450 programs passed;
  • full unit: 1,532 passed, 1 expected skip, 0 failed/leaked;
  • TSan: 7/7 focused recovery tests, 5/5 focused lifecycle seeds, and the new keepalive-result regression passed.

This advances the synchronization/correctness foundation for #476; it does not claim the issue pause/throughput acceptance gates or close the roadmap item.

9 days ago

A clean full #460 matrix run found a separate pre-existing precise-root omission after the suspended-activation recovery fix: Function.frame captured VM chains were not traced or relocated. Both f5363e46 and its literal parent reproduced the stale StringCell panic, so this was not introduced by the #476 generator change. The ownership fix and regressions landed in 9d0227a3 via #510; exact shared collections now passes 21/21 repeated samples and CI-equivalent TSan mid-GC passes 90/90 programs.

9 days ago

A second moving-GC root class found while exercising the recovery/benchmark path is now fixed on main by 33150087, closing #511.

Promise jobs had registered interpreter roots, but native helper execution copied Microtask, Reaction, Value, and raw Promise addresses into unregistered Zig locals. Requested compaction could rewrite the registered job and leave those copies stale. The fix runs the registered job in place and adds precise relocatable Promise/Value roots across observable reentry. Debug, TSan, Promise-focused, structural relocation, exact benchmark-reproducer, and full-unit validation all pass.

The remaining post-correctness full-scale Promise memory/time issue is isolated in #512.

9 days ago

The #512 follow-through is complete on main and adds two relevant collector-policy results without claiming #476 acceptance:

  • perf(gc): amortize nursery scans over job roots (#512) charges heuristic nursery cadence against the exact registered Promise-job root frontier. This prevents repeated full-frontier retracing at a fixed 2 MiB cadence while remembered-set failure, forced collection, and explicit smaller thresholds remain immediate. One 60,000-job × 8-lane sample improved from 422.708234500 s to 36.310953209 s measured workload time with checksum 920578560 unchanged.
  • perf(gc): reclaim quiescent promotion debt (#512) uses the last quiescent live baseline to trigger a heap-level full sweep when transient burst promotion debt would otherwise sit below an inflated mid-burst old threshold. The exact seven-sample publication row now completes in 217.76 s real time with all frozen checksums intact.

The final bounded eight-lane row is 36.3% faster than the immediately preceding scheduling slice; maximum RSS is 21.9% lower and peak physical footprint is 47.8% lower. Full units pass 1,537 with one expected skip and no failures/leaks. Focused moving-nursery, remembered-set fail-closed, shared-heap coordination, and promoted-graph reclamation coverage accompanies the changes.

These results remove avoidable trace repetition and cross-sample promoted-garbage retention exposed by a shared precise-root workload. #476 remains open because its parallel/incremental pause objective still requires the frozen exact-parent phase artifact, complete phase attribution, shared pause p50/p95/max improvement, and the full GC/no-GIL/TSan evidence inventory.

9 days ago

#597 is complete in 8e211b2f. Context-embedded root Environment stores now use the sound external-root barrier contract instead of missing managed-owner classification across every slab class. Managed Environment old-to-young edges remain owner-aware; focused incremental/nursery/TSan witnesses and full units pass.

9 days ago

Concurrent completion follow-up #656 is complete in cd217904. Exact finish identity now transfers once-only Wasm/slab/compaction maintenance to a safe quiescent boundary. Linux normal/TSan unit witnesses and normal/TSan mid-GC plus lifecycle fuzz passed; normal mid-GC covered 900 programs with 0 failures. No redundant full sweep, disabled concurrent collector, or timing heuristic was introduced.

9 days ago

Opened #673 after the cooperative shared-nursery witness produced zero GC attempts in two separate eight-shard full runs, while both exact focused reruns passed. The child requires a root cause and deterministic real tranche crossing without lowering thresholds, sleeps, retries, or weakened telemetry.

9 days ago

Deterministic cooperative nursery coverage from #673 landed in fec063f8. Both worker mutators now rendezvous only after shared allocation tracking is armed, preserving the production tranche and every pause/park/phase assertion. Verification: 20/20 Debug, 3/3 eight-shard placements, ReleaseSafe, TSan, and 1767-pass full units with zero failures/leaks.

9 days ago

Closed #694 via d5d63780: reusable block/call Environments no longer rewrite immutable GC ownership while concurrent marking can traverse parent edges. The exact C-API parallel-GC TSan witness and the full 1792-test sanitizer matrix are warning-free.

9 days ago

Current acceptance evidence is not green: Linux f2340f2 non-TSan threadfuzz-midgc reports 3 collection-completion failures among 900 programs (Condition.wait seeds 8/17, ThreadLocal lifecycle seed 12). Tracked in #812. Exact job: https://github.com/zig-utils/zig-js/actions/runs/33137549779/job/98740893155 . Root cause/introducing revision are not established; this must not be treated as a sanitizer race report or fixed by weakening collection requirements.

Sign in to comment on this issue.