ReviewOS

zig-utils/zig-js

GC: enable automatic compaction at shared precise safepoints

#446
Closed chris opened this 9 days ago ·
9 days ago

Parent: #134 Extends: #144 Related: #145, #433 Blocks: #147, #246

Explicit fail-closed compaction is complete, but the README removal gate still requires production policy that can schedule movement automatically and coordinate it safely with concurrent/shared-realm execution. This issue owns automatic triggering and shared precise stop windows; it does not weaken the existing raw-pointer or conservative-stack fail-closed rules.

Acceptance

  • Define a deterministic fragmentation/retained-backing trigger with bounded hysteresis and exact telemetry; no benchmark-source recognition.
  • Schedule compaction from ordinary allocation/collection pressure and consume it at every declared precise interpreter, baseline, optimizer, embedding, and shared-realm safepoint.
  • Coordinate all live shared-realm mutators and concurrent/minor collectors before movement, with bounded park/abort behavior and no movement under conservative or opaque native frames.
  • Preserve stable identity, weak/finalization ordering, handles, inspector/debugger roots, Wasm roots, and exact resumable JIT state across the automatic stop window.
  • Make OOM, unsupported stacks, active host callbacks, and concurrent collector conflicts failure-atomic and observable through telemetry rather than silent partial movement.
  • Add single-mutator, no-GIL, concurrent-marker, native callback, optimizer, fault-injection, leak, TSan, and long-running fragmentation coverage.
  • Publish an exact-parent fragmentation/pause/throughput matrix and update the release compatibility inventory and README only from committed evidence.
9 days ago

Implemented the next slice in 3b01795e (feat(gc): schedule quiescent automatic compaction).

What changed:

  • Added automatic quiescent full-GC compaction when fragmented/retained slab backing has at least 512 KiB reclaimable pressure.
  • Added telemetry for automatic requests, attempts, outcomes, moved cells, and moved bytes.
  • Added a regression test proving the full-GC pressure path moves reachable cells and trims backing.
  • Updated README/docs/inventory/release matrix wording to distinguish the now-enabled quiescent policy from the still-open shared/mid-script automatic compaction evidence gate.
  • Hardened the Test262 runner so each subtree is driven in an isolated process; parent-side FD/spawn pressure can no longer hide late built-ins as 0/0.

Verification:

  • zig build test262 --summary all: 48,506/48,506 valid, 4,669/4,669 negative, skipped 0, parse/runtime/host failures 0.
  • zig build test -Dtest-filter="automatic quiescent compaction" --summary all: passed.
  • zig build test -Dtest-filter=compaction --summary all: passed.
  • zig build gc-relocation-inventory-check release-compatibility-check --summary all: passed.
  • zig fmt --check src/context.zig conformance/test262.zig: passed.
  • git diff --check: passed.

Remaining on this issue: shared/mid-script automatic compaction evidence and production scheduling at every declared shared safepoint remain open; the docs now say that explicitly instead of claiming more than this commit proves.

9 days ago

CI follow-up pushed in 03606d01 (fix(build): support lowercase Zig optimize tags). The preceding compaction slice exposed a Zig 0.17 dev transition: local dev.1441 still names builtin.mode tags in title case, while CI dev.1471 uses lowercase tags. Debug-only context/C-API ownership checks now compare the tag name case-insensitively, preserving the declared dev.956+ window. Verified with zig build, zig build threads-test-bin -Dtsan=true, and zig build threads-test-bin -Doptimize=ReleaseSafe. Flips 0 test262 cases; no #446 acceptance item changes.

9 days ago

Shared precise-stop compaction is implemented and pushed in e607a3b7 (feat(gc): compact at shared precise safepoints).

What changed:

  • Cooperative nursery pressure now schedules the existing deterministic >=512 KiB reclaimable-backing policy for automatic movement.
  • A shared moving collection proceeds only after every active interpreter publishes the same request generation from a declared precise + relocation-safe checkpoint.
  • Native waits, conservative stacks, opaque frames, and incomplete rendezvous remain fail-closed; the stop aborts after the existing bounded 100 ms window and leaves the request pending for a later safe checkpoint.
  • Automatic telemetry now separates shared attempts/timeouts and records shared rendezvous total/max, pause total/max, outcome buckets, moved cells, and moved bytes.
  • A real no-GIL test drives two Thread workers plus the main lane through optimized native checkpoints, proves a protected live object changes address without changing identity/content, and verifies collector/request cleanup.
  • The relocation inventory now says shared mid-script movement is enabled only at declared moving safepoints, while the exact-parent publication benchmark remains an open release gate.

Verification:

  • zig build test -Dtest-filter=compaction --summary all: 11/11 passed, 0 leaked.
  • zig build test -Dtsan=true -Dtest-filter="moving stop" --summary all: 4/4 passed, 0 leaked, no TSan warnings.
  • zig build test -Doptimize=ReleaseSafe -Dtest-filter="moving stop" --summary all: 4/4 passed, 0 leaked.
  • zig build gc-relocation-inventory-check release-compatibility-check --summary all: passed (8/12 release gates green).
  • python3 tools/docs-link-check.py and bun run docs:build: passed (81 link-checked files; 48 built pages).
  • zig fmt --check and git diff --check: passed.
  • zig build test-parallel: the new shared-compaction test passed; four unrelated assertions failed and each reproduces unchanged on unmodified main (three Class-A artifact warm-up assertions and one heap-bounded recursion assertion).
  • Flips 0 Test262 cases; no Test262 expectations changed.

Remaining before closing #446: concurrent-marker/native-callback/fault-injection/long-running coverage and a committed exact-parent fragmentation/pause/throughput matrix. The release matrix and README notice intentionally remain open until that evidence is published.

9 days ago

56dfc7d6 extends the shared exact-stop policy with automatic moving-minor work (related #447): shared allocation pressure now wakes optimizer safepoints through a distinct movement word, exact peers relocate the survivor set, opaque peers remain bounded/fail-closed, and timeout retry is deterministic. The implementation also adds failure telemetry and broad root/weak/handle/OSR coverage. Full verification: 1,458/1,458 units; 8/8 focused Debug, ReleaseSafe, and TSan; 0 leaks. The test/evidence checklist items remain open pending the dedicated concurrent-marker/native-callback/fault/long-run audit and exact-parent fragmentation/pause/throughput publication.

9 days ago

Committed publication evidence is now complete for this gate slice:

  • e1a51b0b publishes 112 exact moving/non-moving generation rows from clean benchmark commit 7e71eecf, with pause, throughput, retained-backing, promotion, copied-cell/byte, failure, and no-GIL rendezvous telemetry.
  • The accepted matrix records maximum elapsed RSD 13.32%, zero movement failures, 480.42 MiB copied across moving age-three rows, and 14 shared moving minors completed after 12 bounded retries (at most two per row).
  • 8f7ea6e0 advances #447 to green and regenerates the README/release inventory at 11/12 gates.
  • bun run docs:build passed all 48 pages; zig build release-compatibility-check passes.

The exact-parent evidence checklist item is checked. #446 remains open only for the explicit concurrent-marker/native-callback/fault-injection/long-running coverage audit.

Sign in to comment on this issue.