ReviewOS

zig-utils/zig-js

Runtime: make no-GIL heap-cap recovery safepoint-owned

#31
Closed chris opened this 9 days ago ·
9 days ago

Parent: #30 / #1

Goal

Let GC-backed heap-cap allocation-failure recovery attempt an abort-safe no-GIL collection only from a thread/interpreter state that can publish roots correctly, instead of failing closed whenever another shared-realm Thread is alive.

Why this is separate

Context.collectForAllocationFailure() currently has only the Context, while the abort-safe parallel collector needs the current Interpreter as the elected collector and assumes safepoint-like conditions for tracing the collector's own roots. Allocation failure can also happen from arbitrary allocation sites, including future side-store sites that may hold property/element locks. A blind allocator-level retry would be deadlock-prone and could violate the parallel collector's root-publication assumptions.

Current evidence

  • zig-gc calls Binding.recoverAllocationFailure() from Heap.create() after a GC cell slab allocation fails, before the new cell is linked into the heap.
  • Context.collectForAllocationFailure() currently returns false in parallel_js when any JS thread is still running.
  • Context.driveParallelCollection() already has the abort-without-sweep protocol #30 wants to reuse, but it requires the current Interpreter and a safe publication point.

Sketch

  • Add an internal current-interpreter/safepoint-owned recovery seam for active JS execution, covering host evaluate* and spawned Thread interpreters.
  • Allow allocation-failure recovery to try the existing parallel root-publication collector only when the current thread can safely act as the collector and holds no engine side-store locks that the tracer may need.
  • Return ordinary OOM unchanged when the collector cannot be safely elected or cannot converge.
  • Keep side-store allocation recovery out of this slice unless it can prove the same lock-order constraints.

Acceptance criteria

  • A no-GIL heap-cap witness proves a live sibling Thread does not automatically force cell-allocation recovery to fail closed when a safe current interpreter can collect.
  • Abort/no-sweep behavior is preserved when peers do not publish or allocation continues during finish.
  • No generic side-store allocator retry is introduced without a lock-order proof.
  • Focused TSan coverage exercises the recovery path without parked-stack conservative scanning races.
9 days ago

Pushed 8db58b6 (fix oom: recover no-gil cell allocation at safepoints) as the first #31 implementation slice.

What changed:

  • GC cell allocation-failure recovery now carries the current active Interpreter through an internal thread-local capability, so no-GIL recovery only runs from active JS execution.
  • parallel_js heap-cap recovery can elect the existing abort/no-sweep parallel collector instead of failing closed merely because a sibling Thread record is alive.
  • Peers publish roots for an already-elected emergency collector even when routine parallel_midscript_gc is disabled.
  • driveParallelCollection now reports whether it actually swept, so allocation retry only happens after a successful abort-safe collection.
  • Added a trace-sensitive lock-depth guard covering object/environment/promise/iterator side-store locks; recovery still fails closed if allocation happens while holding locks the tracer may need.
  • Added a focused GC-cell heap-cap witness with a live sibling ThreadRecord, plus TSan coverage.

Validation:

  • zig build test -Dtest-filter="parallel_js heap_limit_bytes recovers GC cells while sibling Thread is alive" --summary none
  • zig build test -Dtsan=true -Dtest-filter="parallel_js heap_limit_bytes recovers GC cells while sibling Thread is alive" --summary none
  • zig build test -Dtest-filter="enable_gc: heap_limit_bytes retries ArrayBuffer byte allocation after collection" --summary none
  • zig build test -Dtest-filter="parallel_js (M3): mid-script parallel collector reclaims garbage while threads run, no GIL" --summary none
  • git diff --check

I intentionally kept generic side-store retry out of this slice. The new guard makes that policy executable: if a property/element/binding/promise lock is held, emergency recovery returns ordinary OOM rather than trying to trace through the same lock graph.

9 days ago

Completed by the safepoint-owned no-GIL heap-cap recovery work and verified by fresh full CI.

Evidence:

  • The no-GIL sibling-thread heap-cap witness now exercises a bounded worker OOM and proves the sibling Thread can still join successfully.
  • The fixture is bounded: if the expected frozen OutOfMemoryError is not produced, it fails explicitly instead of looping forever.
  • Focused local validation included the exact test, the CI-shaped unit shard containing it, and the exact test under TSan.
  • Fresh full CI is green for b2e7dbe5e9bc2a27a87ff1bad8be8b0be777e46e: https://github.com/zig-utils/zig-js/actions/runs/29170980761

The broader side-store/lock-order emergency-recovery work remains tracked by #30.

Sign in to comment on this issue.