ReviewOS

zig-utils/zig-js

Runtime: make heap caps collect/retry before unrecoverable OOM

#29
Closed chris opened this 9 days ago ·
9 days ago

Parent: #24, #27, #14.

While continuing #27 after the catch-binding fix, the PR-249 semantics/oom-one-thread.js witness still fails with a whole-thread/top-level OutOfMemoryError:

FAIL semantics/oom-one-thread.js: OutOfMemoryError: Context heap limit exceeded

The previous #27 slice made local catch (e) under Context.Options.heap_limit_bytes work for focused allocation pressure. The remaining witness shape is different: after the hog thread catches/reports OOM and drops its JS references, later allocations must recover. Today the heap cap is enforced by BudgetAllocator at the context allocator layer. In arena-backed mode there is no individual reclamation, and even enabling GC/parallel mid-script collection locally did not make the witness pass; the cap still reaches an unrecoverable OOM path rather than a collect-then-retry/recover path.

Local probes (not committed):

  • Enabling GC/parallel_js/parallel_midscript_gc for just semantics/oom-one-thread.js changed one run to a raw top-level error.OutOfMemory with no JS exception when stringifying would otherwise print ?, and with a higher 320MiB cap still failed as reserved OutOfMemoryError.
  • This suggests the missing contract is not simple catch binding anymore; it is heap-cap recovery: when a capped GC-backed context hits the cap, the runtime needs a safe collection/retry or emergency-headroom policy before declaring OOM unrecoverable.

Acceptance:

  • For GC-backed capped contexts, allocation pressure attempts an appropriate safe collection/retry before surfacing unrecoverable OutOfMemoryError.
  • The policy is safe with threads: no collection while unscanned running peers can hold native-stack roots, unless using the abort-safe parallel mid-script protocol.
  • Dropping JS references after an OOM lets later allocations succeed once the collector can reclaim unreachable cells/backing stores.
  • semantics/oom-one-thread.js passes or progresses to a more specific non-recovery blocker.
  • Existing focused #27 regressions remain green: optional catch, catch binding, string pressure, and sibling Thread survival.
  • Update docs/threads/limits.md and docs/threads/testing.md to distinguish fail-closed arena caps from reclaimable GC-backed capped contexts.
9 days ago

Pushed b557f38 (test threads: run heap-limit probes with gc) as a #29 setup and probe-realism slice.

What changed:

  • conformance/threads_test.zig now enables GC for any corpus case using heap_limit_bytes, not only for parallel_js or files that literally call gc().
  • semantics/oom-one-thread.js therefore now exercises a GC-backed capped context instead of an arena-backed cap that could never reclaim individual JS cells.
  • docs/threads/testing.md now records that distinction.

Focused validation:

  • zig build test with filter: Context heap_limit_bytes lets sibling Threads survive one Thread OOM
  • zig build threads-reference-probes --summary all
  • git diff --check

The PR-249 witness still fails with the same expected evidence: FAIL semantics/oom-one-thread.js: OutOfMemoryError: Context heap limit exceeded.

So the remaining blocker is now sharper: even in a GC-backed capped context, the runtime still needs a safe collect/retry or emergency-headroom path while the allocating Threads are active. This change does not close #29; it makes the probe test the intended class of failure.

9 days ago

Resolved the PR-249 heap-cap recovery blocker with zig-gc b99de49 and zig-js 78b553a.

What landed:

  • zig-gc now lets an embedder recover from GC cell slab OOM and retry allocation once.
  • zig-gc conservative address-index scratch now uses the aux allocator, so emergency stack-scanning collection does not consume the capped cell backing while trying to free it.
  • zig-js implements the recovery hook by running a conservative full collection only when roots are safe: single-thread, or GIL-mode with peers parked/published. It remains fail-closed for no-GIL running peers.
  • Heap-limit PR-249 probes are GC-backed, and semantics/oom-one-thread.js is promoted into the allowlist.
  • PR-249 coverage is now 231/259 executable files, with 28 executable reference-only files remaining.

Validation:

  • zig-gc: zig build test --summary all
  • zig-js: zig build test with filter heap_limit_bytes allocation pressure
  • zig-js: zig build test with filter Context heap_limit_bytes lets sibling Threads survive one Thread OOM
  • zig-js: zig build test with filter enable_gc: conservative native-stack scan keeps a stack-only value alive
  • zig-js: zig build threads-test -Dthreads-case=semantics/oom-one-thread.js --summary all, now PASS
  • zig-js: zig build threads-reference-audit --summary all
  • zig-js: zig build threads-reference-probes --summary all
  • git diff --check

Follow-up for the remaining broader perfection work is #30: no-GIL emergency recovery and side-store/ArrayBuffer-byte retry policy. Closing this issue because the GC-backed capped-context recovery target and the PR-249 witness are now handled.

Sign in to comment on this issue.