Embedding: preserve checkpoint OOM at host evaluation boundaries

Parent: #480 Related: #474, #495, #885, #886, #887

Observed boundary

While implementing #885's job-execution OOM ownership, a native fault callback resolved its thenable target to another queued thenable, then returned error.OutOfMemory. The direct Interpreter.drainMicrotasks witness observes the abrupt engine error and retains the committed adoption job for exact recovery. Calling through Context.evaluate instead returned success: the host checkpoint catches and discards the error, then another checkpoint can run the retained suffix before the host sees any failure.

Source audit at 9fb57fa01035d7e5712f9283972cb92c48c26bef:

• Context.evaluate uses machine.drainMicrotasks() catch {} and drainFinalizationCleanupJobs() catch {} around waiter handling and again after Thread completion.
• Module evaluation has the same unconditional checkpoint catches.
• The underlying queue transaction in #886 preserves untouched work; these host catches are a distinct error-reporting boundary.

This is an observed synthetic native OOM plus a source audit, not a measured heap-budget exhaustiveness claim. The current #885 implementation converts failures owned by an intrinsic Promise into committed rejection; those successfully delivered rejection outcomes must remain distinct from an unowned checkpoint failure.

Required implementation

• Define and implement the host-facing result of engine OOM during script/module checkpoints, finalization cleanup, waiter completion, and post-Thread cleanup.
• Preserve the original script/module abrupt completion when cleanup also fails; do not replace a rooted primary exception with a secondary checkpoint error.
• Report unowned engine failure through the existing Zig/public C/private ABI error channels appropriate to each pinned contract, without changing exported layouts or conflating ordinary JS job exceptions with an evaluation exception.
• Keep successfully committed Promise rejection as rejection, and preserve the exact untouched queue suffix and GC roots until authorized later execution or teardown.
• Audit the repeated checkpoints so secondary drains cannot silently consume the failure boundary. Never replay an already-run callback, clear a pending queue to recover, or add a retry loop that hides allocator failure.

Acceptance

• Deterministic allocator/native-fault witnesses distinguish top-level success + checkpoint OOM, top-level throw + cleanup OOM, ordinary JS job throw, and a successfully committed OOM Promise rejection.
• Script and module entry points agree on documented error ownership; original exception identity survives moving/concurrent GC and post-Thread cleanup.
• Recovery preserves FIFO/exactly-once callback execution and retained adoption work in direct, nested, and host checkpoints.
• Focused Debug/ReleaseSafe/suppression-free TSan, public/private ABI fixtures, relevant async/thread integration, full units, and affected Test262 diagnostics pass.

Security scope

This is a bounded-heap/embedding error-integrity boundary. The engine must preserve state and report failure accurately under script-controlled allocation pressure; it does not add a sandbox or permission layer.

Related entry point from the same frozen source audit: threadJoinFn in src/jsthread.zig also uses self.drainMicrotasks() catch {} before returning/rethrowing threadResult(rec). Include synchronous Thread.join() in the checkpoint error-ownership matrix. A worker's original thrown result must retain precedence if draining its transferred work also fails; a successful worker result must not silently hide an unowned engine OOM. #887's committed completion/queue ownership remains a separate invariant.

Two additional baseline defects are now measured at 54bb7b0 (issue-896-before.log; 7 filtered tests, 5 pass, 2 fail, no leaks):

1. A script throws a globally retained TypeError("script-primary") after scheduling a reaction that throws RangeError("cleanup-secondary"). Context.evaluate returns error.Throw, but Context.exception is no longer an object; the exact primary error was lost during cleanup.
2. An uncaught-exception monitor forces an actual nursery move with native stack scanning disabled. A following capture callback receives the old pointer rather than the relocated exception. Both callbacks ran once and the test verifies that movement occurred before comparing pointer identity, without dereferencing the stale pointer.

An isolated candidate now retains the script/module primary exception in a pre-reserved completion root and roots a borrowed exception slot throughout nested fatal dispatch, allowing the caller's slot to be rewritten. Timer, next-tick, and private ABI callers use that rooted slot; exported C signatures are unchanged. The candidate is queued for verification, not yet committed or claimed passing.

This is the exception-identity part of #896. Reporting unowned checkpoint OOM, stopping later drains at that failure boundary, and preserving untouched work remain separate outstanding requirements; #897's infallible admitted-waiter completion is relevant to that work.

The exception-rooting candidate at d500e4b now passes all six matching host checkpoint Debug tests (0 failed, 0 leaked). This includes both newly added primary-module preservation and actual nursery relocation during script cleanup. It is not pushed or claimed complete yet.

The remaining checkpoint implementation is being tested as one related change: explicit error results from waiter settlement and timer keepalive, stopping repeated host drains at the first unowned engine failure, preserving a primary script/module exception, and synchronous join error precedence. New witnesses check the exact retained thenable-adoption job and FIFO suffix, recovery without callback replay, actual allocator failure, and the distinction between owned Promise rejection and unowned OOM. Existing #897 harvest-boundary witnesses will now assert the retained native/context suffix at the reported failure boundary before recovery.

Landed fix(runtime): retain primary exceptions across host cleanup (#896) on main as Chris, without trailers.

Validation: six focused Debug tests pass, zero failures/leaks. Affected Test262 comparison is unchanged across all 2,218 selected cases: Promise 677, Atomics 390, async-function statements 74 / expressions 93, async-generator statements 301 / expressions 623, async-arrow expressions 60. All pass before and after; flips 0.

The separate checkpoint-error propagation candidate now passes 14 focused Debug tests, including public C, private VM pending-error preservation, actual allocator failure, script/module primary-error precedence, join, timer and finalization recovery. All 16 waitAsync regression tests also pass. ReleaseSafe, suppression-free TSan and broader integration are still pending. This issue remains open.

Two additional allocator-exhaustion witnesses now fail on the prior code (0 passed, 2 failed, 0 leaked) and pass with the C-boundary candidate (2 passed, 0 failed/leaked). The callback replaces the public Context/hidden private owner arena interface with a failing allocator and actually requests allocation; this reaches the post-checkpoint boxing boundary with no remaining handle capacity.

The public witness additionally verifies that an original thrown object survives cleanup OOM, an attempted subsequent entry performs no script effects while still exhausted, and recovery executes the retained suffix once. The private witness verifies a real pending exception, no second drain before explicit clear, and once-only recovery.

The candidate reserves JSEvaluateScript's single completion handle before user code runs, and creates immutable static-string OOM handles at public Context/private VM construction. Private engine failure publishes the existing pending-exception channel without allocating. Exported signatures/layouts are unchanged. The combined Debug checkpoint filter passes 19 tests with zero failures/leaks; final full-unit, ABI, ReleaseSafe and TSan validation is running before push. Arbitrary private JavaScript-throw boxing outside these engine-error checkpoints is not claimed allocator-exhaustion-safe.

Separately, the earlier checkpoint/identity integration passed all 17 checkpoint, 2 identity and 16 waitAsync ReleaseSafe tests; its 17 suppression-free TSan checkpoint tests passed. Default400, verify300, lifecycle3360, broad80 and amplify30 fuzz programs passed. The mid-GC 900-program run had one collection-completion failure; all failed evidence and six exact-seed baseline/candidate replays are now retained in #907. It is not treated as a clean full mid-GC gate or attributed to these changes without evidence. Finalization callback admission/relocation remains #906.

The implementation batch is pushed to main:

• d0a0d528 — fix(runtime): preserve unowned checkpoint failures at host entry (#896)
• da4b0105 — fix(threads): hand off terminal checkpoint errors with queued jobs (#896)
• 78314ee6 — fix(embedding): admit completion handles before allocation failure (#896)

Together with 65dbb404's primary-exception rooting, these preserve rooted primary completions, report unowned checkpoint OOM, stop repeated drains, retain exact FIFO/adoption ownership, preserve published asyncJoin results, and report exhausted C/private VM engine errors without another allocation. #905's identity/lifetime fix is also integrated.

Final measured validation:

• Full Debug: 2,380 passed, one Linux-only skip, zero failed/leaked.
• Checkpoint witnesses: 19/19 Debug, ReleaseSafe and suppression-free TSan; the two exhausted-handle witnesses were 0/2 before, 2/2 after.
• Identity2 and waitAsync16 pass ReleaseSafe and suppression-free TSan.
• Public C, Home private, global lifecycle, readable-stream and Bun C-API extension fixtures pass Debug/ReleaseSafe/unsuppressed TSan, including their public/private ABI audits.
• Ten affected Test262 subtrees: 2,893 -> 2,893 passes, 0 -> 0 failures; gains 0, regressions 0. Promise677, Atomics390, async-function74/93, async-generator301/623, async-arrow60, module-code599, FinalizationRegistry47, WeakRef29.
• Relevant PR-249: 18/18; default400, verify300, lifecycle3360, broad80 and amplify30 fuzz pass.

The mid-GC 900-program run has one retained collection-completion failure in #907; six seed-2 comparison replays pass but do not establish cause. #906 separately owns finalization record/root admission. Keeping #896 open while finishing the remaining private abrupt-completion boxing audit: arbitrary private JavaScript throws must not be advertised as allocation-exhaustion-safe merely because the private engine-OOM checkpoint now is. No full mid-GC or exhaustive heap-budget claim is made.

Further private-boundary audit found 24 fallback sites that still call the fallible privateSetPendingValue path to report an already-observed allocation failure. A separate-checkout witness is prepared for exported Latin-1 property access and module-error reporting under exhausted context/owner handle allocators. It has not been run yet; the current serialized validation is finishing #906. The intended fix reuses the VM's existing immutable OOM exception cell.

The same audit identified a distinct precise-GC alias/lifetime concern for ordinary pending exceptions and the cached termination exception; that source finding is tracked in #908. Rewriting Context.private_pending_exception_root does not visibly rewrite the separate exception/value boxes, and clearing pending state drops the root while the termination cache remains. No stale dereference is claimed until measured.

The broader arbitrary-JavaScript-throw admission work remains open here: encoding and boxing after a primary throw can still fail. It must preserve canonical value identity and reentrant admission, not merely replace a lost primary exception with OOM.

Landed 54b1d01c (fix(embedding): publish admitted OOM exceptions (#896)).

This removes the remaining allocation-after-failure path from all 24 private-ABI OOM fallbacks. They now publish the VM group's immutable, construction-time oom_exception cell directly instead of calling the fallible value encoder/boxer after a realm or owner arena has failed.

Evidence:

• New forced-exhaustion witness: pre-fix 0/1 (pending exception remained null), post-fix 1/1.
• The witness exhausts both realm and VM-owner arenas and covers Latin-1 property-key conversion plus module-loader error construction.
• TSan witness: 1/1.
• Host checkpoint suite: 19/19.
• Private pending-exception ownership: 1/1.
• C API audit: 117/117 declarations complete, all public symbols present.
• Home private ABI: 447/447 private symbols implemented; 392/392 value shims linked and runtime matrix passed.
• Exact pre-fix/candidate Test262 test/language/module-code: 599/599 -> 599/599, 0 flips.

I am leaving #896 open because the broader arbitrary-JavaScript-throw admission work remains separate from this completed OOM slice.

Completed the remaining arbitrary-JavaScript-throw admission boundary in 7824ed3d (fix(embedding): pre-admit private throw cells (#896)).

Each private VM now admits one canonical value projection, one exception cell, and exact identity-map capacity before a boundary can execute user code. An uncaught object or string throw consumes those cells without allocating after the throw. The next private boundary replenishes consumed cells before entry. Published addresses are never reused, primitive throws avoid an unnecessary projection, existing object handles remain canonical, and an admission failure publishes the VM's immutable OOM exception before user code runs.

The exact-parent witness installs an arbitrary thrown object, exhausts both the realm and VM-owner arenas inside the native callback, then returns Throw:

• parent 69400b3f: 0/1, MissingException;
• candidate: 1/1, repeated twice in one VM;
• the second throw gets distinct exception/value cells, while the first projection remains readable and exact;
• focused TSan: 1/1.

Validation:

• host checkpoint suite: 20/20;
• private exception-root cases: 6/6;
• pending-exception cases: 3/3;
• private OOM cases: 2/2;
• Home private ABI in Debug, ReleaseSafe, and unsuppressed TSan: 447/447 private symbols and 392/392 runtime shims;
• public C audit/fixtures: 117/117 declarations complete and present.

Together with d0a0d528, da4b0105, 78314ee6, 65dbb404, and the admitted OOM fallback in 54b1d01c, this satisfies the script/module checkpoint ownership, primary-exception identity, FIFO recovery, post-Thread cleanup, public/private ABI, and arbitrary private-throw requirements. Finalization ownership and the historical mid-GC failure remain separately owned by #906 and #907; they are not hidden by this closure.