Threads: make completion rooting and join settlement OOM-atomic

Parent: #482 Related: #474, #480, #493, #885, #886

Problem

A completed Thread’s return value is not published into ThreadRecord.result until after several GC-capable operations. Because a running peer’s native stack is not conservatively scanned by the parallel collector, threadMain tries to bridge that interval with:

machine.gc_temp_roots.append(machine.arena, out) catch {};

Its own comment identifies this as the return value’s only precise root. If the append fails, microtask drains, waiter settlement, timers, or task pumping can advance collection and reclaim/relocate the value while it is still used.

After publishThreadCompletion, rec.result and rec.settling_joins are already traced under join_mutex; the later ignored temp-root appends are therefore redundant, not the only owners. However, the corresponding promise.resolve/reject calls and final drains also use catch {}, and finishThreadJoinSettlement publishes joins_settled = true even if a pending asyncJoin Promise was never settled or its required reaction publication failed.

#885 owns general Promise state-to-microtask publication. #886 owns already-existing queue transfer/restoration. This issue owns Thread completion’s pre-publication result root and its completion-state transaction.

Required implementation

• Publish the Thread result into a traced, join-mutex-protected ThreadRecord slot before any post-call GC-capable drain, without making done observable early.
• Keep rec.result and rec.settling_joins as the authoritative post-publication roots; remove redundant best-effort temp-root appends or turn them into proved non-fallible optimization-only storage.
• Publish done and joins_settled only at states that accurately describe the result, pending join capability settlement, and rerouted reaction ownership.
• Propagate or materialize OOM through each asyncJoin result exactly once; never silently leave it pending.
• Compose with Thread termination, nested asyncJoin, thenable assimilation, local-to-realm queue closure, concurrent marking/moving GC, and teardown.
• Preserve the current synchronous join rule: it waits for done && joins_settled && exited.

No-workaround rules

• No swallowed required root/settlement error, reliance on an unscanned running stack, disabling parallel GC, global GIL, leaking Thread records, capping valid pending joins, retrying user callbacks, or marking completion early.
• A barrier is not a substitute for durable rooting across later collection progress.
• Do not retain redundant fallible root copies as a false safety claim when the ThreadRecord already owns the post-publication roots.

Acceptance

• A deterministic allocator-fault witness proves the pre-publication result remains rooted when the temp-root allocator is exhausted.
• Forced concurrent GC between function return, each drain/waiter step, completion publication, join settlement, and Thread exit cannot reclaim or relocate an unpublished result or Promise.
• Every pre-existing asyncJoin Promise fulfills/rejects exactly once, or receives the exact engine OOM completion, before joins_settled becomes observable.
• Root tracing/relocation tests prove rec.result, pending_joins, and settling_joins ownership at each state transition.
• Nested joins, thenables, throwing Threads, termination, waitAsync/property-async rerouting, and late local-queue closure preserve order and liveness.
• Bounded heap, moving/parallel GC, lifecycle/mid-GC/threadfuzz, full units, and suppression-free TSan pass without leaks or suppressions.
• Exact Promise/async/Thread Test262 and PR-249 coverage has zero regressions; documentation makes no unsupported recovery guarantee.

Security boundary

Thread return values and pending join counts are script-controlled, and allocation pressure can coincide with concurrent marking. Required precise roots must be fail-closed engine invariants, not best-effort bookkeeping.

Source-level ownership proof for the implementation:

• Binding.traceRoots marks rec.result, rec.pending_joins, and rec.settling_joins unconditionally while holding join_mutex; it does not require done.
• Therefore the successful-call path can replace its fallible gc_temp_roots.append(out) catch {} with an allocation-free publication of out to rec.result under join_mutex, preceded by the ordinary insertion barrier, while leaving done = false. Neither synchronous join() nor asyncJoin() exposes the provisional slot because both branch on the locked done state.
• The thrown path already retains its exception in the active Interpreter until completion publication; the reserved OOM value is a Context root. It needs the same explicit state proof but not a speculative allocation.
• After publishThreadCompletion, the local pending_joins slice aliases the same backing stored in rec.settling_joins, so the ignored result/join temp-root appends should be removed rather than “fixed”; they add failure points without adding ownership.
• finishThreadJoinSettlement must remain gated on successful settlement of the complete frozen snapshot. Making that failure path exact depends on #885’s Promise-to-microtask transaction; queue rerouting/restore remains #886.

No code is being mixed into the currently running #875 revision.

Partial implementation landed on origin/main: fix(threads): preserve completion roots and queue handoff (#886, #887) (paired with #886).

ThreadRecord.result now receives successful values and the exact body exception/OOM completion under join_mutex before any post-call drain, waiter settlement, timer, or task pump. done remains false during those checkpoints. This also fixes a separate observable bug: a throwing microtask previously overwrote the body exception before it was copied to the completion record.

The record and settling_joins remain the authoritative post-publication roots; redundant best-effort temp-root copies are removed. Settlement iterations and synchronous join reload the result after GC-capable work. The new allocator-fault witness exhausts temp-root storage, then proves result/child retention with native stack scanning disabled before done, tests relocation, and proves collection after ownership is removed.

Validation on the committed source:

• Full Debug unit suite: 2,326 passed, 1 skipped, 0 failed, 0 leaked (2,327 linked), across 10 shards.
• 19 focused tests pass in Debug, ReleaseSafe, and suppression-free TSan (TSAN_OPTIONS=halt_on_error=1).
• Seed-1 no-GIL threadfuzz: 400/400 default, 300/300 exact-result verify.
• Exact-parent Test262 diagnostics are identical: Promise 677 → 677, async-function statements 74 → 74, expressions 93 → 93, async-generator statements 301 → 301, expressions 623 → 623, async arrows 60 → 60. 1,828 → 1,828 passing, flips 0, regressions 0 in those measured subtrees.
• Formatting and diff whitespace checks pass. Local transcripts, before/after diagnostic logs, source hashes, and the full shard plan/logs are retained at /tmp/zig-js-thread-transfer-evidence/ (local evidence, not a portable checked-in report).

Still outstanding: asyncJoin resolve/reject errors are still swallowed, and early worker setup failure still lacks a complete pending-join settlement transaction. Those paths must deliver an exact failure or commit thenable resolution/job ownership before joins_settled is published. The broader forced-GC, startup/termination, PR-249, lifecycle/mid-GC, and full TSan acceptance checks also remain. This issue stays open; the landed root fix is not a claim that those OOM guarantees are complete.

The remaining completion transaction is now implemented locally and in validation; it is not yet committed.

• Each asyncJoin capability reserves a settlement descriptor in its final realm/host queue before publication. Failed resolution retains ownership of that slot. The caller reloads the authoritative Promise root and commits the exact engine OOM rejection without allocating or retrying a then getter.
• Promise settlement now has a shared allocation-free locked commit. Ordinary settlement still performs its fallible reservation before changing state; prepared completion invokes the same commit with already-owned capacity.
• Both worker startup allocation failures are removed by preparing the local queue and reserving an active-interpreter registry slot before OS-thread spawn. Other registrations respect outstanding reservations. The worker initializes its interpreter after the GC registration gate, then activates its reserved slot without allocation.
• The entry callable and argument values are now ThreadRecord roots from before spawn through the body call, including tracing and relocation. A worker waiting to register cannot rely on an unscanned native argument stack.
• Final settlement no longer swallows resolve/reject failures, and joins_settled asserts every prepared token has been consumed before releasing the snapshot roots.

The initial Debug witnesses pass: competing registry growth under exhausted allocation; entry-root retention/relocation with native scanning disabled; 17 pending joins under first-root, second-root, and GC-capable then-getter OOM; allocation-free thrown-result rejection; delayed thenable adoption; and a throwing getter that reentrantly calls asyncJoin. The JS cases run in both GIL modes. Broader TSan/ReleaseSafe, full-unit, conformance, PR-249, and fuzz checks are underway sequentially.

One acceptance clarification: joins_settled must mean that the snapshot's resolution/rejection and required job publication have committed. A Promise adopting a thenable may legitimately remain pending after Thread exit; synchronous join must not wait for an arbitrary future thenable result. The new test explicitly proves that join returns while adoption is pending and that all 18 early/late capabilities later deliver the exact result once.

Landed on origin/main: fix(threads): commit prepared asyncJoin completion (#887).

The completion transaction now reserves required publication before an asyncJoin capability escapes. Fallible root registration and then lookup retain that reservation; an engine OOM becomes the exact rooted OOM rejection with no Promise/queue growth and no replay of user code. The shared locked settlement commit preserves ordinary Promise scheduling as well.

Pending joins reserve the final realm/host destination. An already-completed Thread preserves the caller's current queue, including worker-local thenable jobs. joins_settled is published after every token is consumed and the final worker queue is handed off; an adopted thenable can legitimately remain pending afterward.

Both startup OOM paths that discarded pending joins have been removed: queue storage and an active-interpreter registry slot are prepared before OS-thread spawn. Ordinary registrations respect those reserved slots. The worker initializes its interpreter after the GC gate, then registers without allocation. The entry callable and arguments remain traced/relocated ThreadRecord roots until the body result is published.

Six new regression tests cover 17-capability OOM completion at both root allocations and a GC-capable then getter, exact thrown-result rejection under exhaustion, deferred adoption, reentrant throwing getters, late-join worker affinity, registry reservation pressure/cancellation, and startup root retention/relocation. GC tests disable native-stack fallback where relevant.

Validation on the committed executable source:

• Full Debug units: 2,332 passed, 1 skipped, 0 failed, 0 leaked (2,333 linked).
• 35 distinct focused tests pass in Debug, ReleaseSafe, and suppression-free TSan (TSAN_OPTIONS=halt_on_error=1).
• PR-249 lifecycle/async-join.js and lifecycle/nested-threads.js: 2/2 in GIL and 2/2 in no-GIL.
• Seed-1 threadfuzz: 400 default, 300 exact-result verify, 80 broad, 900 mid-GC subcases across 20 seeds, 3,360 lifecycle subcases across 60 seeds. 5,040 generated programs, zero failures.
• Exact-parent Test262 diagnostics are identical: Promise 677 → 677, async-function statements 74 → 74, expressions 93 → 93, async-generator statements 301 → 301, expressions 623 → 623, async arrows 60 → 60. 1,828 → 1,828 passing, flips 0, regressions 0 in these measured subtrees.
• Formatting/whitespace checks pass. Local logs, source hashes, clean tracked dependency commits, validation driver, and full shard plan/logs are retained in /tmp/zig-js-887-evidence/ (local evidence, not a portable checked-in report).

Keeping #887 open for remaining acceptance work, including full-suite TSan and explicit moving-GC fault injection across the completion/checkpoint boundaries. General Promise job-execution OOM after successful job publication remains tracked in #885. A separate source-review finding about capped admission's completion-state lock is now #895; that test-only path is not covered by the default-cap test claim above.

Closing as implemented and accepted.

The production transaction landed in a8f1b3cc and 9fb57fa0; the final real-moving-GC witness landed in 69400b3f.

Final acceptance evidence:

• Temp-root exhaustion publishes the return graph into ThreadRecord.result while done remains false; normal and actual moving collections retain it, rewrite both parent and child pointers, preserve the exact value, and reclaim it after record ownership clears.
• The new production-compactor witness passes in Debug and TSan. The focused completion set is 4/4, including a mid-script parallel-GC thrown completion and a body exception surviving a throwing microtask drain.
• The 17-capability settlement matrix covers first-root, later-root, then-getter, and thrown-result failure phases; every capability receives one exact completion before joins_settled, with no outstanding reservations.
• Startup callable and argument roots are retained and relocated before worker registration; completion, pending-join, settling-join, GIL-task, and property-waiter relocation mirrors their traced roots.
• PR-249 lifecycle cases pass in GIL and no-GIL modes. The landed validation covered 5,040 deterministic default/verify/broad/mid-GC/lifecycle fuzz programs with zero failures.
• Linux integration run 34287554681, which contains the production #887 commits, passed full Debug and TSan unit shards, every Debug/ReleaseSafe/TSan no-GIL corpus shard, all configured threadfuzz and TSan-fuzz profiles, and private ABI. Its sole unrelated failure was the #903 detached-buffer reference expectation.
• Fresh exact-parent Promise/async matrix after the final test: Promise 677/677, async-function statements 74/74, expressions 93/93, async-generator statements 301/301, expressions 623/623, async arrows 60/60 on both sides: 1,828/1,828 -> 1,828/1,828, 0 flips.

The completion state now has durable root ownership and prepared settlement ownership at every fallible boundary, without a global GIL, pending-join cap, callback replay, or leaked Thread record.