ReviewOS

zig-utils/zig-js

Atomics: make waitAsync completion OOM-atomic

#897
Closed chris opened this 9 days ago ·
9 days ago

Parents: #474, #482 Related: #480, #493, #885, #886, #887, #896

Source-level ownership finding

At 9fb57fa01035d7e5712f9283972cb92c48c26bef, both waitAsync completion paths remove their last completion record before a fallible Promise resolution whose error is discarded:

  • Property-mode settlePropAsync calls promise.resolve(... ) catch {} after notify/expiry has detached its ticket from g.prop_async, then unconditionally frees the ticket/key. Queue routing is protected against Thread exit, but root registration and settlement publication can still fail.
  • Shared-buffer Interpreter.settleAsyncWaiters harvests the native outcome, removes the matching entry with takeAsyncWaiter, calls promise.resolve(... ) catch {}, and finally abandons/clears remaining native/context waiter state.

Consequently, an allocation failure at completion can discard the sole notification/timeout outcome while leaving the already-exposed Promise pending. #885 makes an individual Promise publication transactional, but these callers currently discard the operation that would retry or complete that transaction. #896 owns reporting of unowned host-checkpoint errors; reporting alone cannot restore a destroyed waiter ticket.

This is a source audit, not yet a deterministic failing-allocator measurement.

Required implementation

  • Give each admitted asynchronous waiter an exact completion/publication owner before exposing its result Promise.
  • Preserve the harvested outcome and a precise, relocatable Promise root until resolution/rejection and required reaction publication commit. Do not free/drop the ticket or consume the last native outcome first.
  • Evaluate reuse of #887's prepared settlement API, preserving the existing queue affinity, notify/timeout ordering, and worker-to-realm queue handoff from #886.
  • Make admission failure, not-equal/immediate-timeout paths, notify/expiry races, completion, cancellation, and Context/Thread teardown release each token/root/ticket exactly once.
  • Preserve lock order across agent state, property waiters, Thread completion, queues, and Promise state. Do not resolve allocating work under a lock that GC/recovery must reacquire.
  • No swallowed OOM, permanently pending orphan, duplicated outcome, callback replay, arbitrary waiter cap, emergency allocation pool, or global serialization.

Acceptance

  • Deterministic fault witnesses cover property and shared-buffer waitAsync, notify and timeout, empty/full destination queues, and multiple harvested outcomes (including the fixed harvest-buffer boundary).
  • Every failure proves exact native/context ticket membership, Promise state, reaction ownership, queue ordering, and eventual once-only completion after recovery.
  • Moving/concurrent GC and bounded-heap recovery retain and rewrite detached completion roots through publication.
  • No-GIL notify/timeout/Thread-exit races, teardown, relevant corpus cases, threadfuzz, full units, suppression-free TSan, and affected Test262/private ABI checks pass.

Security scope

Waiter fanout and memory pressure are script-controlled. This is engine state integrity under resource exhaustion, without adding an embedding sandbox or permission layer.

9 days ago

Additional source audit on d4c2a63e found that the registration transaction must be fixed alongside completion:

  • The shared-buffer path publishes a native ticket before creating its Promise capability, appending the engine entry, or finishing the result object. A failure can therefore leave a native ticket without a returned capability.
  • On engine-list append failure, it calls agent.abandonAsync(owner), removing every existing waiter for that owner rather than cancelling only the new ticket.
  • agent.waitAsyncEnqueue currently maps native list/ticket allocation failures to timed_out. Resource failure must not manufacture a timeout outcome.
  • Detached property completion arrays and harvested shared-buffer entries are also outside their previous precise root tables before the allocating resolution call.

These remain source-level findings; deterministic fault witnesses are still needed. The implementation should prepare exact admission/root/publication ownership, preserve primitive ok/timed-out outcomes through infallible commit, and cancel only the affected ticket. Queue reservations need explicit handling across worker exit: #886's transfer contract requires the source queue to have no live reservations. Simply carrying a token that still points at a destroyed worker queue would be incorrect.

9 days ago

The property completion gap now has a deterministic failing-allocator witness: with fresh exhausted temporary-root storage and destination capacity already available, settlePropAsync consumes/frees the detached ticket but leaves its Promise pending (expected .fulfilled, found .pending). The test reports no allocator leaks; the failure is lost completion ownership.

An isolated candidate has passed initial Debug checks for shared-buffer completion under exhausted temporary-root storage, engine-list admission OOM preserving an older native waiter (the later notify still wakes exactly one), and intrinsic Promise/result-object semantics despite a replaced global Promise and inherited async/value setters.

This candidate is not landed or complete. Property completion still needs owned queue reservations across notify/timeout and Thread exit, together with precise roots for detached tickets. Native metadata-allocation fault coverage, moving-GC ownership, and the full no-GIL/TSan matrix remain to be verified. Work is isolated from the #885/#895 validation snapshot.

9 days ago

The isolated candidate now passes 16 focused Debug waitAsync tests, 0 failures, 0 leaks (issue-897-fourth-probe.log). The previous deterministic property-completion witness now fulfills the exact target under exhausted temporary-root storage and publishes its dependent reaction.

The candidate gives property waiters prepared local/host publication slots, an intrusive FIFO completion batch registered as an interpreter root before detachment, and a join_mutex-protected owner index that includes detached tickets during Thread exit. Completion transfers/cancels each reservation without allocating and removes the precise root before freeing native metadata. Notify/expiry keep stable table compaction.

New passing witnesses cover:

  • a detached property ticket migrating with every local reservation at Thread exit, followed by cancellation of the still-pending siblings;
  • a real nursery move with native-stack scanning disabled, rewriting both nested property completion frames and the harvested shared-buffer completion root;
  • 33 shared-buffer notifications across the fixed 16-outcome harvest boundary, followed by exact dependent Promise completion;
  • primitive prepared settlement without queue/root allocation, including duplicate completion against a full destination queue.

Two initial new fixtures incorrectly installed undefined detached handlers; those were corrected to ordinary dependent Promises before the 16/16 run. Additional native-admission allocation fault coverage and a GIL/no-GIL Thread-exit/notification race are now written but not yet compiled or run. ReleaseSafe, TSan, corpus, and broad acceptance remain outstanding; nothing in this comment closes the issue or claims those checks passed.

The ownership audit also found a separate current-Thread TLS lifetime/realm-entry concern, tracked as #905. Its description is explicitly a source finding pending a deterministic multi-Context witness.

9 days ago

Landed and pushed as Chris: fix(atomics): own waitAsync completion through publication (#897). No co-author or other trailers.

Measured validation:

  • 19 focused waitAsync tests pass in each Debug, ReleaseSafe and suppression-free TSan, with no failures/leaks.
  • Native and engine allocation-fault witnesses preserve earlier waiter membership, exact queue reservations, FIFO notification, and once-only dependent completion. Other witnesses cover a full destination queue, 33 harvested outcomes, detached Thread-exit handoff, and actual nursery relocation of nested property/shared-buffer completion roots.
  • All 14 selected PR249 runs pass across GIL/no-GIL modes.
  • Paired Test262 results are identical: Atomics 390, Promise 677, async-function statements 74 / expressions 93, async-generator statements 301 / expressions 623, async-arrow expressions 60. Total 2,218 passes, 0 failures; flips 0 cases in these subtrees.
  • After integrating upstream Intl diagnostics, current main passed the 19 waitAsync tests, the Intl regression, and the 2,218-case conformance rerun again.

Full combined-suite/remaining fuzz/ABI acceptance stays open. Earlier #885/#895 broad validation completed lifecycle 3,360 programs and TSan default 60 programs without failures; those older-source results are recorded separately, not claimed as #897 validation. #896 now has measured primary-exception and moving-dispatch failures plus a passing initial isolated root fix; #905 has a measured wrong-Context Thread.current witness. Those follow-ups remain open.

9 days ago

Final integration verification for fix(atomics): own waitAsync completion through publication (#897) is complete on the pushed engine source in 09279120. The already-measured native/engine allocation faults, 33-outcome harvest boundary, nested moving roots, timeout/notify races, and Thread-exit reservation handoff now have full combined-suite and ABI coverage. The Context identity follow-up #905 is also fixed and closed.

  • Final local Debug suite: 2,380 passed, one Linux-only skip, zero failures/leaks.
  • Public C, Home/private ABI lifecycle/readable-stream, and Bun C API extension fixtures/audits pass in Debug, ReleaseSafe, and unsuppressed TSan.
  • Linux CI 34287554681 passes every full Debug/TSan unit shard, no-GIL Debug/ReleaseSafe/TSan corpus shard, ABI job, and normal/TSan fuzz profile. The sole red job was the stale #903 reference diagnostic, reconciled separately in 656f4d81; this is not a claim that that whole CI run was green.
  • Exact paired Test262 diagnostics remain identical across the ten measured Promise/Atomics/async/module/weak subtrees: 2,893 passed, zero failures, zero flips/regressions.
  • Linux mid-GC: 900 programs from seed 1, zero failures; suppression-free TSan mid-GC: 90 programs, zero failures.

Closing this transaction issue. The earlier local incomplete-sweep observation is preserved and remains under investigation in #907; #896's remaining private exception-box admission audit and #906's finalization cleanup ownership work remain separate open tasks. No timeout, workload, expected failure, or sanitizer suppression was relaxed to obtain this evidence.

Sign in to comment on this issue.