ReviewOS

zig-utils/zig-js

Promises: make rejection tracker publication OOM-atomic

#884
Closed chris opened this 11 days ago ·
11 days ago

Parent: #474 Related: #480, #493, #874, #885, #888

Problem

Promise rejection-tracker state currently changes before the corresponding realm queue publication:

  • settlement sets rejection_queued = true, unlocks the Promise, then performs a fallible unhandled-queue append;
  • first post-notification handler attachment sets rejection_handled_notified = true, unlocks the Promise, then performs a fallible handled-queue append.

If either append returns OutOfMemory, the Promise can claim the notification is queued/already accounted for while no queue root exists. zig-js supports bounded-heap recovery, so aborting the current evaluation does not justify permanently losing or duplicating the host event.

A reserve-before-commit transaction is insufficient: native resolving functions consume their shared once-only bit before calling resolve/reject, so reservation OOM could leave a pending Promise whose exact settlement cannot be retried. #874 therefore owns the shared allocation-free FIFO representation; this issue owns the Promise-state/publication transaction across Promise.lock and realm_lock.

Required implementation

  • Publish unhandled and later-handled tracker state through the allocation-free #874 FIFO, removing the allocator failure boundary entirely.
  • Use one explicit canonical realm_lock → Promise.lock transaction, with a Promise-lock probe followed by revalidation after acquiring the canonical order.
  • Commit the queue root, GC insertion barrier, and corresponding Promise state before releasing either lock.
  • Preserve exact FIFO/reentrant ordering, one-time unhandled delivery, early-handled skipping, later rejectionHandled delivery, precise tracing/relocation, and shared-realm synchronization.
  • Keep the ordinary fulfilled and already-handled fast paths free of realm-lock acquisition.

No-workaround rules

  • No swallowed OOM, best-effort notification loss, unbounded emergency reserve, global serialization, event cap, retry loop that can duplicate callbacks, or Context-destruction-only recovery.
  • Do not clear a state bit after failure unless a deterministic later operation is guaranteed to republish the exact pending event.
  • Do not leave a fallible allocator operation between consuming a once-only capability and publishing required rejection-tracker state.

Acceptance

  • Deterministic failing-allocator witnesses prove both unhandled and later-handled publication perform zero allocations while flags and FIFO membership commit exactly.
  • Repeated settlement, pre-notification handling, post-notification handling, empty queues, and reentrant callbacks deliver or suppress each event exactly once.
  • Concurrent handler attachment cannot deadlock, lose, reorder, or duplicate notification under no-GIL execution.
  • Moving/concurrent GC rewrites and roots the intrusive queue; Promise remains in its 256-byte allocation class; bounded-heap recovery and Context teardown remain exact.
  • Private rejection dispatch, relevant threadfuzz, full units, and suppression-free TSan pass.
  • Exact Promise Test262 coverage has zero regressions; documentation makes no unsupported throughput or OOM guarantee.

Security boundary

Rejection fanout and allocation pressure are script-controlled. The engine must fail closed without corrupting tracker state, retaining an unrooted managed pointer, or silently suppressing a host-visible event after recovery. The distinct Promise-to-microtask scheduling transaction remains tracked by #885.

11 days ago

Design audit update: the safer candidate is a reserve-before-commit transaction, not an intrusive Promise link.

Proposed lock/order proof:

  1. For rejected settlement with a tracker queue, take realm_lock, reserve one physical FIFO slot fallibly, then take Promise.lock, revalidate that the Promise is still pending/unhandled, and commit settlement state + rejection_queued + appendAssumeCapacity before releasing either lock. Allocation failure occurs before any Promise state transition, leaving the Promise retryable and the queue unchanged.
  2. For first late handler attachment, probe under Promise.lock without mutating, release it, take realm_lock, reserve one handled slot, reacquire Promise.lock, and revalidate. Only then set is_handled / rejection_handled_notified and publish with appendAssumeCapacity. Competing handlers converge because the recheck occurs under the canonical realm → Promise order already used by host drain/GC.
  3. Add the precise-GC root insertion barrier at publication so a queue append after the realm-root scan cannot escape the active marker.

An intrusive FIFO would remove allocation but add a managed link to every Promise, affect slab sizing, and require the concurrent marker to walk the entire chain before dequeue can clear links. That is unnecessary risk unless reservation proves insufficient. No implementation is being mixed into the currently running #875 revision.

11 days ago

A deeper call-site audit invalidated the reserve-before-commit candidate described above. Native resolving functions set their shared promise_resolving_already bit before calling resolve/reject; therefore an OOM returned by a rejection-queue reservation could leave the Promise pending while its once-only capability is already spent. A later exact retry is not guaranteed, so “leave settlement pending on reservation failure” is not an atomic recovery design.

The implementation is now using the issue’s allocation-free option:

  • one realm-lock-protected intrusive successor on each Promise;
  • head/tail/length roots in Context, with O(1) append/pop and a length-bounded tracer iterator;
  • canonical realm_lock → Promise.lock publication after a probe/revalidation, committing the tracker flag and queue root together with no allocator boundary;
  • moving-GC relocation for both queue endpoints and each Promise successor;
  • teardown clears every successor;
  • a compile/test invariant keeps Promise in its existing 256-byte GC allocation class (the pre-change payload was 248 bytes).

This simultaneously removes #874’s front-shift cost and #884’s fallible publication boundary. The full compile/runtime evidence is still pending behind the isolated #875 lifecycle sweep; acceptance boxes will be updated only after those gates pass.

11 days ago

Collector-boundary audit: the rejection FIFO and its Promise-state publication are allocation-free, but the generic GC insertion barrier is not yet an unconditional OOM proof. The owned zig-gc implementation can win a mark claim and then silently lose a fallible barrier_buf/mark_stack append; because the mark bit is already set, the terminal root rescan does not necessarily retrace that cell's outgoing graph.

That collector-wide defect is now tracked explicitly in #888 with failing-allocator, abort-state, TSan, and full-GC acceptance. It is not being disguised as a Promise-specific workaround. For #884, the durable Context queue root and Promise tracker flags still commit together without a caller-visible allocator boundary; final acceptance will state the #888 dependency accurately rather than claiming the existing generic barrier is OOM-infallible.

11 days ago

The Home integration evidence now has a matching rendered manifest in 09279120 — docs(abi): reconcile the Home rejection tracker render manifest (#884).

The earlier source change in 4a1e5d8 added two intended sentences, but its manifest captured incomplete rendered bytes. Rendering the exact parent reproduces its checked 247,143-byte/hash entry; the current page differs only by those two sentences and matches Linux CI's 247,277-byte output. Regenerated with the pinned owner BunPress renderer; only that page entry and the tree hash changed. Checked docs:build and all 328 documentation links now pass. No new implementation/performance claim or manual generated-text edit; flips 0 test262 cases.

11 days ago

Closing as implemented and accepted.

The rejection tracker transaction and intrusive FIFO landed in 4a1e5d82; the Home render manifest was reconciled by 09279120. Its collector dependency #888 is now implemented, accepted, and closed.

Final evidence:

  • rejection tracker publication is allocation-free and state exact: 1/1 in Debug and 1/1 under TSan. The forced-failure allocator records zero allocation attempts for unhandled publication, repeated settlement, pre-handled rejection, and later-handled publication.
  • rejection queue is wide allocation-free FIFO: 1/1 in Debug and 1/1 under TSan, including 4,096 queued Promises, empty transitions, clear/unlink, and reentrant suffix ordering.
  • Promise fits the 256-byte GC slab: 1/1; the successor overlays the mutually exclusive awaiting slot and preserves the production allocation class.
  • Private process rejection/handled dispatch: 1/1, preserving exact reason and Promise identities and pinned event ordering.
  • Moving/concurrent GC traces both queue endpoints and every length-bounded intrusive node under realm_lock, and relocation rewrites the endpoints plus each linked successor. Context teardown unlinks the queues while the precise heap is live.
  • Linux integration run 34287554681 passed full Debug/TSan units, all Debug/ReleaseSafe/TSan no-GIL corpus shards, every configured threadfuzz profile and TSan fuzz profile, private ABI, Test262 parallel, and all Wasm profiles. Its sole unrelated failure was the #903 detached-buffer reference expectation.
  • Fresh exact-parent Promise Test262 with the matching historical dependency pins: 677/677 -> 677/677, 0 flips and 0 regressions.

The implementation leaves fulfilled and already-handled fast paths free of the realm lock and contains no retry loop, emergency reserve, event cap, or fallible queue publication boundary.

Sign in to comment on this issue.