ReviewOS

zig-utils/zig-js

Runtime: cooperatively hand off CPU-bound JavaScript slots

#1040
Closed chris opened this 8 days ago ·
8 days ago

Parent: #502 Found by: hosted CI for 8297fbb

Verified failure

The automatic runtime scheduler grants logical_cpu_count - 1 general runnable slots. On the four-CPU Linux runner, both of these promoted no-GIL cases start four shared-realm Threads, put every admitted Thread into a nonblocking loop, and wait on the host thread for all four to publish readiness:

  • cve/mc-life-wasm-grow-relocate.js: Debug shard 2 timed out after 600 seconds; ReleaseSafe eventually panicked while out of memory after 507 seconds.
  • cve/mc-grow-s4-detach-nullvec-repro.js: Debug shard 3 timed out after 600 seconds.

Their checked evidence is 313 ms and 151 ms respectively. The fourth Thread remains queued forever because the first three never call a blocking primitive. Weighted admission cannot preempt a runnable Thread, and the untyped host wait owns no scheduler slot to release.

This is a production liveness gap in the bounded scheduler, not a corpus timeout problem. Any program that starts more CPU-bound shared-realm Threads than the automatic limit and waits for all of them can deadlock before user-level stop conditions become reachable.

Scope

  • Add cooperative runnable-slot handoff at bounded JavaScript execution checkpoints when scheduler waiters exist.
  • Cover both tree-walker and bytecode VM execution paths.
  • Keep the no-contention cost amortized and avoid a coordinator lock when no waiter is published.
  • Preserve interpreter roots, GIL/no-GIL state, termination checks, and exact runnable/blocked accounting across a handoff.
  • Add a deterministic one-slot regression whose first Thread spins until a queued peer publishes readiness.
  • Re-run both promoted corpus witnesses in Debug and ReleaseSafe no-GIL modes.

Definition of done

  • A one-slot shared-realm CPU-loop handoff test completes without sleeps or an unbounded watchdog.
  • Tree-walker and VM/IIFE-shaped variants both exercise the handoff.
  • Runnable/blocked/slot counters reconcile exactly after completion.
  • Hosted Debug and ReleaseSafe shards pass the scheduler-exposed corpus cases (mc-grow-buffer-storm.js, mc-life-wasm-grow-relocate.js, and the original detach witness).
  • Focused unsuppressed TSan and the full unit gate pass.
  • #502 records the landed commit and evidence.

Guardrails

Do not raise the automatic slot limit, special-case these corpus files, exempt JavaScript Threads from admission, weaken the corpus deadline, or add wall-clock sleeps as scheduler synchronization.

8 days ago

Implemented and pushed to main in e9c7c0b3.

Evidence:

  • one-slot runtime scheduler primitive: all 17 runtime_threads.zig tests pass;
  • end-to-end one-slot tree-walker and required-bytecode regressions pass normally and under unsuppressed TSan;
  • Debug no-GIL corpus: mc-life-wasm-grow-relocate.js passes in 3,285 ms and mc-grow-s4-detach-nullvec-repro.js passes in 26,786 ms;
  • ReleaseSafe no-GIL corpus: the same cases pass in 310 ms and 163 ms, matching their checked baselines;
  • runtime-thread audit passes with 6 production call sites across 6 resource classes and 157 reviewed test-only spawns;
  • docs link check resolves all 378 files and the pinned BunPress v0.2.14 manifest matches;
  • final full unit gate at f9ed7030: 2,583 passed, 1 skipped, 0 failed, 0 leaked in 561.9 s.

The implementation checks one acquire-loaded waiter counter every 1,024 steps, enters no coordinator lock without contention, and freezes/publishes interpreter roots across a no-GIL slot handoff. GIL-mode execution retains its existing GIL scheduler path.

8 days ago

Reopening from hosted main CI evidence on f9ed7030: run 37728534308 timed out after 600 seconds in both Debug and ReleaseSafe shard 2 for:

  • cve/mc-grow-buffer-storm.js
  • cve/mc-life-wasm-grow-relocate.js

Each shard passed 60/62 other cases. The local one-case witness for mc-life-wasm-grow-relocate.js was insufficient to prove the hosted scheduler state: this is the same progress guarantee #1040 owns, and the fix is not complete until both cases terminate in the full shard environment in both modes.

8 days ago

Follow-up fix pushed to main in 04c6c598.

Hosted shard diagnosis: on a small host, cooperative scheduler handoff leaves earlier JavaScript interpreters registered but frozen in the runnable-slot queue. A later unshared WebAssembly grow or ArrayBuffer operation opens a realm-wide mutator stop. The currently runnable peer publishes the stop generation, but the scheduler-parked peers cannot reacquire the only slot to publish, so the stop owner waits forever.

The repair pins every gc_parked wake transition under the existing root lock. A stop owner may count that stable interpreter as stopped; when the peer wakes, it publishes any active stop generation and stays frozen until the stop ends. The same helper now governs scheduler, join, task, property-wait, and cooperative-GC parks. The stop owner is explicitly exempt from joining its own generation.

Evidence at 04c6c598:

  • deterministic park/stop/resume unit passes normally and under unsuppressed TSan;
  • forced one-slot Debug: mc-life-wasm-grow-relocate.js 2.870 s, mc-grow-buffer-storm.js 32.480 s;
  • forced one-slot ReleaseSafe: 0.338 s and 0.539 s;
  • forced one-slot unsuppressed TSan: 23.075 s and 240.578 s, no race report;
  • full unit gate: 2,585 passed, 1 skipped, 0 failed, 0 leaked in 582.2 s;
  • runtime-thread audit: 6 production call sites across 6 classes and 158 reviewed test-only spawns;
  • docs link check: 378 files; pinned BunPress v0.2.14 manifest matches.

The issue remains open until the hosted Debug and ReleaseSafe shards pass both cases in their original full-shard environment.

8 days ago

Hosted closure evidence is complete in run 37736911024:

  • mc-grow-buffer-storm.js maps to shard 2; Debug and ReleaseSafe shard 2 both passed.
  • mc-grow-s4-detach-nullvec-repro.js and mc-life-wasm-grow-relocate.js map to shard 3; Debug and ReleaseSafe shard 3 both passed.

Together with the deterministic one-slot tree-walker/VM witnesses, focused unsuppressed TSan, and the 2,585-test unit gate recorded above, this completes every definition-of-done item for the cooperative scheduler handoff in 04c6c598.

Sign in to comment on this issue.