ReviewOS

zig-utils/zig-js

Workers: cancel queued starts during hard termination

#1032
Closed chris opened this 8 days ago ·
8 days ago

Parent: #502 Related: #980, #982, #984

Executed reproduction

On exact main at 9940ddca8913645caf8c7452aa18014d39b8aed1, a focused production-Worker test sets the public runnable scheduler limit to zero, spawns a script Worker, waits until its typed start is queued, calls terminate(), and joins from a helper thread with a bounded 250 ms observation window. The join does not complete until the scheduler limit is restored. The focused root reports:

worker.test.terminated worker exits while runnable scheduler is paused...FAIL (TestUnexpectedResult)

The Worker stop word and channel wakeups are only observed inside workerMain; runtime_threads.startThread is still parked in the allocation-free slot queue before that entry point. The queue has no owner cancellation edge, so termination cannot complete while an embedder intentionally pauses runnable capacity.

Required change

  • Give caller-owned typed thread starts an explicit cancellation handle that can remove a still-queued waiter under the coordinator mutex and wake the OS thread.
  • Preserve FIFO/weighted scheduling and exact thread, stack, slot, waiter, and lifecycle accounting in grant-versus-cancel races.
  • Route script and module Worker termination through that handle. A canceled pre-entry Worker must close inspector/message outputs and publish its terminal state without evaluating user source.
  • Keep graceful close() drain semantics: only hard termination may cancel a queued start.
  • Expose schema-versioned cancellation telemetry; do not raise capacity, run user code outside a slot, poll, sleep, or add a hidden GIL.

Definition of done

  • The zero-capacity reproduction terminates and joins without restoring capacity first.
  • Deterministic tests cover cancellation before grant, grant/cancel races, queue removal at head/middle/tail, and exact telemetry/invariants.
  • Script and module Workers share the production cancellation path; canceled source/module callbacks do not run.
  • Normal and unsuppressed-TSan focused gates pass with zero leaks.
  • #502 is updated with the landed commit and evidence.
8 days ago

Implemented on main in ddad1d55.

Hard termination now cancels an ungranted script/module Worker start under the coordinator mutex, removes any queue position in O(1), balances live/blocked/stack/waiter accounting, and completes terminal channel/inspector cleanup without evaluating source or loading a module graph. A simultaneous grant and cancel have one mutex-serialized winner; a granted Worker observes the existing stop word before source evaluation. Graceful close() keeps its drain semantics.

Telemetry advances to schema 9 with resource start_cancellations and per-priority cancellations. The fail-closed inventory now token-audits both typed spawn APIs and reconciles 6 production sites plus 151 reviewed test-only direct spawns.

Validation:

  • exact pre-fix reproduction failed until capacity was restored; final script/module reproduction joins while capacity remains zero;
  • standalone coordinator: 15/15 normal;
  • head/middle/tail removal and 32 grant/cancel races: pass normally and under unsuppressed TSan;
  • real script/module Worker cancellation: pass normally and under unsuppressed TSan, with no source/module callback execution;
  • full unit gate: 2,571 passed, 1 skipped, 0 failed, 0 leaked across four shards;
  • production build, runtime-thread audit, 378-file docs-link gate, and benchmark/tooling gate (15/15 build steps, 19/19 validators): pass.

Test262 flips were not measured because the corpus is unavailable in this checkout.

Sign in to comment on this issue.