ReviewOS

zig-utils/zig-js

Workers: make channel delivery fallible and resource-bounded

#40
Closed chris opened this 9 days ago ·
9 days ago

Parent: #1

Problem

Worker channel delivery previously allowed queue-allocation failure to look like a successful post, and unbounded inbox/outbox growth could exhaust process memory outside heap_limit_bytes.

Delivered

  • Channel push is fallible and distinguishes closed-channel rejection, allocation failure, per-message rejection, queued-byte exhaustion, and message-count exhaustion.
  • Zig Worker.postMessage, worker-global postMessage, and the C Worker API propagate delivery failure.
  • Worker configuration exposes documented independent inbox/outbox per-message, queued-byte, and message-count limits; the C limit constructor applies the supplied limits in both directions.
  • Queue byte/message accounting covers live FIFO entries only and remains exact across pop, compaction, close, drain, rejection, and deinit.
  • Structured-clone frames and SAB manifest references transfer or release exactly once on every success/failure path.
  • Complete frame construction is bounded from its first write, including header and SAB-manifest overhead.
  • Rejection is nonblocking and cleanup occurs after releasing the channel lock.
  • Focused tests cover closed push, injected metadata OOM, caps, recovery after pop, compaction, C/JS-visible errors, SAB rejection ownership, and lifecycle rejection oracles.
  • Capacity arithmetic and Worker wait-time conversion are checked.
  • Worker lifecycle, structured-clone, no-GIL, TSan, C-API, docs, and fuzz gates are green.

Contract

A post that returns successfully is queued exactly once. A rejected post is observable and releases its frame exactly once. Default limits are 64 MiB per message, 256 MiB queued bytes, and 1024 queued messages per channel; callers can configure stricter limits, including zero. Delivery never parks while holding a channel or JS/GC trace-sensitive lock.

Acceptance

Final combined gate: https://github.com/zig-utils/zig-js/actions/runs/29236051962 — 31 success, 1 expected nightly-only skip, 0 failures.

9 days ago

First implementation chunk landed in 8d54423. Channel.push is now an owned, fallible operation: success transfers exactly one frame to the FIFO; closed, per-frame, queued-byte/message, or metadata-allocation rejection unlocks first, releases the frame/SAB manifest exactly once, and returns an observable error. Live queued bytes/messages are exact across pop/drain/deinit, dead FIFO prefixes compact before avoidable growth, Worker.spawnWith/spawnModuleWith expose independent direction limits, and JSWorkerCreateWithLimits exposes the same caps to C. Focused queue OOM/cap/recovery, main-side JS, worker-global JS, and C exception tests are green locally. Current gate: https://github.com/zig-utils/zig-js/actions/runs/29228523797

9 days ago

The first matrix exposed stale lifecycle-fuzzer expectations, not a delivery defect: script/module graceful-close and terminate branches still required post-close messages to be silently accepted. c5c7a09 updates all six oracles to require observable Throw, clears the expected interpreter exception, and preserves exact reply/delivery/SAB cleanup counts. The exact TSan lifecycle profile is green locally: 112 programs from seeds 1-2, 0 failures, wall-ms 80910. Replacement gate: https://github.com/zig-utils/zig-js/actions/runs/29229501553

9 days ago

The lifecycle-oracle follow-up gate for c5c7a09 completed green: 31 success, 1 expected nightly-only skip, 0 failures: https://github.com/zig-utils/zig-js/actions/runs/29229501553

Pushed the next isolated #40 slice in 2989d5f (workers: bound clone frame construction).

This moves the per-message limit into structured-clone construction itself, so serialization reserves against the complete framed size (header, SAB manifest, and payload) and rejects with MessageTooLarge before constructing an unbounded frame. Retained SAB tokens unwind if a later graph write crosses the limit. Worker-facing Zig, worker-global, and C paths preserve the same RangeError mapping.

Focused local checks were batched before the push: structured-clone frame-limit/token cleanup, worker-global rejection, C-API configured-limit rejection, docs build, and diff check. Full gate: https://github.com/zig-utils/zig-js/actions/runs/29231655050

9 days ago

The 2989d5fc matrix reached 26 successful jobs before tsan-threadfuzz-midgc found an unrelated #13 test-oracle timing gap: a released waiter path used a fixed pump window instead of joining the threads first. The log contained no TSan race and no Worker channel/clone failure.\n\nThe isolated oracle correction is bdb369e7; exact script/module focused TSan reproducers pass at seed 1. The replacement full #40/#13 gate is https://github.com/zig-utils/zig-js/actions/runs/29232877388.

9 days ago

The serialization-cap + #13 oracle replacement gate completed green: https://github.com/zig-utils/zig-js/actions/runs/29232877388 — 31 success, 1 expected skip, 0 failures.\n\nPushed the final three-commit #40 chunk:\n- 0f5c4a34 — explicit SAB retain/release regression for rejected zero-byte-cap Worker delivery.\n- f04a6dd1 — checked queue metadata capacity arithmetic.\n- f2b3a55e — clamp unrepresentable u64 Worker wait timeouts before signed duration conversion.\n\nThese were already covered by focused local tests and diff checks; the consolidated final acceptance gate is https://github.com/zig-utils/zig-js/actions/runs/29235127135.

9 days ago

Final #40 gate https://github.com/zig-utils/zig-js/actions/runs/29235127135 found one independent #30 runtime issue: the live-peer ArrayBuffer emergency-recovery witness missed its 25 ms publication floor under TSan. All Worker-specific tests in that shard, including timeout clamping and structured-clone/channel coverage, passed.\n\nThe bounded emergency-recovery fix is 379f40f4; replacement combined #30/#40 gate: https://github.com/zig-utils/zig-js/actions/runs/29236051962.

9 days ago

Closing #40 as complete. The final body now records the delivered ownership, error, limit, accounting, nonblocking, C/JS propagation, and test contracts. Final combined acceptance gate: https://github.com/zig-utils/zig-js/actions/runs/29236051962 — 31 success, 1 expected nightly-only skip, 0 failures.

Sign in to comment on this issue.