ReviewOS

zig-utils/zig-js

Wasm: feed Response chunks directly into StreamingCompiler

#410
Closed chris opened this 8 days ago ·
8 days ago

Parent: #406 Depends on: #407, #409 Related: #408 Profiles: #141, #142, #163, #164 Roadmap: #134

Wire the engine-native Fetch Body reader directly to the owned Wasm StreamingCompiler lifecycle. The standard Web API remains observationally equivalent either way, but #406 explicitly requires real per-chunk feeding rather than a post-EOF aggregate handoff.

Scope

  • Add an internal ReadableStream byte-consumer sink that receives each normalized chunk in pull order without invoking user code under engine, GC, or compiler locks.
  • Create and root one VM-owned StreamingCompiler lifecycle before the first body pull; feed every string/BufferSource chunk exactly once and finalize exactly once at EOF.
  • Preserve bodyUsed, locking, cancellation, abrupt pull, detached/out-of-bounds views, invalid chunks, zero-length chunks, overflow, allocation failure, and early compiler failure.
  • Release the compiler deterministically on source rejection, cancellation, context teardown, outer-Promise abandonment, and every setup/finalization failure.
  • Queue the existing ordinary compile/instantiate jobs from the finalized bytes with identical feature selection, diagnostics, imports, result shapes, and realm ownership.
  • Prove multi-chunk equivalence for representative MVP, SIMD, threads, exceptions, memory64, GC, and Core 3 modules without rerunning complete corpora.
  • Add focused Debug, ReleaseSafe, TSan, GC-relocation, sibling-realm, and external Home/Bun lifecycle evidence.
  • Refresh the pinned contracts, concise README/docs, #406, #143, and #134; close only after exact fixing-main CI is green.

This issue is complete only when the compiler feed observes the actual Response chunk sequence before EOF; a byte-dropping sink or a single post-consumption aggregate add is not sufficient.

8 days ago

Implemented and pushed in two direct-main commits:

  • 84ba1915 — engine-native ReadableStream byte sink, actual per-chunk StreamingCompiler feeding, exact-once finish/abort, failure cleanup, zero/invalid/detached chunks, representative feature profiles, precise GC, and abandoned-operation teardown.
  • f5575885 — pinned #410 contract plus refreshed #408/#409 contracts, ABI docs, WebAssembly docs, and concise README wording.

Focused streaming compilation evidence passes 6/6 in Debug and exits cleanly in ReleaseSafe and TSan. SIMD, threads, exception handling, memory64, and GC/Core 3 each prove two actual compiler spans; MVP proves three non-empty spans plus a zero-length no-op. Existing sibling-realm and external Home/Bun lifecycle evidence remains shared. Benchmark and conformance scores are unchanged because this path does not alter those workloads. Awaiting only exact fixing-main CI.

8 days ago

CI follow-up: c6501103 makes the new #410 contract a required Home/Bun audit input while preserving #409 as #408’s immediate follow-up. Both local profile audits pass at Home 440/471 and Bun 431/461; shared scanner tests pass 4/4.

8 days ago

Exact fixing-main CI is green. Focused engine, reference inventory, and the complete private-ABI boundary all pass; final certification: https://github.com/zig-utils/zig-js/actions/runs/29787583184/job/88502417191

Sign in to comment on this issue.