ReviewOS

zig-utils/zig-js

Wasm: implement streaming compilation and private compiler feed

#406
Closed chris opened this 9 days ago ·
9 days ago

Parents: #140, #141, #143 Profiles: #163, #164 Roadmap: #134

Implement genuine streaming WebAssembly compilation and the remaining pinned compiler-feed ABI. This must use the same parser, validator, compiler, and feature profiles as ordinary WebAssembly.Module; buffering bytes behind a resolved placeholder is not completion.

Scope

  • Pin Home 7ed99c02, Bun 4982b91e, the JSC streaming compiler bridge, response-extraction hook, and exact source identities.
  • Add realm-local WebAssembly.compileStreaming and WebAssembly.instantiateStreaming with standard Promise assimilation and result shapes.
  • Validate Response status, application/wasm MIME type, used/locked bodies, rejection, and abrupt thenables before consuming bytes.
  • Stream ordered string/BufferSource body chunks through an incremental compiler state with bounded growth, overflow checks, and no user code under engine/GC locks.
  • Implement JSC__Wasm__StreamingCompiler__addBytes against an owned opaque compiler handle; reject stale, foreign-VM, null-pointer/nonzero-length, and post-finalization use without memory corruption.
  • Preserve empty, malformed, validation-error, compile-error, trap-at-instantiation, missing-import, cancellation, detached-view, sibling-realm, invalid-handle, and OOM/failure-atomic paths.
  • Prove byte-for-byte and diagnostic equivalence with ordinary compilation across the declared MVP, Core 2, SIMD, threads, exception, tail-call, memory64, and Core 3/GC profiles where applicable.
  • Add bounded Home/Bun Debug, ReleaseSafe, and TSan compiled consumers; refresh inventories, WebAssembly docs, concise README evidence, and parent roadmaps.
  • Keep the frozen wg-1.0 corpus pin unchanged; record modern coverage against the existing WG3 and upstream-main shadow profiles.
  • Close only after the exact fixing main CI job is green.

Children

  • #407 — preserve ReadableStream Fetch Body lifecycle
  • #408 — add compileStreaming and instantiateStreaming
  • #409 — lifetime-safe private StreamingCompiler feed
  • #410 — feed Response chunks directly into StreamingCompiler

The opaque ABI cannot be a byte-dropping no-op, and the JavaScript APIs cannot be aliases that skip Response and streaming lifecycle behavior.

9 days ago

#407 is implemented on main through a35f025e. Fetch bodies now preserve real streams, derive bodyUsed from disturbance, share all six #405 conversions, and tee live bodies for Response/Request cloning with combined cancellation. Focused Debug, ReleaseSafe, and TSan evidence passes; exact main CI is pending before closure.

9 days ago

#408 is implemented on main by ea64275b + e457d0b9: standard streaming APIs now consume genuine Fetch Body streams, enforce the WG3 Response/MIME boundary, preserve imports/realms/errors, and enter the ordinary compiler/instantiator with stable bytes. Debug, ReleaseSafe, and TSan focused gates are locally green. #408 remains open only for exact fixing-main CI; #409 remains the private incremental compiler-feed slice.

9 days ago

#409 is implemented on main by 1174cbe7 + 0cc83217. The pinned Home/Bun addBytes boundary now uses the same owned lifecycle as the JavaScript streaming APIs, with VM-affine never-dereferenced tokens, idempotent finalization, deterministic teardown, and Debug/ReleaseSafe/TSan external consumers. #407–#409 now await the exact 0cc83217 main private-ABI job.

9 days ago

Added #410 for the remaining strict umbrella requirement: feed actual Response chunks into the #409 lifecycle before EOF. #408 remains the complete observable Web API and #409 the complete private ABI, but #406 will stay open until the two are integrated per chunk and exact CI is green.

9 days ago

Exact fixing-main CI closed #405 and children #407-#409: https://github.com/zig-utils/zig-js/actions/runs/29785272707/job/88495394827. Parent remains open for #410, which is the real Response-chunk-to-compiler bridge.

9 days ago

All children #407-#410 are complete and exact fixing-main CI is green: https://github.com/zig-utils/zig-js/actions/runs/29787583184/job/88502417191

Sign in to comment on this issue.