ReviewOS

home-lang/home

Native streams: finish pipeline and internal-wrapper corpus failures

#514
Closed chris opened this 9 days ago ·
9 days ago

Follow-up to #66 and #512. The byte-identical 252-file pinned Node stream scan originally passed 246 and left six failures. The Brotli failure is #513. This issue tracks the pipeline and internal-wrapper group.

Implemented on codex/runtime-port in dc6231ea44034b5ec8a52193623b67a8516e5f07:

  • test-stream-pipeline-http2.js now exits naturally. Home retains the stream returned by the pinned streamStart callback, compiles the pin's native C++ header materializer from owned source, dispatches the required header tuple, and ports raw outbound header pairs/value rules.
  • Sixteen byte-identical pinned HTTP/2 header/validation fixtures pass, as does the expanded real-loopback cancellation regression and JSC exception-check mode.

The full stream scan is now 248/252. Four failures remain and this issue stays open:

  • test-stream-pipeline.js still times out in the HTTP/1 case beginning on line 271. Baseline correction: the local Bun release executable reporting 1.4.0-canary.1+4982b91e3 also times out on the complete unchanged file at 45 seconds and produces the same empty keep-alive response/unfinished request pipeline in the isolated diagnostic. Node rejects the unframed follow-up bytes with a 400 response and closes the connection. The exact pinned uWS parser waits for more bytes while the method token lacks a delimiter. No Home-specific defect is established by this case; it remains a failing full-suite/Node-compatibility boundary, not a passing test.
  • test-stream-wrap.js, test-stream-wrap-drain.js, and test-stream-wrap-encoding.js still lack internal/test/binding / internal/js_stream_socket in Home. The exact pin has neither source module and neither entry in exposedInternals. The local release control rejects bun:internal-for-testing even with the feature environment variable, so it cannot establish execution parity for these three cases. Actual wrapper/socket/shutdown behavior remains unimplemented; placeholder exports or test bypasses are not a fix.

Every upstream file was checked byte-for-byte against pin 4982b91e3702094330f3be3883354c52b8c01323. Timed-out parents remain failures. Native stream builtins still partly use configured external builtin literals; runtime success does not prove complete source ownership.

Evidence: zig-out/stream-codec-h2-all-results.json, http2-raw-corpus-results.json, http2-headers-results.json, pipeline-blocks-results.json, http-pipeline-chunks-results.json, and per-case output. Related Brotli work: #513. Compression lifetime: #516. Full port: #66.

Baseline correction documented in b4707dea5, pushed as Chris Breuer without coauthors. Additional evidence: zig-out/http1-pipeline-{lifecycle,pinned}-results.json and http1-wrap-flag-results.json. All four failures remain open.

9 days ago

Reverified unchanged on the final binary from fde7b7ebc64a2d327a1665578d4b396f8a5e93a1 (#512): the full 252-file stream scan still passes 246/252, with the same six failing filenames/statuses as the parent. This issue’s failure remains open; making the reduced-realm unit suite green did not count it as fixed. See #66 and the related #513 for the other remaining stream failures. Evidence: zig-out/stream-final-all-results.json, stream-final-comparison.json.

9 days ago

HTTP/2 pipeline and header progress landed on codex/runtime-port at dc6231e.

Two native contract mismatches were fixed against exact pin 4982b91e3702094330f3be3883354c52b8c01323:

  • streamStart returned the JavaScript stream, but Home discarded it; the parser now retains that object as the native stream context.
  • streamHeaders expects [rawHeaders, headersObject, sensitiveNames], while Home dispatched only a flat array. Home now compiles the pin's native header materializer from owned source and feeds it packed HPACK output. The outbound encoder also handles raw [name, value, ...] arrays without enumerating numeric indices and applies the pinned value/index-name rules.

Verification on Darwin arm64:

  • unchanged test-stream-pipeline-http2.js now exits naturally in 0.36 s;
  • the expanded real-loopback HTTP/2 regression exits naturally and checks cancellation on both sides, headers, raw semantics, sensitive names, duplicate joining, numeric :status, set-cookie, continuations, trailers, multiplexing, reset, ping and GOAWAY;
  • 16 byte-identical pinned HTTP/2 header/validation fixtures pass 16/16;
  • the same loopback regression passes with BUN_JSC_validateExceptionChecks=1;
  • native generation tests pass 3/3 with 111 assertions and the final binary contains the owned Bun__h2__materializeHeaders symbol;
  • the all-stream scan improves from 246/252 to 248/252.

test-stream-pipeline.js still times out and is not counted as fixed. Block isolation identifies its top-level HTTP/1 case starting at line 271: Home emits an empty keep-alive response and never completes the request pipeline; Node closes and completes. Installed Bun 1.3.14 separately coalesces the response and also times out, but the exact pin remains the target. The three internal js_stream_socket wrapper fixtures remain unresolved. Thus #514 stays open.

Evidence: zig-out/http2-raw-corpus-results.json, http2-headers-results.json, http2-exception-check.json, stream-codec-h2-all-results.json, pipeline-blocks-results.json, http-pipeline-chunks-results.json, and stream-codec-h2-source-integrity.json. The full port remains open under #66.

9 days ago

Baseline correction after the compression checkpoint: the local Bun release control reporting 1.4.0-canary.1+4982b91e3 reproduces the same HTTP/1 timeout and event sequence as Home. The full unchanged pipeline file times out at 45 seconds in that control. Node instead emits 400 and closes after the unframed follow-up bytes. The exact source parser waits for an unfinished method token, so this is not yet evidence of a Home-specific port defect.

The exact pin also lacks the two internal wrapper modules. The release control's bun:internal-for-testing gate prevents a comparable runtime wrapper test; that is not a passing or equivalent-execution result. All four stream files remain failing/unimplemented for the requested full-suite goal.

Updated #514 and published the source/control boundary in b4707dea5, as Chris Breuer without coauthors. No runtime workaround, upstream assertion change, or pass-count adjustment was made. The next implementation work returns to the confirmed Home CLI gaps in #508.

9 days ago

Internal stream-wrapper work is complete on codex/runtime-port in 0be19c718, authored and committed solely by Chris Breuer without coauthors.

Home now exposes real test-gated internal/js_stream_socket and internal/test/binding behavior through the ABI-matched generated bun:internal-for-testing literal. The adapter forwards byte streams, rejects decoded strings/object mode with exact ERR_STREAM_WRAP, defers shutdown behind pending writes/backpressure, and reports cancellation exactly once after destruction. The generator preserves the pinned module's existing native @lazy(...) dispatch IDs and fails on wrapper/anchor/ABI drift.

Verification on Darwin arm64:

  • unchanged test-stream-wrap.js, test-stream-wrap-drain.js, and test-stream-wrap-encoding.js: 3/3 pass;
  • lifetime probe, in normal, JSC exception-check, and VM-destruction modes: each run checks 200 cancelled shutdowns, 100 deferred shutdowns, and 200 invalid-chunk rejections with single delivery;
  • byte-identical pinned test-stream* scan: 251/252, up from 248/252; all 252 sources were checked against pin 4982b91e3702094330f3be3883354c52b8c01323 before execution;
  • generator/ABI tests: 3/3 / 115 assertions;
  • native debug build: 17/17 steps;
  • scoped runtime units: 1,828 pass / 19 skip / 0 fail, 19/19 steps;
  • focused Pickier, Zig formatting and whitespace checks pass.

The sole remaining stream failure is the unchanged test-stream-pipeline.js HTTP/1 timeout. Pinned Bun reproduces it, but a reproduced baseline failure is not a parity pass. That work is now isolated in #547 with explicit acceptance criteria. Closing #514 records completion of its HTTP/2 and internal-wrapper implementation scope; it does not claim 252/252 or whole-suite completion. #66 remains open.

Sign in to comment on this issue.