ReviewOS

home-lang/home

Port core stream builtins and complete direct-reader cancellation

#858
Open chris opened this 8 days ago ·
8 days ago

Stream module graph checkpoint (2026-10-09): source and verified evidence are on origin/main. Home now generates 68 registry modules, adding 38 connected classic/promise/Web/consumer/iterable stream modules and primordial helpers. Web adapters are aligned with the Bun pin, including chunk validation, abort/cancel propagation, safe promise collection and error causes. Native ReadableStream, JSReadableStream and NodeModuleModule compile from Home. Reader lock/detachment and direct-reader teardown use the pinned lifecycle; discarded cancellation promises are handled. Native class identity dispatch is checked before output generation. 257 unchanged original entries validate 9,238 registered passes, 252 process checks, zero failures and six original TODOs: all 252 selected standalone stream scripts, two Node stream files and three Web/compression/fetch-body files. The body matrix passes HTTP/1.1 and HTTP/3 with HTTP/1 disabled in the HTTP/3 row. All 257 source files and four common helpers match the pin. Debug 33/33; ABI 70 tests / 1,122 assertions; five native regression groups including all 13 IPC controls pass; peak 5,692 MB under 8,192 MB. All 1,769 committed archives verify and all 257 reports restore/validate; the compiled generated output is retained and the three new native bodies match Home sources. Targeted Pickier and Zig formatting checks pass. The first ABI run's stale ownership-list failure receives no passing credit. Runtime files are unchanged across integration over separately published TypeScript documentation/evidence. Five original TODOs concern _events fields; the sixth is Bun.file pipe reading. These TODOs, remaining registry/native dependencies, full-suite and Release/ASAN/all-platform parity remain unfinished under #66/#858.

Earlier core-stream checkpoint (2026-10-09): Owned core stream generation and reader cancellation and verified evidence are landed. Home regenerates 280 functions in 21 stream families, validates signatures/directives/error IDs/native dispatch against the linked ABI, preserves 83 unowned function bodies byte for byte, and compiles the replacement core builtin object. Its generated class header remains byte-identical to the linked ABI. Pinned source alignment includes BYOB cancellation, promise peek and read-once updates.

Direct cancellation settles pending direct reads and invokes source cancel once with reason, receiver and creation context. Errors propagate; lazy, reentrant and concurrent cancellation work; legacy close is supported; releaseLock rejects the pending direct read; late pull rejection after cancellation is ignored as a closed-stream event. Native direct/default/BYOB controls pass with verified completion markers, including forced GC and closed/errored cases. The parent control reaches the 120-second supervised bound and receives no success credit.

Bindings 25/25; complete Debug build 31/31, 5,628 MB peak under 8,192 MB; generator/ABI tests 10/10 with 818 assertions. All 18 original entries validate: 9,540 registered passes, zero failures, 269 original skips and three original TODOs. The 268 renderer-export skips and one ASAN-only skip are not passing credit. All 30 audited test/fixture inputs remain unchanged. All 451 compressed artifacts are tracked and hash-verified; restoring all 18 journals validates.

Remaining: original unflushed-read TODO reachability, premature server disposal in the WebSocket TODO, optional renderer/ASAN/platform coverage, wider native sink ownership, remaining core/registry builtins and whole-suite parity under #66. This issue stays open for the remaining coverage/ownership audit; the demonstrated reader cancellation defects are fixed.


Native direct-stream cancellation remains incomplete under #66/#856. Worker and WebSocket creation-context delivery is now fixed, but direct stream probes on the same Home executable establish:

  • A reader waiting on an unflushed direct write remains unsettled after await reader.cancel(reason) completes. Pending reads must settle when cancellation closes the stream.
  • With an explicit controller.flush(), the initial read settles, but reader cancellation still never calls the supplied underlyingSource.cancel callback. Define and implement the intended direct-source cancel/legacy close behavior, including reason delivery, creation context, rejection propagation and exactly-once cleanup.
  • The original AsyncLocalStorage direct-stream TODO awaits an unflushed write before reaching cancellation; its original 90,000 ms timeout therefore does not isolate the cancel callback defect. Keep that distinction explicit.

ReadableStreamInternals is still externally generated at link time, and Home's mirror has older code than pin 4982b91, including missing BYOB cancellation updates and promise-peek changes. Port the core builtin generation and its ABI inputs from Home source, align the pinned stream/reader/controller sources together, then implement and verify direct cancellation and pending-reader semantics. Preserve the original corpus; do not use fixture rewrites, weakened assertions, delayed cancellation or synthetic passing credit as a substitute for runtime behavior.

Verify pending default/direct/BYOB reads, already closed/errored streams, callback reason/this/context, cancellation errors, repeated cancellation, forced GC and native response/sink cancellation. Run connected original stream tests together and retain their raw captures, JUnit and complete EOF validation. Whole-suite and platform parity remain unfinished.

Source, native diagnostic and source evidence.

8 days ago

Core generation update: Home now regenerates all 363 core function bodies in all 37 families and the C++ scaffolding, with a complete-set check and linked-header ABI validation. No unowned core function bodies remain in this generator. Source, evidence.

The stream cancellation fixes and native controls remain green alongside the full-core activation. The 32-entry core API selection validates 908 passes, zero failures, two original skips and three TODOs; complete stream/renderer/ASAN/platform and wider native sink ownership coverage remain incomplete. The pending original stream/WebSocket fixture outcomes stay explicit. Full parity remains tracked by #66, with remaining IPC handle variants in #860.

8 days ago

Stream module graph checkpoint (2026-10-09): source and verified evidence are on origin/main. Home now generates 68 registry modules, adding 38 connected classic/promise/Web/consumer/iterable stream modules and primordial helpers. Web adapters are aligned with the Bun pin, including chunk validation, abort/cancel propagation, safe promise collection and error causes. Native ReadableStream, JSReadableStream and NodeModuleModule compile from Home. Reader lock/detachment and direct-reader teardown use the pinned lifecycle; discarded cancellation promises are handled. Native class identity dispatch is checked before output generation. 257 unchanged original entries validate 9,238 registered passes, 252 process checks, zero failures and six original TODOs: all 252 selected standalone stream scripts, two Node stream files and three Web/compression/fetch-body files. The body matrix passes HTTP/1.1 and HTTP/3 with HTTP/1 disabled in the HTTP/3 row. All 257 source files and four common helpers match the pin. Debug 33/33; ABI 70 tests / 1,122 assertions; five native regression groups including all 13 IPC controls pass; peak 5,692 MB under 8,192 MB. All 1,769 committed archives verify and all 257 reports restore/validate; the compiled generated output is retained and the three new native bodies match Home sources. Targeted Pickier and Zig formatting checks pass. The first ABI run's stale ownership-list failure receives no passing credit. Runtime files are unchanged across integration over separately published TypeScript documentation/evidence. Five original TODOs concern _events fields; the sixth is Bun.file pipe reading. These TODOs, remaining registry/native dependencies, full-suite and Release/ASAN/all-platform parity remain unfinished under #66/#858.

Sign in to comment on this issue.