Native auto-install: verify EOF and streaming parity across platforms

Current checkpoint — 90ced0ed6

Buffered EOF-delimited bodies now retain incremental decoding but only publish partial notifications when the caller requested streaming. This repairs partial manifest parsing and duplicate installer task publication without removing the pending-task assertion or clamping counters. The successful fragmented-install probe also exposed libarchive option ordering: options clear the selected format, so tar is now explicitly selected after options and before opening. Streaming extraction remains enabled.

The published parent fails the new real-install regression with a half-parsed manifest; the first HTTP candidate fails archive opening; the final candidate passes normally, with VM destruction/JSC exception checks, and in the full regression run. The original Python EOF-404 parent SIGABRT is retained. All three final EOF/Content-Length/keep-alive controls exit 0 with empty stderr, six correct caught errors and two unauthenticated requests. The streaming handshake requires the first chunk before releasing the tail. The tar-format repair follows the linked external vendor behavior; it is not claimed as a line-for-line fix already present in pinned Rust.

Native build 17/17; runtime units 1,824 pass / 19 skips / 0 failures (19/19 steps); runtime regressions 56/56, including 25 native files. All 46 measured upstream test sources match the exact Bun pin. No assertion, workload, deadline, skip or TODO was weakened. Full logs and failed controls remain in the evidence records.

Related decompression fixes are tracked in #527 and #528. Evidence: zig-out/eof-native-{parent,candidate1,candidate2-trace,candidate3,final4,destruction}.json, resolve-404-eof-{parent,candidate1}.json, resolve-404-eof-final4-{close-eof,close-length,keepalive-length}.json, eof-regression4s.json, eof-unit4.log, and eof-implementation-pin.json.

Published on codex/runtime-port as Chris Breuer without coauthors, not merged to main. Verification uses the shared Darwin checkout; unrelated compiler changes are excluded. Linux/Windows behavior and branch integration remain unverified; this issue stays open for those checks. Skips/TODOs and full source ownership are not completed parity. The full port and 100% logical Bun-suite compatibility under #66 remain incomplete.

Original reproduction (before this checkpoint)

Follow-up to #66, #518 and #525. A deterministic local HTTP/1.0 registry returning 404 exposes a native auto-install task-accounting panic after JavaScript catches the missing-package errors.

The owned diagnostic uses Python's ThreadingHTTPServer on loopback, returns actual 404 responses and closes each connection, and runs six resolutions (unscoped/scoped package × require, require.resolve, import.meta.resolveSync) in an ESM eval. Config/cache are fresh task-owned directories; no credentials are sent. No upstream test source is modified.

• Published Home 828ecc7293d55ee96810b2e9fe0ddbec427f494f: prints the six caught errors, then SIGABRT (-6).
• Initial #525 diagnostic candidate: canonical messages are fixed, but the same SIGABRT remains.
• Pinned Bun release 1.4.0-canary.1+4982b91e3: exit 0, empty stderr, six correct errors, two registry requests.

The retained debug assertion is manager.pendingTaskCount() > 0 in packages/runtime/src/install/PackageManager/runTasks.zig while draining async_network_task_queue. The stack continues through the VM event loop while waiting for the ESM entrypoint promise. Do not remove the assertion or clamp counters: establish exactly-once task completion and ownership for repeated synchronous resolutions, network errors and connection closure, and add coverage in the existing local-registry regression. The Bun.serve-based fixture does not currently reproduce the panic, so its pass is not evidence that this case is fixed.

Evidence: zig-out/run-resolve-404-control.py, resolve-404-{parent,candidate1,release-control-esm}.json. Full stdout/stderr and request-path/authorization-presence records are retained; no raw credential values are recorded. Parent and candidate produce only two requests. This is a pre-existing installer/event-loop gap separate from canonical diagnostic formatting. Darwin only; other platforms and full native parity remain unverified. #66 remains incomplete.

The final #525 candidate narrows this failure to EOF-delimited HTTP response bodies:

Each case uses a fresh task-owned config/cache and the same six synchronous resolution calls. The failing case prints the now-correct resolver errors before aborting. Adding a length header is only a diagnostic control, not a fix or a change to the failing test. The pinned Bun control already handles the original EOF-delimited response successfully.

Evidence: zig-out/resolve-transport-matrix3.json, run-resolve-404-transport.py, and resolve-404-candidate3-{close-eof,close-length,keepalive-length}.json. Native binary SHA-256: 58bfeae385e18530ec2a15b551534f0241c18fc0753a4ec2d86c8304b17193b2. Investigate completion publication/accounting when response length is determined by EOF; retain the assertion and original failing response shape.

The failure is upstream of task accounting. handleResponseBodyFromMultiplePackets processes an unknown-length body incrementally and reports processed as progress even when response_body_streaming is false. A buffered registry consumer then sees a partial callback and publishes completion before EOF; the later final callback can publish the same task again.

The expanded native registry fixture also reproduces a functional failure in the published parent without relying on the debug assertion: a real manifest split across two EOF-delimited writes is parsed after its first half and fails with Unterminated string literal. It uses actual local package metadata and tarball bytes, not a simulated installer result.

The candidate keeps incremental decoding but gates partial body notifications on the streaming signal, matching the existing chunked-response behavior. Completion assertions and counters are unchanged. Tests cover fragmented 404s, cold successful manifest/tarball installation, and a streaming handshake where the server withholds the tail until the child confirms it read the first part. A unit test covers unknown-length buffered/streaming modes and a length-delimited control. Build and unchanged fetch-suite verification are in progress.

Evidence: zig-out/eof-native-parent.json, eof-fetch-source-integrity.json, eof-build1-source-snapshot.json; the prior raw HTTP/1.0 parent panic is reproduced again in resolve-404-eof-parent.json. All failure logs are retained. Two verified superseded candidate cache directories were removed to free space; published controls and logs were preserved (eof-owned-cache-cleanup.json). This is not yet a completion claim.

Current checkpoint — 90ced0ed6

Buffered EOF-delimited bodies now retain incremental decoding but only publish partial notifications when the caller requested streaming. This repairs partial manifest parsing and duplicate installer task publication without removing the pending-task assertion or clamping counters. The successful fragmented-install probe also exposed libarchive option ordering: options clear the selected format, so tar is now explicitly selected after options and before opening. Streaming extraction remains enabled.

The published parent fails the new real-install regression with a half-parsed manifest; the first HTTP candidate fails archive opening; the final candidate passes normally, with VM destruction/JSC exception checks, and in the full regression run. The original Python EOF-404 parent SIGABRT is retained. All three final EOF/Content-Length/keep-alive controls exit 0 with empty stderr, six correct caught errors and two unauthenticated requests. The streaming handshake requires the first chunk before releasing the tail. The tar-format repair follows the linked external vendor behavior; it is not claimed as a line-for-line fix already present in pinned Rust.

Native build 17/17; runtime units 1,824 pass / 19 skips / 0 failures (19/19 steps); runtime regressions 56/56, including 25 native files. All 46 measured upstream test sources match the exact Bun pin. No assertion, workload, deadline, skip or TODO was weakened. Full logs and failed controls remain in the evidence records.

Related decompression fixes are tracked in #527 and #528. Evidence: zig-out/eof-native-{parent,candidate1,candidate2-trace,candidate3,final4,destruction}.json, resolve-404-eof-{parent,candidate1}.json, resolve-404-eof-final4-{close-eof,close-length,keepalive-length}.json, eof-regression4s.json, eof-unit4.log, and eof-implementation-pin.json.

Published on codex/runtime-port as Chris Breuer without coauthors, not merged to main. Verification uses the shared Darwin checkout; unrelated compiler changes are excluded. Linux/Windows behavior and branch integration remain unverified; this issue stays open for those checks. Skips/TODOs and full source ownership are not completed parity. The full port and 100% logical Bun-suite compatibility under #66 remain incomplete.