ReviewOS

home-lang/home

Native HTTP/3: investigate intermittent uploads and finish platform verification

#530
Open chris opened this 9 days ago ·
9 days ago

#539 environment resolution

0fd884f59 closes #539 after a controlled serialized-versus-concurrent comparison. The unchanged adversarial file passes 12/12 serialized Home and 12/12 serialized pinned Bun processes, all 27/27 / 163 assertions. Eight-way waves reproduce the historical first-upload deadline pattern in both binaries on the saturated shared host, with all UDP ports distinct. No runtime or test semantics changed; heavyweight HTTP/3 verification is now serialized away from builds. This does not close #530: other platforms, integration, and dependency ownership remain open.

Latest published transport results

2c5df8cce is pushed on codex/runtime-port as Chris Breuer without coauthors; not merged to main. #531 implements request compression and #535 repairs CONNECT writer/envelope ownership. Complete unchanged HTTP/3 client now passes 52/52 / 241 assertions, matching pinned Bun; the two compression failures below are historical. Full unchanged body-stream again passes 9,086/9,086 / 135,984 assertions. HTTP/2 client passes 60/60 / 197, adversarial 15/15 / 33, lifetime 7/7 / 21.

HTTP/3 reliability remains open in #539. The initial final-candidate adversarial run failed 21 pass / 6 fail / 3 errors / 150 assertions, followed by three complete 27/27 / 163 replays. Published-parent replay failed 19 pass / 8 fail / 4 errors / 125; pinned Bun had a 27/27 replay and a 24 pass / 3 fail / 160 replay. These first-upload stalls are retained, not erased by successful retries or attributed to the environment without evidence.

Native build 17/17; units 1,827 pass / 19 skip / 0 fail, 19/19 steps; sequential regressions 60/60, including 29 native files, full spawn 126 passes / 5 skips / 5,484 assertions, full child-process 32 passes / 1 skip / 1 TODO / 72. All five EOF/fetch and both abort files pass. The disk-space failure is retained separately from the successful full unit rebuild/rerun. Evidence: zig-out/compress-publication.json, compress-h3-replay.json, compress-body-stream-final5.json, compress-unit3.json, compress-regression2s.json, and docs/docs/NATIVE_RUNTIME_BINDINGS.md.

Remaining: causal diagnosis/repair of #539, TLS context/CA/version behavior in #532, platform verification, integration and full source ownership. #530 and #66 remain incomplete.

Earlier TLS/HTTP transport evidence

Published TLS/HTTP transport checkpoint: 29c196835

Pushed on codex/runtime-port as Chris Breuer, without coauthors; not merged to main. The TLS C ABI now includes the missing min/max fields and correct certificate-verification offsets. HTTP/3 refuses unsupported trust settings and automatic upgrades that would drop them. Multiplexed progress now delivers the first response chunk during a duplex upload while HTTP/1 buffered EOF behavior remains intact.

  • Full unchanged body-stream matrix: 9,086/9,086 / 135,984 assertions. This replaces the prior 4,203 HTTP/3 failures; both ABI-only and final rebuilt candidates pass.
  • HTTP/3 adversarial: 27/27 / 163. HTTP/3 client: 50 pass / 2 fail / 235; both remaining failures are request compression in #531, so whole HTTP/3 is not complete.
  • HTTP/2 client: 58 pass / 2 fail / 2 errors / 185, still missing request compression. Adversarial: 13 pass / 2 fail / 29, with verification-host session isolation and invalid header names tracked in #533.
  • TLS server validations: 16/16 / 32. TLS context/CA ownership and user-facing version options remain in #532; restoring ABI fields alone does not implement those options.
  • Native build 17/17; runtime units 1,824 pass / 19 skips / 0 failures, 19/19 steps; sequential regression 58/58 commands, including 27 native files, full spawn 126 pass / 5 skips / 5,484 assertions and full child-process 32 pass / 1 skip / 1 TODO / 72.
  • All five complete fetch files pass in the sequential rerun, including gzip 24/24 with its original 1,025 MiB payload and streaming 110 passes / 5 TODOs. Both abort files retain their passes. The new native TLS/HTTP3/duplex fixture passes normally and with VM destruction checks; published parent fails. Four-way Home/Bun QUIC loopback controls all return the exact response.

The initial parallel validation had original spawn/streaming timeouts; a unit attempt exhausted disk and could not write a valid result. Those attempts remain failures/invalid evidence. Only obsolete task-owned objects were removed, with all executables preserved. The complete sequential reruns pass without changing deadlines. An incorrect expectation in the newly written native fixture was corrected to enable certificate verification for the enforced identity callback; upstream tests were untouched.

72 measured upstream test/harness sources and six consulted implementation sources match pin 4982b91e3702094330f3be3883354c52b8c01323. Final native SHA-256: 039a30eee0d7ef2098b92898a1b9962cd3a1ebf5631a9fc037cd98df1e09202c. Evidence is documented in docs/docs/NATIVE_RUNTIME_BINDINGS.md and zig-out/h3-source-integrity-final.json, h3-corpus-final3.json, h3-unit4.json, h3-regression4s.json, eof-fetch-h3-sequential4.json, h3-publication.json, and complete per-run logs where successfully recorded.

#530 remains open for remaining HTTP/3 and platform/integration work. #531, #532 and #533 retain the specific failures above. Shared Darwin checkout only; unrelated compiler work is excluded. Skips/TODOs, other platforms, integration, full source ownership and 100% logical Bun-suite compatibility under #66 remain incomplete.

Earlier evidence after fb161314a

This commit fixes the separate #529 abort-settlement bug; it does not repair HTTP/3. The entire unchanged body-stream matrix remains 4,883 pass / 4,203 fail / 10 between-test errors in the candidate versus 4,883 pass / 4,203 fail / 2 errors in the parent. Both runs contain HTTP3StreamReset errors and original five-second case timeouts; both complete runs emit identical failing test-name counts. The additional candidate errors are retained, not excused or relabeled. Pinned Bun passes all 9,086 tests / 135,984 assertions.

The published parent's complete HTTP/3 client/adversarial files are also failing: 6 pass / 46 fail and 5 pass / 22 fail, against pinned Bun 52/52 and 27/27.

A four-way loopback control isolates the server side before JavaScript dispatch: both Home and Bun clients fail against Home's HTTP/3-only server, and its JS handler is never reached. Both clients succeed with the exact 200 response against Bun's HTTP/3-only server. Explicit QUIC selection and http1: false rule out silent TCP fallback. The next implementation work is the native server request path; no speculative HTTP/3 change is included in this commit.

Evidence: zig-out/body-h3-cross1.json, body-h3-{parent1,release1}.json, body-corpus-{parent1,release1,candidate1}.json, body-matrix-comparison1.json, and body-source-integrity-final1.json. All 64 measured source files match the exact pin. Full stdout/stderr and failed cases remain recorded. #66's historical checked HTTP/3 item has been corrected; full parity is incomplete.

Initial reproduction

Follow-up to #66 and the historical #180/#181 HTTP/3 completion claims. A fresh strict-native run of the entire unchanged js/web/fetch/body-stream.test.ts on published Home 90ced0ed660150c26af009cbea64c40b46993805 is 4,883 pass / 4,203 fail / 2 between-test errors / 72,378 assertions. The pinned Bun release control is 9,086 pass / 0 fail / 135,984 assertions on the same unchanged source. Home completed in 200.49 seconds; this is not an outer-timeout result.

Failures occur in the HTTP/3 matrix, beginning with the first direct-stream response and HTTP3StreamReset. The matrix explicitly requests QUIC via protocol: "http3" and runs the server with http1: false; silently falling back to HTTP/1 or disabling cases is not a fix. The existing response-body cloning diagnostic is insufficient to establish complete transport parity.

Investigate the actual native HTTP/3 server/client streaming and lifetime contracts against pin 4982b91e3702094330f3be3883354c52b8c01323, retaining all original sizes, assertions and deadlines. The older checked HTTP/3 tracker item cannot stand as proof that this current matrix passes. The separate active-upload abort/response-settlement gap is #529.

Evidence: zig-out/body-corpus-{parent1,release1}.json, full body-corpus-*-body-stream.{stdout,stderr} logs, body-source-integrity1.json, and body-matrix-counts1.json. Source matches the exact pin. No runtime delegation or compatibility adapter was used. Shared Darwin checkout only; other platforms and full parity remain unverified.

9 days ago

The failure also reproduces in the complete unchanged HTTP/3 client and adversarial files: Home 6 pass / 46 fail / 11 assertions and 5 pass / 22 fail / 5 assertions, respectively; pinned Bun 52/52 / 241 assertions and 27/27 / 163 assertions. All sources match the exact pin.

A four-way loopback diagnostic narrows the failing side:

ServerClientResult
HomeHomeHTTP3StreamReset; JS handler never reached
Homepinned Bunfailure; JS handler never reached
pinned BunHomestatus 200, exact response body
pinned Bunpinned Bunstatus 200, exact response body

Both server instances explicitly use http3: true, http1: false; clients explicitly select protocol: "http3". Evidence: zig-out/body-h3-{parent1,release1}.json, body-h3-cross1.json, full per-case logs, and body-source-integrity-final1.json. This localizes investigation to the Home server path before JavaScript request dispatch; no speculative fix has been committed for this issue. The #529 response-abort fix is independently under verification.

9 days ago

Current evidence after fb161314a

This commit fixes the separate #529 abort-settlement bug; it does not repair HTTP/3. The entire unchanged body-stream matrix remains 4,883 pass / 4,203 fail / 10 between-test errors in the candidate versus 4,883 pass / 4,203 fail / 2 errors in the parent. Both runs contain HTTP3StreamReset errors and original five-second case timeouts; both complete runs emit identical failing test-name counts. The additional candidate errors are retained, not excused or relabeled. Pinned Bun passes all 9,086 tests / 135,984 assertions.

The published parent's complete HTTP/3 client/adversarial files are also failing: 6 pass / 46 fail and 5 pass / 22 fail, against pinned Bun 52/52 and 27/27.

A four-way loopback control isolates the server side before JavaScript dispatch: both Home and Bun clients fail against Home's HTTP/3-only server, and its JS handler is never reached. Both clients succeed with the exact 200 response against Bun's HTTP/3-only server. Explicit QUIC selection and http1: false rule out silent TCP fallback. The next implementation work is the native server request path; no speculative HTTP/3 change is included in this commit.

Evidence: zig-out/body-h3-cross1.json, body-h3-{parent1,release1}.json, body-corpus-{parent1,release1,candidate1}.json, body-matrix-comparison1.json, and body-source-integrity-final1.json. All 64 measured source files match the exact pin. Full stdout/stderr and failed cases remain recorded. #66's historical checked HTTP/3 item has been corrected; full parity is incomplete.

9 days ago

The HTTP/3 investigation found a concrete TLS ABI mismatch: Home omitted ssl_min_version/ssl_max_version from the C options struct. LLDB observed request_cert = 600 and reject_unauthorized = 3 at the C boundary, with UDP and ALPN working but no accepted connection. Restoring the fields and their digest entries restores all four Home/Bun client-server loopback combinations.

The first candidate passes the entire unchanged body-stream file: 9,086/9,086 tests / 135,984 assertions, and HTTP/3 adversarial 27/27 / 163. Follow-up fixes for duplex progress and unsupported custom TLS options bring HTTP/3 client to 50 pass / 2 fail / 235 assertions. The two remaining compression cases are explicitly still failing; this is not whole HTTP/3 completion. The final rebuilt artifact and regression suite are still being verified; these changes are not published yet.

Focused follow-ups now track the remaining work:

  • #531: request compression across HTTP/1, HTTP/2 and HTTP/3.
  • #532: native TLS SecureContext/CA verification and version-option plumbing.
  • #533: HTTP/2 Host-verification session isolation and invalid response-header names.

#530 remains open for remaining HTTP/3 work and platform/integration verification. All 71 measured upstream tests and six consulted implementation sources match pin 4982b91e3702094330f3be3883354c52b8c01323. Full logs, failed attempts, original deadlines and workloads are retained. #66 remains incomplete.

9 days ago

Published TLS/HTTP transport checkpoint: 29c196835

Pushed on codex/runtime-port as Chris Breuer, without coauthors; not merged to main. The TLS C ABI now includes the missing min/max fields and correct certificate-verification offsets. HTTP/3 refuses unsupported trust settings and automatic upgrades that would drop them. Multiplexed progress now delivers the first response chunk during a duplex upload while HTTP/1 buffered EOF behavior remains intact.

  • Full unchanged body-stream matrix: 9,086/9,086 / 135,984 assertions. This replaces the prior 4,203 HTTP/3 failures; both ABI-only and final rebuilt candidates pass.
  • HTTP/3 adversarial: 27/27 / 163. HTTP/3 client: 50 pass / 2 fail / 235; both remaining failures are request compression in #531, so whole HTTP/3 is not complete.
  • HTTP/2 client: 58 pass / 2 fail / 2 errors / 185, still missing request compression. Adversarial: 13 pass / 2 fail / 29, with verification-host session isolation and invalid header names tracked in #533.
  • TLS server validations: 16/16 / 32. TLS context/CA ownership and user-facing version options remain in #532; restoring ABI fields alone does not implement those options.
  • Native build 17/17; runtime units 1,824 pass / 19 skips / 0 failures, 19/19 steps; sequential regression 58/58 commands, including 27 native files, full spawn 126 pass / 5 skips / 5,484 assertions and full child-process 32 pass / 1 skip / 1 TODO / 72.
  • All five complete fetch files pass in the sequential rerun, including gzip 24/24 with its original 1,025 MiB payload and streaming 110 passes / 5 TODOs. Both abort files retain their passes. The new native TLS/HTTP3/duplex fixture passes normally and with VM destruction checks; published parent fails. Four-way Home/Bun QUIC loopback controls all return the exact response.

The initial parallel validation had original spawn/streaming timeouts; a unit attempt exhausted disk and could not write a valid result. Those attempts remain failures/invalid evidence. Only obsolete task-owned objects were removed, with all executables preserved. The complete sequential reruns pass without changing deadlines. An incorrect expectation in the newly written native fixture was corrected to enable certificate verification for the enforced identity callback; upstream tests were untouched.

72 measured upstream test/harness sources and six consulted implementation sources match pin 4982b91e3702094330f3be3883354c52b8c01323. Final native SHA-256: 039a30eee0d7ef2098b92898a1b9962cd3a1ebf5631a9fc037cd98df1e09202c. Evidence is documented in docs/docs/NATIVE_RUNTIME_BINDINGS.md and zig-out/h3-source-integrity-final.json, h3-corpus-final3.json, h3-unit4.json, h3-regression4s.json, eof-fetch-h3-sequential4.json, h3-publication.json, and complete per-run logs where successfully recorded.

#530 remains open for remaining HTTP/3 and platform/integration work. #531, #532 and #533 retain the specific failures above. Shared Darwin checkout only; unrelated compiler work is excluded. Skips/TODOs, other platforms, integration, full source ownership and 100% logical Bun-suite compatibility under #66 remain incomplete.

9 days ago

#539 is closed by the verified environment-isolation checkpoint 0fd884f59: 12/12 serialized processes pass in each of Home and pinned Bun, while intentional eight-way waves reproduce the same deadline failures in both. No runtime or upstream-test change was made. #530 stays open for platform, integration, and native dependency ownership.

Sign in to comment on this issue.