ReviewOS

home-lang/home

Eliminate Home-injected Bun corpus TODO and skip rewrites

#202
Open chris opened this 23 days ago · 118 comments
23 days ago

Current verification status

Parent tracker: #66. Logical 100% parity is incomplete. The historical adapter-based 4,708-file green result below is not proof that the original assertions exercised equivalent runtime behavior.

The native-only runner and corrected discovery are on main in 1d62a395b, with the config-argument correction in 82156e60d and scanner evidence preservation in 7fa9fc32d, all published as Chris Breuer without coauthors. The synthetic corpus prelude, rewrite engine and bootstrap fallback are removed. All 4,754 original pre-exclusion entries route natively: 1,863 test-runner entries and 2,891 scripts. The exact audit reports zero missing/extra paths or mode differences; 11/11 discovery tests pass. This is inventory evidence, not 4,754 passing files.

The first native-only production build passed 46/46 steps; the focused harness passed 5/6. The failed Node assertion control found split --config path arguments, corrected to upstream CI's --config=path. Direct Home and pinned Bun controls reach and fail the intended assertion with that spelling. Corrected optimized/focused builds are running; their final result is pending. Retained native routes passed HTML 145/one original skip, Blob 65, body/stream 9,467/four skips, Headers/Response 170, Request 24, FormData boundary 5 and microtasks 2. Independent HTML aggregate: 530 assertions, 145 passes and one skip. #701/#694 remain unresolved.

The triage scanner now preserves one attempt, its full individual log and exit code, refuses existing output, and returns nonzero for failed/incomplete/empty scans. It never retries or escalates outer bounds. Pinned Bun controls verified one original-deadline failure, one outer-bound interruption and one pass, each started once; overwrite refusal preserved all logs. Its strict-name discovery still is not the full CI inventory.

A separate preserved Home/Bun comparison of all 78 previously omitted entries is queued after the corrected builds. Docker guards without a daemon receive no Docker coverage credit. Remaining work includes broad native failures, the full CI launch profile, separate script/test/empty accounting, complete gate outcome retention, integration/vendor/service setup and platform/build ownership. This issue remains open and 100% parity remains incomplete.

#703 still owns the full CI launch profile, service/vendor setup, full outcome preservation and the broad runtime failures that native execution exposes. The native capture's 120-second outer bound and upstream CI launch options still need reconciliation; ordinary test deadlines have not been extended. #701 and #694 remain unresolved. Other platform execution and independent native build ownership remain incomplete under #66.

The user's requirement is no workarounds. Preserving an adapter's green status must never justify fabricated outputs, fake memory counters, reduced workloads, extra skips, retries, warm-ups or relaxed original deadlines. Temporary failures must be reported truthfully and tracked while the missing behavior is implemented. Final completion still requires the actual suite to pass logically.

Required completion evidence

  1. Preserve original executable sources, assertions and workloads; distinguish original upstream skips/TODOs from missing Home behavior and exclude them from implemented coverage.
  2. Execute the original tests and child processes in the real Home runtime, with faithful upstream discovery, fixture layout, startup context, flags and expected-failure handling. Remove fixture-specific semantic substitutes and fallback success responses.
  3. Publish exact native results and retain observed failures. A filename count or adapter summary is insufficient evidence of logical parity.
  4. Verify the complete intended upstream suite with zero unexpected failures, unsupported behavior, Home-injected skips/TODOs or substituted results, including the original memory/lifecycle assertions. Current per-file native matrices do not establish a complete single-VM aggregate.

The final objective is unchanged. The historical requirement to keep the adapter gate green throughout does not override truthful results or the user's no-workarounds instruction.

Historical issue statement

The text below is retained for context, not as a current verification claim:

Parent tracker: #66 Related execution tracker: #201

The full copied Bun test tree now executes green at the file gate (4,708/4,708 files, 0 failures, 0 unsupported registrations, 0 runner logs/leaks), and the copied tree exactly mirrors all 12,996 entries in ~/Code/bun/test aside from Home metadata files.

However, logical 100% still requires distinguishing upstream-intentional TODO/skips from Home-injected rewrites and eliminating the latter by porting the missing behavior. Current injection sites include:

  • cron in-process firing and Bun.serve cookie suites
  • WebCrypto generateKey WPT vectors
  • CSS parser invalid-input fuzzing
  • sparse-file behavior
  • Bun.color exhaustive coverage
  • subprocess IPC/stream blob/bytes behavior
  • JSONL stress, OOM, UTF-8, streaming, and adversarial cases
  • non-JS loaders and snapshot update/serialization workflows
  • test.failing done-callback reporting

Acceptance criteria:

  1. Add accounting that separates upstream TODO/skips from Home-added TODO/skips.
  2. Remove each Home-added rewrite only after implementing and validating its underlying runtime behavior.
  3. Keep the full 4,708-file gate at 0 failures, 0 unsupported, and 0 leaks throughout.
  4. Reach zero Home-injected TODO/skip rewrites without weakening upstream assertions or stress semantics.

Track focused feature issues from this issue as needed.

23 days ago

Pushed 951c86eb4 (test(runtime): restore Bun HTML server matrices) to main.

Removed two more Home-injected whole-file TODO rewrites and routed the unchanged files through exact native HTTP/HTML execution:

  • js/bun/http/bun-serve-html-manifest.test.ts: 4 pass, 1 snapshot, 16 assertions.
  • js/bun/http/bun-serve-html.test.ts: 16 pass, 9 snapshots, 102 assertions.

Focused ReleaseFast JSC corpus gate: 20 pass, 0 TODO/skip, 0 fail, 0 unsupported. Coverage includes manifest validation and headers, bundled assets, static and wildcard routes, plugin success/failure/empty/concurrent loading, JSX runtime in development and production, and mixed API/HTML routing. Both files are byte-identical to Bun 4982b91e3702094330f3be3883354c52b8c01323.

The related HTML entry suite was also probed unchanged: 5 pass / 1 fail / 56 assertions. Only bun *.html fails because Home treats the glob as a literal missing path. That real gap is tracked in #681; its TODO remains until fixed.

Remaining rewriteNativeTodoCorpus sites: 10.

23 days ago

Pushed 6e27f3fc75cc4e1c1577df133c8bbd5d39d4d422 to main, removing two more blanket corpus TODO rewrites.

Restored unchanged native execution:

  • js/bun/http/bun-server.test.ts: 45 pass, 2 upstream skips, 0 fail.
  • js/bun/webview/webview-chrome-ws.test.ts: 5 authentic upstream todos when no interactive Chrome remote-debugging session is approved, 0 fail. This replaces Home's synthetic one-test blanket TODO with the exact Bun source behavior.

Both files are byte-identical to Bun checkout 4982b91e3702094330f3be3883354c52b8c01323. Exact routing and result-count regression assertions were added. The focused ReleaseFast/JSC gate passed all three project test targets.

Whole-file rewriteNativeTodoCorpus call sites are now down from 10 to 8.

The paired strict probes exposed real remaining blockers without changing tests:

  • serve.test.ts: 188 pass, 1 upstream skip, 1 upstream todo, 11 fail. Split into #682, #683, #684, and #685.
  • shell/leak.test.ts: 4 pass, 38 fail because relative process.argv0 cannot be spawned after the fixture changes cwd. Tracked in #686.

No fixture rewrite, skipped case, relaxed deadline, reduced workload, or synthetic compatibility behavior was added.

23 days ago

Runtime progress pushed in b4effd236ab23cd3572d28ccc56665b9785e2972.

Implemented four unchanged js/bun/http/serve.test.ts contracts behind the remaining whole-file suppression:

  • null-body response framing for 204, 205, and 304, using Bun's 101/103/204/205/304 render path;
  • strict boolean validation and exact diagnostics for TLSOptions.requestCert, TLSOptions.rejectUnauthorized, and TLSOptions.lowMemoryMode.

Targeted result: 5 pass / 0 fail, including the surrounding multi-TLS validation case.

Full unchanged result: 192 pass / 1 upstream skip / 1 upstream todo / 7 fail, improved from 188 / 1 / 1 / 11. The remaining failures are exactly #684 and #685, so the whole-file TODO stays until both are implemented. #682 and #683 are closed.

Formal validation:

  • clean ReleaseFast/JSC install build passed;
  • focused ReleaseFast/JSC home_rt: substrate compiles gate passed all 4 targets;
  • no test rewrites, skips, relaxed deadlines, or reduced workloads.

The blanket TODO inventory remains at 8 files, but four formerly hidden assertions now pass natively.

23 days ago

Pushed f5ceee0b7b2d0a0aff6772471c0b5e6e1c96e0c1 (test(runtime): restore Bun shell leak corpus) to main.

Removed the Home-injected whole-file TODO rewrite for js/bun/shell/leak.test.ts after fixing the worker lifetime race in fe85ff1dd7be5d4026839513294ff1d93aa7bc77 (#686).

The exact path now routes through the native Home test runner. Regression coverage asserts exact-path classification, test-runner mode, absence of the old synthetic label, and the complete result contract: 42 pass, 0 TODO, 0 fail, 0 unsupported, 0 allowed-empty files.

Validation:

  • copied source is byte-identical to Bun 4982b91e3702094330f3be3883354c52b8c01323;
  • four consecutive unchanged full native runs passed 42/42 (168/168 total) after the pre-fix race reproduced as 41 pass / 1 exit-139 failure;
  • corpus router passes Zig AST validation;
  • no workload, assertion, timeout, or concurrency was reduced.

Whole-file rewriteNativeTodoCorpus call sites are now down from 8 to 7. serve.test.ts remains suppressed only for the five response-stream backpressure gaps in #685; its other 194 tests now pass unchanged.

23 days ago

Progress in cf32ab6: removed the whole-file js/bun/http/serve.test.ts TODO rewrite after porting Bun response-stream backpressure and drain behavior. The unchanged file now passes 199 tests with 0 failures and 2 upstream todos in four consecutive runs.

Whole-file native TODO rewrites drop from 7 to 6. Remaining suppressions are now limited to patch internals (#679), REPL (#680), HTML entry subprocess serving (#681), the broad shell parser/TestBuilder file, and the two WebView integration files. #685 is closed.

23 days ago

Port-progress update: REPL CLI and HTML glob entries landed on main in 010a6d8; focused issues #680 and #681 are closed.

Evidence against exact untouched fixtures from Bun 4982b91:

  • repl.test.ts: 117 pass, 0 fail, 248 assertions, repeated twice
  • bun-serve-html-entry.test.ts: 6 pass, 0 fail, 91 assertions, repeated twice
  • both copied fixtures remain byte-identical to Bun

This removes two more whole-file corpus TODO rewrites. The remaining whole-file rewrites are now four: patch internals (#679), Bun shell integration, WebView WebKit, and WebView Chrome.

The production ReleaseFast JSC executable builds successfully. A focused home_test-only link attempt still hits the existing unresolved-JSC-export limitation (447 symbols), so validation for this batch was performed through the built native executable and the exact upstream suites; the corpus classification source also passes Zig AST checks.

23 days ago

Port-progress update: patch internals landed on main in ab08189 and #679 is closed.

The unchanged Bun patch matrix now passes twice consecutively at 23 pass, 0 fail, 0 TODO, and 0 unsupported. This covers makeDiff, apply, creations/deletions/renames/mode changes, multi-hunk edits, invalid argument exceptions, CRLF and old-style parsing, oversized rename ENAMETOOLONG, and truncated header safety. The fixture is byte-identical to Bun 4982b91, and the production ReleaseFast/JSC build passes.

The whole-file native TODO rewrite inventory is now three: Bun shell integration, WebView WebKit, and WebView Chrome.

23 days ago

Shell corpus progress: e83b67b is pushed to main and closes #689.

The unchanged, byte-identical Bun shell fixture improved from 357 pass / 83 upstream TODO / 21 fail / 1 inter-test error / 991 assertions to 369 pass / 83 upstream TODO / 9 fail / 0 inter-test errors / 1028 assertions. The exact 461-test native-VM run repeated with the same result. ReleaseFast/JSC also passed.

No corpus TODO, skip, assertion, timeout, workload, or fixture content changed.

The nine remaining active shell failures are now split into:

  • #690 environment propagation and assignment/export semantics: 5 failures
  • #691 redirect buffer pinning and input snapshot ownership: 3 failures
  • #692 broken-pipe pipeline lifecycle: 1 failure

The shell fixture still contains 83 TODOs, all inherited verbatim from upstream Bun; this work did not add or rewrite any.

23 days ago

Shell parity advanced on main in 0bc37d2. Ported Bun redirect ownership semantics: stdin ArrayBuffer data is snapshotted at command start, output buffers are transfer-pinned and GC-rooted through native completion, and pins are balanced during cleanup. Also synchronized PWD with the corpus runner internal cwd. Exact unchanged bunshell.test.ts results are stable across two runs at 377 pass / 83 upstream todo / 1 fail, up from 369 / 83 / 9. Closed #690 and #691; only #692 remains for this fixture.

23 days ago

The unchanged Bun shell fixture is now failure-free on main after f575a8d. The corpus launcher reconstructs the Bun repository-relative layout from packages/runtime, loads the mirrored root bunfig explicitly, and routes tests through test/test. Two consecutive original root-level runs: 378 pass / 83 upstream todo / 0 fail / 1035 assertions across 461 tests. Issue #692 is closed; issues #690 and #691 were closed by the preceding 0bc37d2 batch.

23 days ago

Published 2129e4a82 to origin/main. Removed the final three whole-file TODO substitutions (shell integration, WebKit, Chrome) and deleted rewriteNativeTodoCorpus. Original tracked test bytes are preserved; native routing now exercises those complete matrices. The new compiled routing/source regression passes, zero logged errors/leaks.

Four ordinary routes report 487 passes / 89 upstream TODOs / 0 failures / 0 unsupported registrations: WebKit 57/1/0, shell 378/83/0, Chrome 52/0/0, Chrome WebSocket 0/5/0. TODOs are not implemented coverage. The first strict Chrome run remains a real 51-pass/1-failure result tracked in #694; a later pass does not erase it.

This does not close the adapter-removal issue: other adapters and test-level substitutions remain. The optimized full aggregate is incomplete (stopped with exit 143), with separately tracked conformance #695, shared feature flags #696 and Request stress memory growth #697. Full corpus parity is not achieved. #66 now has the current main publication and complete evidence/limits.

23 days ago

Published 5e45c58a3 to origin/main. All four Request corpus files now route to the production runtime: 24 passes, zero failures/TODOs/unsupported cases, with unchanged source, allocation workloads and memory limits. The previous bootstrap stress/subclass execution is replaced with native coverage, and native file/directory/subset/full-gate dispatch no longer initializes the unused bootstrap VM.

The focused native/mixed-engine regressions pass 4/4 (three named Request tests plus the registration guard), with the names independently verified from the compiled metadata. Full native build 46/46; shell shared-dispatch regression 378 pass / 83 upstream TODO / 0 fail. #697 closes this migration; remaining bootstrap shims and adapters are still open here. #698 records the next Headers/Response activation and its required real process context. No whole-suite parity claim.

23 days ago

Next removal batch is #699, following the Headers/Response launch-context fix in #698. Seven unchanged body/stream files have strict-native baselines of 9,467 passes, four upstream skips and zero failures in both Home and pinned Bun. This includes the full 9,086-case HTTP/3-inclusive body-stream matrix.

The body-clone adapter audit identified three concrete substitutes to remove when activating those files: __home_spawn_fetch_body_clone_fixture fabricates successful child output; __home_body_normalized_reader_chunks deduplicates chunks only for that filename; the ReadableStream wrapper selects special pull handling only for that filename. Associated bootstrap-only execution coverage must migrate to the real native runner. This remains pending work, and those old adapter passes are not evidence of logical parity.

23 days ago

#698 is now published on origin/main in 649c3c48f. All six original Headers/Response files pass through ordinary native Home routes: 170 passes, no failures/TODOs/unsupported cases. The native launch context and absolute-path VM rewrite now preserve the mirrored Bun project root; a source link exposes Home's real runtime source to unchanged repository-relative shell commands.

Final build is 46/46; focused harness is 6/6 with independently verified test names. Request, shell, WebKit, native Node script, bootstrap microtasks and expected-failure accounting remain passing. #66 is updated and #698 is closed. The next concrete activation/removal work is #699; the remaining bootstrap adapters and full aggregate/parity are still incomplete.

23 days ago

#699 is published on origin/main in 04644b2d0. All seven original body/stream files now execute natively: 9,467 passes, four upstream skips, zero failures or unsupported cases. The fabricated clone child result, filename-specific chunk deduplication and special clone stream branch are removed. The focused harness passes 4/4 with verified discovery; the optimized build passes 46/46.

#66 is updated and #699 is closed for that completed activation/removal scope. The adjacent original Request deadline failure remains open in #701; a later diagnostic pass was not counted as a repair. #700 is the next Blob activation/removal batch. Full aggregate/parity and the remaining adapters are still incomplete.

23 days ago

The current Blob batch (#700) removes the UTF-16 child interception and the filename-specific Object.defineProperty/prototype-descriptor substitute together. Its six-file native matrix and retained bootstrap byte-storage test are compiling.

The next concrete removals are recorded in #702: fake FormData-to-JSON child output and the FormData-file-error-leak-fixture.ts branch returning fabricated RSS values. All five original HTML web files already have strict-native baselines in Home and pinned Bun: 145 passes, one upstream platform skip and 530 assertions per runtime. The native memory child measures actual growth; this is separate from the pending ordinary-route migration.

23 days ago

#700 is published on origin/main in f491cacd7. All six original Blob files execute natively: 65 passes, no failures/TODOs/unsupported cases. The UTF-16 child interception, global filename-specific prototype-property override and synthetic descriptor lookup are removed. The build passes 46/46 and the focused harness 4/4 with verified discovery.

The current ordinary body, Headers/Response, Request and selected bootstrap regressions pass. #701 remains open for the previously observed Request deadline failure. Direct auditing against the pinned Git objects verifies all executable source bytes, raw symlink targets and executable modes, with only the historical expectations-file content difference.

#66 is updated and #700 is closed. #202 now explicitly distinguishes its historical adapter-green claim from real native parity. #702 remains the next HTML web activation/removal batch; full aggregate/parity and remaining semantic substitutes/build ownership are still incomplete.

23 days ago

#702 is in implementation/testing: the complete five-file HTML web directory is routed natively and both fabricated FormData child-output/RSS paths are removed. The batched optimized build and focused harness are still running; no new passing result is claimed yet.

The broader exact-predicate/discovery audit is now tracked in #703. In this candidate, the 4,708-file Home inventory still routes 4,267 files through the bootstrap adapter; only 441 route natively. Pinned Bun's CI discovery selects 4,754 tracked paths before exclusions, with 78 absent from Home's classifier and 32 Home-only helper/fixture entries. Four mode differences and a missing upstream Node test config need semantic reconciliation. These are inventory findings, not suite passes. #703 is the next larger native execution migration after #702, and #202's historical adapter green result remains insufficient proof of logical parity. #701 and #694 remain open.

Sign in to comment on this issue.