ReviewOS

home-lang/home

fix(test): runtime worker tests must prove they ran on Home, not Bun

#809
Open chris opened this 9 days ago ·
9 days ago

Problem

If Bun exists at one of three hard-coded paths (one of them /Users/chrisbreuer/...), home run hands the file to Bun unless HOME_NATIVE_VM is set. scripts/runtime-regression.sh doesn't set it, and tests/runtime/workers.test.mjs — which covers Worker, MessageChannel and BroadcastChannel — has no process.execPath check (28 of 80 runtime tests lack one). Worker/threads results may be measuring Bun.

Evidence (origin/main @ d4b219507)

Scope

  • Set HOME_NATIVE_VM=1 in the regression script (or assert which engine ran).
  • Add an execPath / engine assertion to every runtime test.
  • Remove the hard-coded developer path from findBunBinary.

Acceptance criteria

  • The suite passes with a fake bun that exits 1 placed on PATH and at each candidate path.

Part of #814.

9 days ago

Checkpoint pushed to main in d7b90b7 (sole author/committer: glennmichael123).

This three-file checkpoint:

  • removes the hard-coded /Users/chrisbreuer Bun candidate from findBunBinary;
  • exports HOME_NATIVE_VM=1 before the runtime-regression capability probe, so the probe, every test, and inherited child environment stay on Home's VM;
  • adds an explicit process.execPath provenance assertion to the Worker / MessageChannel / BroadcastChannel regression.

Verification completed:

  • Zig AST check for src/main.zig, bash syntax, Pickier, and git diff checks pass.
  • The focused worker suite reports ALL PASS through home-debug.
  • The same suite reports ALL PASS with a fake bun that exits 97 first on PATH, proving that test execution did not delegate through PATH.
  • origin/main resolves to the exact commit above; author and committer are both glennmichael123 gtorregosa@gmail.com.

Verification boundary: fresh Debug and ReleaseSafe native links were attempted serially under the required 3,840 MB guard, but the build tree reached 4,324 MB and 4,395 MB respectively and was killed by the guard. I am not claiming a fresh full-link or complete 80-file runtime-suite result from this checkpoint. This issue remains open for the all-files provenance sweep and the complete fake-candidate-path acceptance matrix.

9 days ago

The full provenance sweep is now on main as ten conventional checkpoints, from d7b90b7 through 697cfda. Every commit changes exactly 3–4 files and has glennmichael123 gtorregosa@gmail.com as sole author and committer.

Current result:

  • scripts/runtime-regression.sh exports HOME_NATIVE_VM=1 before its capability probe, so the probe, all tests, and inherited children use Home's VM.
  • findBunBinary no longer contains the hard-coded /Users/chrisbreuer developer path.
  • all 80 tests/runtime/*.test.mjs files now assert that process.execPath is a Home executable; the source audit reports zero files without the assertion.
  • the complete runtime-regression suite passes 80/80 with zero failures while a fake bun that exits 97 is first on PATH.
  • the full guarded run peaked at 781 MB and host low-water was 39%.
  • each changed test was also run in its focused 3–4-file batch with the same fake-Bun trap; Pickier and diff checks passed.
  • origin/main and local HEAD both resolve to 697cfda.

The earlier build boundary remains: fresh Debug and ReleaseSafe links exceed the required 3,840 MB process-tree ceiling (4,324 MB and 4,395 MB observed) and were killed by the guard. src/main.zig passed Zig AST validation, but I am not claiming a fresh full native link. The complete runtime result used the existing known-good home-debug binary; the runner and test changes themselves are interpreted at execution time.

I am leaving #809 open until a fresh link can be completed within the agreed memory policy or the project adopts a verified lower-memory native link path. The runtime provenance and no-delegation acceptance gate itself is now green.

9 days ago

Current-main audit: d7b90b7 already exports HOME_NATIVE_VM=1 before the runtime capability probe, removes the developer Bun path from findBunBinary, and adds Home executable identity checks. All 80/80 tests/runtime/*.test.mjs files now reference process.execPath. Dynamic fake-Bun verification is still pending: the current home-debug lacks JSC, while the available JSC-enabled home binary is stale and fails on an older node:path surface before the provenance assertion. Three guarded current-source JSC build attempts were safely stopped at 2,210 MB/2,048 MB, 3,080 MB/3,072 MB, and 4,048 MB/3,760 MB; the final limit is below this tasks 3,840 MB hard ceiling. I am leaving the issue open rather than claiming unverified acceptance or risking the host.

9 days ago

Current-main follow-up: the newer dual-mode path fixtures tests/runtime/native-path-code-units.test.mjs, native-path-format-coercion.test.mjs, and native-path-unicode-case.test.mjs do not contain a Home executable-provenance assertion. They generate Node snapshots under --record, so any future guard must preserve that reference mode and reject Bun/Home delegation in comparison mode. I prepared that scoped guard and confirmed Node recording, but the available zig-out/bin/home answered no native JS/TS runtime yet for the guarded HOME_NATIVE_VM=1 home run comparison. I removed the unverified edits; no test or historical evidence manifest was changed. This newer-fixture provenance gate remains open until a current JSC-enabled Home binary can run it within the agreed 3,840 MB ceiling.

9 days ago

Partial fix landed on main: 771c052 (fix(test): fail closed on native runtime probe). The regression harness now exports HOME_NATIVE_VM=1, asks Home for its explicit JSC build capability, and treats a failed/unknown native probe or an invalid explicit HOME_BIN as failure instead of a passing skip. Its probe checks that process.execPath resolves to the Home binary. Focused fake-binary controls pass (query failure, genuine non-JSC skip, invalid response, missing marker, and nonzero probe), as do bash syntax, zig fmt --check for src/main.zig, and git diff --check. The JSC-enabled build and full runtime suite are NOT verified: guarded debug and optimized JSC builds exceeded the fixed 3,840 MB ceiling (4,132 MB and 4,015 MB); a guarded non-JSC build also reached 3,844 MB. I did not raise the ceiling. Keep this issue open until the JSC-enabled suite and fake-Bun acceptance test run under an approved safe build environment. Related: #66.

Sign in to comment on this issue.