ReviewOS

home-lang/home

Retain full native corpus execution and case outcomes durably

#722
Open chris opened this 9 days ago ·
9 days ago

Parent #66; follows #703. Native outcome storage and selected production controls are implemented in 719d636/aa8b9a5 and extended by 61670c72b. Related #723 (child lifetime/platform verification), #724 (full CI selection/setup/vendor execution), #202/#709. All published as Chris Breuer without coauthors on origin/main.

Implemented and verified on native macOS:

  • Exclusive durable result directories; all selected paths flushed before execution; launch-attempt/completion/final records with executable/source hashes, exact argv/cwd/environment/deadlines and raw stream hashes. Interruptions, unstarted files, failed writes and overwrite attempts remain explicit failures/incomplete evidence.
  • Native JUnit retained outside temporary storage for ordinary test-runner files; separate Node/script launch contracts. Hash/XML/case-counter validation rejects malformed or incomplete evidence. Schema 2 includes stream completeness; schema 1 remains readable with completeness unknown.
  • Separate pass/fail/skip/TODO counters, plus distinct script/empty/platform and verified negative-fixture outcomes. ANSI presentation does not hide strict negative diagnostics; rejected negatives retain their actual failing cases.
  • Original deadlines now cover child lifetime after pipe EOF, without losing ownership/exit status during observation. Real libc-FFI controls establish actual EOF, early success and the unchanged 20-second timeout. Native Linux/Windows execution remains pending in #723; compile-only checks are not execution credit.

Verification: 23/23 build steps; 12/12 native harness tests; 2/2 child-ownership units; 3/3 journal units; 9/9 validator units; targeted Pickier passed. All real CLI controls pass, including mixed cases, invalid negative contracts, continuing after failures, empty/scripts, overwrite refusal, open/closed-pipe timeouts, signal, interruption and failed writes.

Six unchanged original files validate: 402 passing registered cases, 1 platform skip, 10 TODOs, one successful Node script, one verified negative process contract and one comment-only file. The negative fixture keeps its original failing JUnit case and contributes no passing test. All six captures are complete; source/snapshot and executable hashes remain unchanged. Home SHA256 fd7c61bf96b18d6954687cd33eb95e429290007f5d199fb985d364e868eb98c4. Final evidence.

Still required before completion:

  1. Port/verify the complete pinned CI selection and setup contract, including the additional Elysia 1.4.28 vendor suite (#724); the 4,754 primary paths are pre-exclusion discovery.
  2. Execute the entire applicable primary and vendor suite with real prerequisites and original deadlines, retaining all failures for feature porting. No retry-based or reduced-workload passing claims.
  3. Preserve full-suite source/snapshot/executable integrity and complete native platform/ASAN verification. Runtime features, default CLI fallback removal and independent native build ownership remain in #66.

Usage, baseline. #722 and #66 remain open.

9 days ago

The launch prerequisite is now verified in 0785d795a, evidence 07b32b554: build 23/23, native harness 9/9, five focused units, and 19,016 matching launch-rule selections. Both Home/Bun cache controls pass; seven private regressions and the grouped original controls completed. #66/#703 now reflect the exact results, including the still-failing original audit snapshot #721. Launch baseline.

This issue remains implementation work: per-file startup/completion persistence and JUnit forwarding are not added yet, and outer skip/TODO counts are still merged. Those gaps must be fixed before treating a broad run as a durable full-suite result.

9 days ago

Outcome implementation is in progress for #722 / #66. Three focused Zig journal units pass: durable selection and exclusive output ownership, raw binary capture plus executable-change rejection, and missing requested JUnit reporting. The native corpus API/CLI now carries separate skipped and TODO counts; ordinary test-runner captures request a retained JUnit file; multi-file CLI selection is recorded before execution. The grouped native compiler/harness build is running. Real mixed-case, script failure, 20-second Node timeout, signal, interruption, overwrite, and failed-write controls are written but have not yet run against the rebuilt executable. No new corpus passing credit is claimed.

Removed approximately 1.2 GB of obsolete compiler outputs from this dedicated worktree after recording their hashes; kept the installed Home binary, source, required native parser archive, and prior evidence. Build admission failures and the earlier Zig reflection API compile errors remain in the evidence logs. Launch verification from 0785d79 was already completed in 07b32b5; the issue body’s earlier “pending” launch wording is stale. Full feature parity remains incomplete.

9 days ago

First production outcome verification is retained. Native build: 23/23 steps, harness 10/10; journal units 3/3; validator units 7/7; targeted Pickier passed. All private real-process controls passed: mixed pass/fail/skip/TODO, script assertions, empty file, continued execution after a failure, report-directory overwrite refusal, actual 20-second Node timeout, SIGTERM, interrupted runner with one incomplete and one unstarted selection, and a real failed capture write. Raw stream hashes and JUnit hashes were verified.

The six-original-file batch retained 402 passing cases, 1 platform skip, 10 TODOs, one successful Node process check, and one comment-only file, with all selected source/snapshot hashes unchanged. The indexed-property expected-failure fixture emitted its required single failure, but the existing strict diagnostic checker incorrectly rejected ANSI-styled expect(received).toEqual(expected). Its raw one-failure JUnit and complete streams remain retained; neither the original fixture nor the launch colors were changed. The validator correctly returned failure for this batch and exposed the aggregate mismatch caused by discarded case counts from an unverified negative fixture.

A grouped correction is now building: evaluate complete diagnostic text while ignoring ANSI presentation and retain real case counts when a negative-fixture contract fails. The expanded harness checks both colored valid diagnostics and styled undefined rejection. No negative assertion is being converted into a passing test case. #722 and #66 remain incomplete pending the corrected native verification and the full suite.

Sign in to comment on this issue.