ReviewOS

home-lang/home

test(bench): require variadic tuple rejection controls before timing

#492
Closed chris opened this 8 days ago ·
8 days ago

The variadic_tuples workload currently validates only acceptance. Add rejection controls on the existing concat result, conditional Head/Tail, captured spread, consumed result, readonly element writes, and out-of-bounds access. Mutate only a temporary copy and require exact diagnostics plus a normal compiler error exit before any warmup/timing.

Initial audit: seven valid/invalid pairs, all three pinned compilers (TS 6.0.3, native TS 7.0.2, Home), 42 checks all pass. This shows the proposed controls are supported and exercise the actual workload values, not surrogate shapes. They should become permanent admission checks under the benchmark tracker #416/#487.

Version the admission schema so fresh results certify these controls and legacy tuple observations remain explicitly provisional. Do not alter frozen timed sources or infer new performance numbers from an untimed audit.

8 days ago

Implemented in cbc757d. The mandatory variadic_tuples gate appends seven invalid statements to a temporary copy of the actual full workload: concat result, conditional Head/Tail, capture spread, consumed result, readonly write, and out-of-bounds read. It requires exactly five TS2322, one TS2493, and one TS2540 diagnostics with exit 1 or 2. Positive validation and all selected workload gates precede any warmup, timing, or result-directory creation.

All 42 independent valid/invalid controls pass on TS 6.0.3, native TS 7.0.2, and Home, both before and after the ownership change. The combined permanent gate passes against the final release. All 29 harness/report tests pass, including copied-source behavior, missing diagnostics/silent acceptance/crash rejection, gate dispatch, and legacy reporting. Schema 3 retains graph gates and certifies tuple controls. Regenerating the old 20260827T202656Z report labels tuples provisional and both graphs ineligible, without changing any observations.

No timed source was changed and no new timing claim was made. README now includes the untimed correctness summary; details: https://github.com/home-lang/home/blob/4e50814e7/docs/docs/TS_PERFORMANCE.md#semantic-metadata-ownership-and-tuple-admission-untimed .

Sign in to comment on this issue.