ReviewOS

home-lang/home

fix(bench): reject compiler crashes in predicate and destructuring admission

#728
Closed glennmichael123 opened this 8 days ago ·
8 days ago

Parent: #416.

The type-predicate and destructuring negative controls in bench/vs_tsgo/run.py reject exit code zero but accept any other status when the diagnostic-code multiset matches. A crash or timeout after printing the expected diagnostics can therefore admit type_predicates, type_predicates_large, or destructuring to timing.

Require normal diagnostic exits (1 or 2), as the other negative controls already do. Retain exact diagnostic multisets. Add regression coverage for signal termination, unexpected positive exits, timeout status, silent success, and valid diagnostic exits. Document the admission correction in TS_PERFORMANCE.md and the linked README table. Historical measurements are not automatically revalidated by this change.

This is a correctness prerequisite for fair performance comparisons, not a claimed compiler speedup.

8 days ago

Published fix: 48ef397 (refs #416; fixes #728), authored by glennmichael123 gtorregosa@gmail.com, without coauthors.

The predicate, large-predicate, and destructuring negative controls now accept only normal diagnostic exits 1 or 2 with the exact diagnostic multiset. Matching diagnostics followed by a crash or timeout no longer admit a compiler to timing.

Verification: all 96 benchmark harness tests pass, including 63 simulated admission decisions in the new matrix. README and TS_PERFORMANCE.md document the result as untimed, preserving the historical timing tables without claiming fresh validation or a speedup.

Local baseline preparation remains incomplete: a single-job ReleaseFast home-tsc build was stopped by the unchanged 3072 MB ceiling at a reported 3101 MB tree footprint. The old binary was not used to publish current performance numbers. Pinned competitor installs and Hyperfine are also absent on this checkout. Full benchmark leadership remains open.

Sign in to comment on this issue.