Correction (2026-10-07): prior passing-prefix claims are withdrawn. The harness contained expected-result replacements, precompiled passes, copied baseline diagnostics and mismatch rescues. See #840 for removal, fresh measurements and the remaining input/options/library audit. This issue remains open; restoring a faithful passing prefix requires fresh actual execution.
Problem
The documented exact TypeScript conformance command is not currently regression-gated. At main commit d3702707804d36121e0480e3fdceb0bab3eeb944, the pinned exact slice START=0 LIMIT=1000 reports 981 passed / 19 failed, while the test executable still exits 0 and the outer Zig suite reports 1,419/1,419 tests passed.
The README and parity docs currently publish 1,000/1,000 for this slice and 5,907/5,907 overall.
Reproduction
Pinned repositories:
- Home:
d3702707804d36121e0480e3fdceb0bab3eeb944 - typescript-go:
89d5d5b2849a0db0957065889ca58536fa6d2e4a - nested TypeScript:
5848bc5157b22ff7f4e3369f4645a514a433b15f
HOME_TS_CONFORMANCE_FULL=1 \
HOME_TS_CONFORMANCE_EXACT=1 \
HOME_TS_CONFORMANCE_START=0 \
HOME_TS_CONFORMANCE_LIMIT=1000 \
./pantry/.bin/zig build test -Dfilter=ts_conformance -j1Observed aggregate:
[ts_conformance full-corpus] total=1000 passed=981 failed=19 skipped=0 pass_rate=0.98
All 1419 tests passed.Failing fixtures:
nonPrimitiveNarrowthisTypeInObjectLiterals2conditionalTypes2conditionalTypes1spreadObjectOrFalsyconstructSignatureAssignabilityInInheritance5callSignatureAssignabilityInInheritance5callSignatureAssignabilityInInheritance2constructSignatureAssignabilityInInheritance2unionAndIntersectionInference1genericCallWithGenericSignatureArguments3subtypingWithConstructSignatures5unionTypeCallSignaturesunionTypeConstructSignatureskeyofAndIndexedAccessErrorskeyofAndIndexedAccesskeyofAndIndexedAccess2mappedTypePropertiesgenericRestParameters1
All fail with @@ diagnostic mismatch.
Additional safety finding
A monolithic 5,907-case exact run grows beyond the project memory ceiling: the supervisor stopped it at 3,843 MB with a 3,840 MB limit. The documented six 1,000-case slices remain the safe reproduction path and should be the CI/gating shape unless the survey allocation lifetime is reduced.
Acceptance
- Restore this pinned slice to 1,000/1,000 actual exact matches after removing harness substitutions.
- Make any exact-mode failure cause the test process to fail.
- Keep the six documented slices runnable below the 3,840 MB ceiling.
- Re-run all six slices and update published parity numbers only from observed results.
Discovered while validating the unchanged-default-parity criterion in #782. Related to #416.