Published checkpoint (2026-10-08 UTC)
Current protocol-4 five-compiler matrix: 20261008T213919Z, source/harness 3491cdb45, compiler inputs db28b2e10, frozen native c16c8ca. Published in 6004a2634 as Chris without coauthors.
All 145 admissions, 600 original rounds / 3,000 retained samples, unchanged before/after fingerprints, and 603 matrix checksums verify. The direct JSDoc gate now requires15 exact errors across five semantic features at first/middle/last families before timing. Base3 modes also retain process evidence. Older positive-only JSDoc rows are explicitly provisional, never retroactively upgraded.
Home leads 1/20 (startup) and beats nativeTS7 on 12/20, using unrounded medians. Bun leads18; Rust imports. Current measured gaps: checkedJS2.14x, CommonJS2.12x, nullsafe2.01x, interfaces1.93x and TSX1.84x slower than fastest. These are descriptive comparisons, not causal patch claims or a universal lead. README10/full-doc20 rows exactly match raw observations.
Pins remain TS6.0.3, nativeTS7.0.2, Rust tsc-rs0.1.0/TS7.1-dev at72b339e4 and Bun1.4.3-canary.1+bd599f5. Both cached extra payloads verify original executable/inventory hashes; no moving-canary substitution. Matrix peak380 MB, host low-water30%, unchanged3840 MB guard/disk floors. Harness136/136, targeted Pickier/diff checks pass.
The additional admission/profile archive retains35 untimed isolated/combined control records, exact fixtures, diagnostic image58 admissions and an unchanged8192-family sample with equal original/renamed instruction bytes. Profile counts are exploratory, not latency or whole-run percentages; #853 owns source-level investigation. No compiler speed change is claimed by #852.
Matrix SHA-256: d1f7b629929bdc7214bbdaeee3fcc5c79571f21d4808a38a6f3377160e6c2ca0 (603 checksums). Admission/profile context SHA-256: db598c898ebbd9ef69ad3b3e5a043049780086ac51e237c150cb9c812aa9d5c0 (50 checksums).
C++ canonical selection remains outstanding. The source-pinned OnlySpecs candidate still fails all8 negative controls and has zero timings; other platforms and full production admission remain unverified. Zod still has195 TS+3HM9002, not production admission. #838/#416/#548 remain open.
Context
The current bench/vs_tsgo suite compares Home with pinned tsc and native TypeScript (tsgo) on identical admitted strict, no-emit projects. Result 20261007T183321Z is the current three-compiler baseline: 20 workloads, 30 rotating-order rounds after three warmups, 1,800 retained samples, and unchanged executable provenance before and after measurement.
Chris requested that the same benchmark also cover the emerging Rust and C++ TypeScript ports and Bun's check command. Numbers from screenshots, different machines, different corpora, or incompatible checking modes are context only and must not be merged into Home's table.
Scope
- Identify the canonical upstream repository, exact revision, license, build procedure, and supported CLI for:
- Theo's Rust
tsc-rs/ts-rustfrontend; - the active C++ TypeScript port;
- Bun's
checkcommand.
- Theo's Rust
- Extend the benchmark harness generically so an admitted competitor can be pinned and measured without weakening the existing
tsc,tsgo, or Home controls. - Run candidates against the identical generated projects and equivalent check-only semantics. If a tool cannot perform the same work, report it separately rather than presenting an incomparable timing.
- Preserve all diagnostics, exit codes, executable hashes, tool versions, warmups, samples, and raw measurement output.
- Keep process order rotating on one machine, publish medians from every retained sample, and package deterministic checksum-covered evidence.
- Document both successful admissions and exclusions, including the exact semantic or tooling reason for each exclusion.
Fairness requirements
- Canonical upstreams and revisions are verified from primary sources.
- Every included compiler accepts the same positive projects and matches the required negative-control diagnostic contract.
- Compiler modes perform equivalent parse, bind, and type-check work with emission disabled.
- Versions and executable/payload hashes are captured before admission and rechecked after measurement.
- All included compilers run on the same host, in the same run, with the same warmups and repeat count.
- No best-of selection, removed outliers, benchmark-specific shortcuts, test weakening, or copied third-party numbers.
- Raw per-round artifacts and a deterministic evidence archive are retained.
Acceptance
- The harness has regression tests for multi-competitor admission, provenance, reporting, and evidence packaging.
- A fresh full-suite run publishes a traceable table for every fairly admitted compiler.
- README and performance docs state versions, host, methodology, exclusions, result ID, and archive checksum.
- Existing correctness gates and the complete Home checker suite remain green.