Goal
Maintain a fair, reproducible TypeScript frontend performance suite and improve Home until it leads JavaScript tsc 6 and stable native TypeScript 7 across every accepted workload.
Benchmark rules
- All compilers receive identical valid source graphs and compiler options.
- Every compiler must validate successfully and silently before timing, and reject the same invalid controls with the expected diagnostics.
- All selected workloads must pass admission before any timing or result-directory creation; no failing row is silently omitted.
- Compiler versions and generated workloads are pinned.
- JavaScript TypeScript 6 and native TypeScript 7 are installed in isolated prefixes.
- Measurements rotate compiler order within every round to balance workstation load.
- Workloads represent general compiler behavior, never benchmark-specific shortcuts.
- Results, environment details, caveats, and reproduction commands stay documented.
Landed foundation
- Nineteen deterministic workloads covering startup, files, complex types, module graphs, TSX, inference, control flow, predicates at two scales, null-safe access, destructuring, overloads, classes, structural objects, interface composition, variadic tuples, checked JavaScript, and recursive generic consumers
- Pinned JavaScript tsc 6.0.3 and stable native TypeScript 7.0.2 toolchain
- Interleaved Hyperfine measurement schedule and silent-success validity gate
- Release allocator, reusable source I/O, and bounded large-program parallelism
- Cached source facts, bind-only export analyses, and projected export-name sets
- Indexed lexical declarations, interface merge predecessors, namespaces, annotations, and validated type display names
- Pruned provably irrelevant overload, inference, import, expando, and compatibility work
- Relation-cache working-set retention and single-probe insertion
- Full performance documentation and linked README summary
Current correctness and measured status
Implementation 1cfc33c (#536), audit d95dfaa (#541), and verified README/report 707045c are pushed to main as Chris with no coauthors. TS 6.0.3 and native TS 7.0.2 are verified; TS7/tsgo is one competitor.
The prepared-owner CommonJS display query removes the extra full source compilation and raw-text recovery. Program140/140, conformance1418/1418, CLI69/69 and harness75/75 pass; unchanged checker/driver targets verify from cache. All19 unchanged admission gates and all15 complete audit case sets preserve every previously passing case. Disk-interrupted logs remain alongside complete retries; no partial audit was accepted.
Same-correctness 30-round alternating comparison: import graph32.4±1.7ms to28.4±0.6ms, 12.4% less time and30/30 paired wins. Barrel27.9±5.2ms to27.3±4.2ms, only2.3% lower with17/30 paired wins: inconclusive, not a reliable improvement. All60 rounds/120 successful samples retained.
Full measurement20260828T234350Z validates570 rounds/1710 successful finite samples, with all513 input files byte-for-byte unchanged. Home has lower means on19/19: import graph28.1ms vs TS7 47.0ms (1.67x), barrel25.6ms vs42.7ms (1.67x), large predicates340.5±18.2ms vs362.2±30.4ms (1.06x;29/30 paired wins). Several rows have substantial shared-host variance. Independent confirmation20260828T234818Z retains30 rounds/90 samples: large predicates Home358.6±11.7ms vs TS7 388.5±16.7ms (1.08x;30/30 paired wins). It is separate, not substituted or pooled. Selected compiler/native-payload/runtime/launcher paths and hashes match before and after both measurements; automated harness provenance is tracked by#544.
CommonJS instance typing is NOT admitted for timing: TS6/TS7 pass66/66, Home22/66, with all44 missing wrong-type/missing-member rejections retained under#541. Export-list controls remain80/96 (#540). Other remaining gaps include owners12/20, nominal origins44/52, globals32/56 and bound globals44/56. The broader goal stays open: real projects, additional rejection coverage, other platforms and semantic correctness remain unproven. Next priorities are actual source-owned CommonJS require types#541 and precise external hidden-binding diagnostics#540.
Frozen candidate commonjs-query.hyU20d/release-v1/bin/home-tsc, SHA-256573257a819a178c9387f6d02a7992c08f995185f5dbc94034eb5eadc0ae956c0. Full results, retained history and reproduction.
Historical indexed-export checkpoint
Implementation b7b9dd9 (#537) and parser ownership fix d6878e8 (#538), with verified README/report 24a34f4, are pushed to main. Compiler versions are verified TS 6.0.3 and native TS 7.0.2; TS 7/tsgo is one competitor.
All 19/19 unchanged workload admission gates pass. Full measurement 20260828T231439Z validates 570 rounds / 1,710 successful finite samples; fresh generation matches 513 measured input files. Home has lower means on 19/19 workloads, but large predicates have only a noisy 1.02x lead: 371.6 ± 125.5 ms versus TS 7 377.7 ± 66.4 ms, 25/30 paired wins. Independent 30-round confirmation 20260828T231856Z retains all 90 samples: Home 320.4 ± 2.8 ms versus TS 7 331.4 ± 4.6 ms (1.03x; 29/30 paired wins). Confirmation is separate, not substituted into the full table.
The same-correctness alternating before/after comparison for #537 improves 525.6 ± 12.4 ms to 353.5 ± 10.5 ms, 32.7% less time and 30/30 paired wins. Deterministic source visits drop 65,536 to 256 for 128 independent exports. Bound-owner edge indexing preserves semantic query work; no control or failed observation was removed.
New exported-variable-list audit improves 0/96 to 80/96. All 32 positives, 32 wrong-type controls and 16 barrel hidden-name controls pass; 16 direct-hidden diagnostics remain TS2305 instead of TS2459 (#540). Existing audit scores remain: factory 240/240, recursive 288/288, generic classes 120/120, static 84/84, bound classes 52/52, callable identity 56/56, callable unions 256/256, graph discovery 28/28, export origins 32/32, owners 12/20, nominal origins 44/52, globals 32/56, bound globals 44/56. Both competitors pass every audit.
Verified suites: checker 4286, conformance 1417, Program 137, parser 891, driver 187, CLI 69, emitter 499 and benchmark harness 72 tests all pass. Pickier reports no errors and one pre-existing README fragment warning. Frozen evidence export-index.NVcIdu, SHA-256 615e9fa1f78c2cad17a0b8911b897ede73c00422138d6ecbb2b74801d4564f14.
Next: #536 removes redundant CommonJS export-query compilation; #540 adds external owner-local diagnostic facts. Correctness gaps, expanded rejection coverage, real projects and cross-platform measurements keep the overall goal open. Shared-host synthetic results are not universal benchmark leadership. Full results and reproduction.
Historical exported-factory checkpoint
Implementation de8fe28 (#534), documentation/README f3740df. Executables are verified TS 6.0.3 and native TS 7.0.2; native TS7/tsgo is one competitor.
All 19/19 workload admission gates now pass for all three compilers. Both original graph projects and their unchanged TS2322/TS2339 rejection controls pass; no failing workload was removed. The 240 exported-factory controls improve from Home130/240 to240/240. Recursive288/288, generic classes120/120, static84/84, bound classes52/52, callable identity56/56, callable unions256/256, graph discovery28/28 and export origins32/32 remain passing. Imported owners improve10/20 to12/20; nominal origins44/52, globals32/56 and bound globals44/56 remain incomplete. Both competitors pass all these audit cases.
Full relevant suites: checker4286/4286, conformance1417/1417, Program133/133, driver187/187, CLI69/69, source-owned focus10/10, harness69/69.
Complete measurement 20260828T224220Z: 30 interleaved rounds after3 warmups, all570 round files /1710 samples validate; fresh generation matches all513 measured input files. Shared Apple M3 Pro workstation with unrelated CPU/filesystem activity. Home has lower means on 18/19 workloads, not universal leadership. Import graph33.7ms vs TS7 50.8ms (1.51x); barrel graph25.1ms vs41.1ms (1.63x). Large predicates lose: Home541.3 ±50.7ms vs TS7 380.6 ±44.8ms, 1.42x slower and0/30 paired wins. That losing row is retained in README and full docs.
Next: #537 establishes and fixes the large-predicate scaling cause against a same-correctness controlled baseline; #536 removes redundant CommonJS export-query compilation. Remaining semantic failures, broader rejection coverage, real-project and cross-platform measurements keep the overall goal open. Frozen evidence: graph-types.4EsbvV; binary SHA256 9f5e333beaf9828a9720ce260ae05a872ebcea3bb51bf5785ffa3ccd7b72f1bc. Documentation: docs/docs/TS_PERFORMANCE.md#exported-factory-checkpoint-performance.
Historical checkpoint before exported-factory typing
Checkpoint: implementation 3b7f0ea (#524), recursive throughput workload de72563, report-integrity fix 74d1163, and documentation/README 91d6ca8. Pinned executables are TS 6.0.3 and native TS 7.0.2; TS 7 and tsgo are one competitor.
The original admission set remains 16/18 for Home: both imported graph rejection gates fail and were not timed. The added recursive workload passes its positive and nine appended negative controls, making expanded admission 17/19 (both competitors 19/19). No failure is removed by expanding the denominator.
| Untimed correctness gate | TS 6.0.3 | Native TS 7.0.2 | Home |
|---|---|---|---|
| Global declarations | 56/56 | 56/56 | 32/56 |
| Bound-global discovery | 56/56 | 56/56 | 44/56 |
| Imported owners | 20/20 | 20/20 | 10/20 |
| Nominal identity | 52/52 | 52/52 | 44/52 |
| Imported generic classes | 120/120 | 120/120 | 120/120 |
| Recursive generic consumers | 288/288 | 288/288 | 288/288 |
| Import/re-export graph admission | 2/2 | 2/2 | 0/2 |
Lazy source-owned definitions improve recursive controls from 112/288 to 288/288 and imported generic classes from 118/120 to 120/120. Local non-exported classes remain non-exported; transparent aliases preserve their target's origin. No fixed-depth unfolding or new permissive fallback. Inheritance, global/owner typing, broader generic forms and original graph linkage remain open (#521, #480, #487).
Fresh full timing 20260828T221149Z has lower Home means on 17/17 admitted timed workloads, with 510 complete round files and 1,530 successful samples. Apple M3 Pro, 30 interleaved runs after three warmups, shared-workstation load. Recursive generics: Home 29.7 ± 0.2 ms, native TS 7 71.8 ± 1.1 ms, TS 6 151.7 ± 2.2 ms (2.42× lower mean). Large predicates: Home 377.0 ± 100.9 ms, native TS 7 393.6 ± 94.1 ms (1.04× lower mean; noisy). Other rows also have substantial variance.
Independent confirmation 20260828T221611Z retains 60 complete rounds / 180 successful samples. Recursive generics: Home 29.6 ± 0.7 ms vs native 73.1 ± 3.9 ms (2.47×; Home faster in 30/30 paired rounds). Large predicates: Home 349.9 ± 17.0 ms vs native 392.7 ± 83.0 ms (1.12×; 28/30 paired rounds). Confirmation does not replace or get averaged into the full run. The predecessor fails recursive rejection controls and is ineligible for a before/after timing claim.
Exact-source tests: checker 4285/4285, conformance 1417/1417, Program 131/131, driver 187/187, CLI 69/69, focused source-owned 9/9, harness 66/66. Release SHA-256: 267691e33056e2c4a8b23b757bc3eed19c8b3d90fa9407120b8985834a844ccd. Reporting now rejects incomplete/extra rounds, incorrect compiler coverage/order, malformed JSON, and failed/invalid samples instead of publishing partial averages.
Full results, variance, preserved history and reproduction. Correctness details.
Not complete: these are local synthetic observations under shared-host load, not universal, real-project or cross-platform leadership. The two original graph gates and broader semantic coverage still fail or remain unverified. The tracker stays open.
Historical measured snapshot (not current admission)
Apple M3 Pro, 30 interleaved runs after three warmups; raw full-suite result 20260827T202656Z, compiler commit 9e45e10, documentation commit 75e02f3. Every observation is retained, including variance and outliers.
| Workload | tsc 6.0.3 | native TS 7.0.2 | Home 0.1.0 | Home vs fastest competitor |
|---|---|---|---|---|
startup | 64.4 ± 1.8 ms | 42.2 ± 11.3 ms | 3.9 ± 2.7 ms | 10.82× faster |
many_files | 213.2 ± 14.8 ms | 53.5 ± 3.1 ms | 30.0 ± 1.5 ms | 1.78× faster |
deep_types | 127.9 ± 2.2 ms | 52.6 ± 1.5 ms | 12.9 ± 0.3 ms | 4.09× faster |
import_graph | 129.2 ± 1.9 ms | 45.0 ± 0.9 ms | 25.8 ± 1.1 ms | Ineligible: fails semantic controls |
reexport_graph | 94.7 ± 3.0 ms | 40.7 ± 0.9 ms | 26.8 ± 0.6 ms | Ineligible: fails semantic controls |
tsx_components | 159.1 ± 1.2 ms | 46.3 ± 0.9 ms | 21.3 ± 0.3 ms | 2.17× faster |
generic_calls | 177.5 ± 4.4 ms | 55.4 ± 11.2 ms | 22.0 ± 1.8 ms | 2.52× faster |
control_flow | 212.2 ± 127.9 ms | 63.1 ± 22.3 ms | 32.9 ± 3.8 ms | 1.92× faster |
type_predicates | 237.3 ± 14.0 ms | 72.4 ± 10.8 ms | 46.5 ± 19.5 ms | 1.56× faster |
type_predicates_large | 1027.8 ± 117.6 ms | 348.8 ± 15.7 ms | 322.1 ± 13.3 ms | 1.08× faster |
null_safe_access | 206.3 ± 12.4 ms | 60.3 ± 2.4 ms | 40.6 ± 1.8 ms | 1.49× faster |
destructuring | 144.3 ± 11.1 ms | 48.5 ± 1.1 ms | 35.3 ± 0.8 ms | 1.38× faster |
overload_resolution | 222.7 ± 20.6 ms | 71.6 ± 10.7 ms | 30.9 ± 2.9 ms | 2.32× faster |
class_hierarchy | 200.0 ± 17.6 ms | 55.8 ± 4.5 ms | 29.8 ± 1.1 ms | 1.87× faster |
structural_objects | 197.7 ± 9.7 ms | 60.7 ± 2.8 ms | 31.5 ± 1.1 ms | 1.93× faster |
interface_composition | 233.3 ± 43.6 ms | 76.2 ± 30.3 ms | 46.5 ± 3.4 ms | 1.64× faster |
variadic_tuples | 268.4 ± 33.0 ms | 81.0 ± 8.5 ms | 46.2 ± 8.3 ms | Provisional: older admission schema |
checkjs_jsdoc | 246.4 ± 48.7 ms | 63.0 ± 16.6 ms | 41.2 ± 6.8 ms | 1.53× faster |
Historical snapshot: 18/18 lower Home means were recorded, with stronger primitive-return checking. This is not current acceptance: the two graph claims were subsequently withdrawn and later compiler changes have not been retimed. Large predicates have a 1.08× mean lead. The predeclared independent 30-round confirmation 20260827T203044Z records Home 342.5 ± 29.3 ms vs native TS 7 370.2 ± 22.0 ms (1.08× lead); TS 6 is 1086.3 ± 96.2 ms. Home is faster than native in 29/30 paired large-case rounds in the full run and 26/30 in the confirmation. The confirmation does not replace or get averaged into the full table. All 1,710 timed samples succeeded; exact versions, balanced ordering, published tables, preserved binary, and frozen predicate hashes were verified.
All 99 program, 4,226 checker, 173 driver, and 16 harness/report tests pass, as do all 18 shared positive projects and 13 negative controls per compiler. Five new return-contract regression groups have been independently checked with the same CLI inputs against both TS baselines (#474). Async is not an accepted workload: #475 and #476 still fail semantic controls, and #478 tracks diagnostic wording. Do not infer full TypeScript parity from these counts.
These are local synthetic results, not proof of universal, real-project, or cross-platform leadership. Several rows have substantial variance; no observations were discarded. The correctness fix is not itself claimed as a performance optimization. Full methods, history, and reproduction.
Previous 18-workload snapshot (retained)
Previous measured snapshot
Apple M3 Pro, 30 interleaved runs after three warmups; raw full-suite result 20260827T195539Z, compiler commit d330198, documentation commit 20084a4. Every observation is retained, including variance and outliers.
| Workload | tsc 6.0.3 | native TS 7.0.2 | Home 0.1.0 | Home result |
|---|---|---|---|---|
startup | 66.7 ± 2.5 ms | 42.0 ± 9.6 ms | 3.6 ± 0.4 ms | 11.73× faster |
many_files | 215.4 ± 3.3 ms | 56.7 ± 7.5 ms | 32.4 ± 6.3 ms | 1.75× faster |
deep_types | 146.0 ± 8.6 ms | 58.3 ± 3.7 ms | 14.1 ± 0.8 ms | 4.12× faster |
import_graph | 139.0 ± 4.4 ms | 49.8 ± 4.1 ms | 29.3 ± 2.0 ms | 1.70× faster |
reexport_graph | 111.0 ± 46.4 ms | 47.0 ± 19.2 ms | 31.1 ± 6.1 ms | 1.51× faster |
tsx_components | 178.0 ± 25.2 ms | 50.2 ± 1.6 ms | 23.3 ± 4.1 ms | 2.16× faster |
generic_calls | 188.4 ± 12.9 ms | 57.3 ± 3.9 ms | 23.2 ± 2.0 ms | 2.47× faster |
control_flow | 190.2 ± 4.8 ms | 59.5 ± 2.2 ms | 33.6 ± 5.4 ms | 1.77× faster |
type_predicates | 237.8 ± 12.7 ms | 70.9 ± 2.0 ms | 41.9 ± 0.8 ms | 1.69× faster |
type_predicates_large | 1010.4 ± 25.3 ms | 351.4 ± 15.2 ms | 323.8 ± 10.2 ms | 1.09× faster |
null_safe_access | 191.9 ± 4.1 ms | 57.2 ± 1.1 ms | 38.7 ± 0.5 ms | 1.48× faster |
destructuring | 138.4 ± 1.6 ms | 47.3 ± 1.0 ms | 34.4 ± 0.5 ms | 1.37× faster |
overload_resolution | 203.1 ± 3.4 ms | 64.6 ± 1.4 ms | 29.1 ± 0.6 ms | 2.22× faster |
class_hierarchy | 186.1 ± 7.9 ms | 53.4 ± 7.8 ms | 28.1 ± 0.4 ms | 1.90× faster |
structural_objects | 185.8 ± 2.9 ms | 58.1 ± 1.6 ms | 30.3 ± 0.6 ms | 1.92× faster |
interface_composition | 214.9 ± 41.7 ms | 64.7 ± 4.3 ms | 45.6 ± 6.0 ms | 1.42× faster |
variadic_tuples | 252.9 ± 6.3 ms | 77.7 ± 2.5 ms | 42.4 ± 1.0 ms | 1.83× faster |
checkjs_jsdoc | 211.5 ± 27.6 ms | 54.3 ± 1.2 ms | 36.9 ± 0.9 ms | 1.47× faster |
Then-reported snapshot: 18/18 lower Home means; retained as history, not current admission or a restored graph speed claim. Large predicates have a 1.09× mean lead. The predeclared independent 30-round confirmation 20260827T200026Z records Home 327.1 ± 10.4 ms vs native TS 7 359.7 ± 14.2 ms (1.10× lead). The confirmation does not replace or get averaged into the full table. All 1,710 timed samples succeeded; exact versions, balanced ordering, both tables, and frozen predicate hashes were verified. All 99 program, 4,221 checker, 173 driver, and 16 harness/report tests pass, as do all shared positive projects and negative controls.
These are local synthetic results, not proof of universal, real-project, or cross-platform leadership. Full methods, retained history, and reproduction: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md
Earlier 13-workload snapshot (retained)
Earlier snapshot
Apple M3 Pro, 30 interleaved runs after three warmups, mean wall time; raw result ID 20260827T085121Z. Background workstation activity increased absolute time and variance; the report retains every sample and rotates compiler order.
| Workload | tsc 6.0.3 | native TS 7.0.2 | Home 0.1.0 | Home result |
|---|---|---|---|---|
| startup | 109.0 ms | 60.2 ms | 6.7 ms | 8.99× faster |
| many_files | 426.6 ms | 93.6 ms | 54.1 ms | 1.73× faster |
| deep_types | 260.0 ms | 90.3 ms | 28.9 ms | 3.12× faster |
| import_graph | 260.0 ms | 79.3 ms | 48.2 ms | 1.65× faster |
| reexport_graph | 195.6 ms | 72.6 ms | 57.9 ms | 1.25× faster |
| tsx_components | 364.1 ms | 90.5 ms | 44.5 ms | 2.03× faster |
| generic_calls | 360.5 ms | 93.8 ms | 51.7 ms | 1.81× faster |
| control_flow | 392.2 ms | 100.3 ms | 77.1 ms | 1.30× faster |
| overload_resolution | 491.2 ms | 132.3 ms | 88.5 ms | 1.49× faster |
| class_hierarchy | 406.5 ms | 96.0 ms | 83.7 ms | 1.15× faster |
| structural_objects | 361.8 ms | 102.3 ms | 69.4 ms | 1.47× faster |
| interface_composition | 532.6 ms | 133.1 ms | 122.8 ms | 1.08× faster |
| variadic_tuples | 634.1 ms | 160.8 ms | 153.5 ms | 1.05× faster |
Then-accepted suite: 13/13 lower Home means.
Next coverage
Preserve actual global declaration types, merging and lexical isolation across files (#480)
Preserve generic imported/re-exported types and owner metadata, including source-order/cycle handling; readmit graph timing only after the unchanged controls pass (#487)
Add shared async/await and Promise type-propagation coverage with negative controls (#473; remaining #475, #476, #478)
Validate the legacy container path before claiming cross-platform results (#464)
Resolve independently tracked semantic/source-lifetime gaps (#467, #472)
Add a fair emit workload when output semantics are validated across all compilers
Add pinned real-world projects only where all compilers accept the same graph and configuration
Profile and resolve every regression or loss introduced by accepted workloads
Keep README and detailed performance documentation synchronized
Commit links and before/after measurements are maintained in the issue comments. The tracker remains open while coverage expands.