ReviewOS

home-lang/home

Track TypeScript frontend benchmark leadership

#416
Open chris opened this 14 days ago · 283 comments
14 days ago

Goal

Maintain a fair, reproducible TypeScript frontend performance suite and improve Home until it leads JavaScript tsc 6 and stable native TypeScript 7 across every accepted workload.

Benchmark rules

  • All compilers receive identical valid source graphs and compiler options.
  • Every compiler must validate successfully and silently before timing, and reject the same invalid controls with the expected diagnostics.
  • All selected workloads must pass admission before any timing or result-directory creation; no failing row is silently omitted.
  • Compiler versions and generated workloads are pinned.
  • JavaScript TypeScript 6 and native TypeScript 7 are installed in isolated prefixes.
  • Measurements rotate compiler order within every round to balance workstation load.
  • Workloads represent general compiler behavior, never benchmark-specific shortcuts.
  • Results, environment details, caveats, and reproduction commands stay documented.

Landed foundation

  • Nineteen deterministic workloads covering startup, files, complex types, module graphs, TSX, inference, control flow, predicates at two scales, null-safe access, destructuring, overloads, classes, structural objects, interface composition, variadic tuples, checked JavaScript, and recursive generic consumers
  • Pinned JavaScript tsc 6.0.3 and stable native TypeScript 7.0.2 toolchain
  • Interleaved Hyperfine measurement schedule and silent-success validity gate
  • Release allocator, reusable source I/O, and bounded large-program parallelism
  • Cached source facts, bind-only export analyses, and projected export-name sets
  • Indexed lexical declarations, interface merge predecessors, namespaces, annotations, and validated type display names
  • Pruned provably irrelevant overload, inference, import, expando, and compatibility work
  • Relation-cache working-set retention and single-probe insertion
  • Full performance documentation and linked README summary

Current correctness and measured status

Implementation 1cfc33c (#536), audit d95dfaa (#541), and verified README/report 707045c are pushed to main as Chris with no coauthors. TS 6.0.3 and native TS 7.0.2 are verified; TS7/tsgo is one competitor.

The prepared-owner CommonJS display query removes the extra full source compilation and raw-text recovery. Program140/140, conformance1418/1418, CLI69/69 and harness75/75 pass; unchanged checker/driver targets verify from cache. All19 unchanged admission gates and all15 complete audit case sets preserve every previously passing case. Disk-interrupted logs remain alongside complete retries; no partial audit was accepted.

Same-correctness 30-round alternating comparison: import graph32.4±1.7ms to28.4±0.6ms, 12.4% less time and30/30 paired wins. Barrel27.9±5.2ms to27.3±4.2ms, only2.3% lower with17/30 paired wins: inconclusive, not a reliable improvement. All60 rounds/120 successful samples retained.

Full measurement20260828T234350Z validates570 rounds/1710 successful finite samples, with all513 input files byte-for-byte unchanged. Home has lower means on19/19: import graph28.1ms vs TS7 47.0ms (1.67x), barrel25.6ms vs42.7ms (1.67x), large predicates340.5±18.2ms vs362.2±30.4ms (1.06x;29/30 paired wins). Several rows have substantial shared-host variance. Independent confirmation20260828T234818Z retains30 rounds/90 samples: large predicates Home358.6±11.7ms vs TS7 388.5±16.7ms (1.08x;30/30 paired wins). It is separate, not substituted or pooled. Selected compiler/native-payload/runtime/launcher paths and hashes match before and after both measurements; automated harness provenance is tracked by#544.

CommonJS instance typing is NOT admitted for timing: TS6/TS7 pass66/66, Home22/66, with all44 missing wrong-type/missing-member rejections retained under#541. Export-list controls remain80/96 (#540). Other remaining gaps include owners12/20, nominal origins44/52, globals32/56 and bound globals44/56. The broader goal stays open: real projects, additional rejection coverage, other platforms and semantic correctness remain unproven. Next priorities are actual source-owned CommonJS require types#541 and precise external hidden-binding diagnostics#540.

Frozen candidate commonjs-query.hyU20d/release-v1/bin/home-tsc, SHA-256573257a819a178c9387f6d02a7992c08f995185f5dbc94034eb5eadc0ae956c0. Full results, retained history and reproduction.

Historical indexed-export checkpoint

Implementation b7b9dd9 (#537) and parser ownership fix d6878e8 (#538), with verified README/report 24a34f4, are pushed to main. Compiler versions are verified TS 6.0.3 and native TS 7.0.2; TS 7/tsgo is one competitor.

All 19/19 unchanged workload admission gates pass. Full measurement 20260828T231439Z validates 570 rounds / 1,710 successful finite samples; fresh generation matches 513 measured input files. Home has lower means on 19/19 workloads, but large predicates have only a noisy 1.02x lead: 371.6 ± 125.5 ms versus TS 7 377.7 ± 66.4 ms, 25/30 paired wins. Independent 30-round confirmation 20260828T231856Z retains all 90 samples: Home 320.4 ± 2.8 ms versus TS 7 331.4 ± 4.6 ms (1.03x; 29/30 paired wins). Confirmation is separate, not substituted into the full table.

The same-correctness alternating before/after comparison for #537 improves 525.6 ± 12.4 ms to 353.5 ± 10.5 ms, 32.7% less time and 30/30 paired wins. Deterministic source visits drop 65,536 to 256 for 128 independent exports. Bound-owner edge indexing preserves semantic query work; no control or failed observation was removed.

New exported-variable-list audit improves 0/96 to 80/96. All 32 positives, 32 wrong-type controls and 16 barrel hidden-name controls pass; 16 direct-hidden diagnostics remain TS2305 instead of TS2459 (#540). Existing audit scores remain: factory 240/240, recursive 288/288, generic classes 120/120, static 84/84, bound classes 52/52, callable identity 56/56, callable unions 256/256, graph discovery 28/28, export origins 32/32, owners 12/20, nominal origins 44/52, globals 32/56, bound globals 44/56. Both competitors pass every audit.

Verified suites: checker 4286, conformance 1417, Program 137, parser 891, driver 187, CLI 69, emitter 499 and benchmark harness 72 tests all pass. Pickier reports no errors and one pre-existing README fragment warning. Frozen evidence export-index.NVcIdu, SHA-256 615e9fa1f78c2cad17a0b8911b897ede73c00422138d6ecbb2b74801d4564f14.

Next: #536 removes redundant CommonJS export-query compilation; #540 adds external owner-local diagnostic facts. Correctness gaps, expanded rejection coverage, real projects and cross-platform measurements keep the overall goal open. Shared-host synthetic results are not universal benchmark leadership. Full results and reproduction.

Historical exported-factory checkpoint

Implementation de8fe28 (#534), documentation/README f3740df. Executables are verified TS 6.0.3 and native TS 7.0.2; native TS7/tsgo is one competitor.

All 19/19 workload admission gates now pass for all three compilers. Both original graph projects and their unchanged TS2322/TS2339 rejection controls pass; no failing workload was removed. The 240 exported-factory controls improve from Home130/240 to240/240. Recursive288/288, generic classes120/120, static84/84, bound classes52/52, callable identity56/56, callable unions256/256, graph discovery28/28 and export origins32/32 remain passing. Imported owners improve10/20 to12/20; nominal origins44/52, globals32/56 and bound globals44/56 remain incomplete. Both competitors pass all these audit cases.

Full relevant suites: checker4286/4286, conformance1417/1417, Program133/133, driver187/187, CLI69/69, source-owned focus10/10, harness69/69.

Complete measurement 20260828T224220Z: 30 interleaved rounds after3 warmups, all570 round files /1710 samples validate; fresh generation matches all513 measured input files. Shared Apple M3 Pro workstation with unrelated CPU/filesystem activity. Home has lower means on 18/19 workloads, not universal leadership. Import graph33.7ms vs TS7 50.8ms (1.51x); barrel graph25.1ms vs41.1ms (1.63x). Large predicates lose: Home541.3 ±50.7ms vs TS7 380.6 ±44.8ms, 1.42x slower and0/30 paired wins. That losing row is retained in README and full docs.

Next: #537 establishes and fixes the large-predicate scaling cause against a same-correctness controlled baseline; #536 removes redundant CommonJS export-query compilation. Remaining semantic failures, broader rejection coverage, real-project and cross-platform measurements keep the overall goal open. Frozen evidence: graph-types.4EsbvV; binary SHA256 9f5e333beaf9828a9720ce260ae05a872ebcea3bb51bf5785ffa3ccd7b72f1bc. Documentation: docs/docs/TS_PERFORMANCE.md#exported-factory-checkpoint-performance.

Historical checkpoint before exported-factory typing

Checkpoint: implementation 3b7f0ea (#524), recursive throughput workload de72563, report-integrity fix 74d1163, and documentation/README 91d6ca8. Pinned executables are TS 6.0.3 and native TS 7.0.2; TS 7 and tsgo are one competitor.

The original admission set remains 16/18 for Home: both imported graph rejection gates fail and were not timed. The added recursive workload passes its positive and nine appended negative controls, making expanded admission 17/19 (both competitors 19/19). No failure is removed by expanding the denominator.

Untimed correctness gateTS 6.0.3Native TS 7.0.2Home
Global declarations56/5656/5632/56
Bound-global discovery56/5656/5644/56
Imported owners20/2020/2010/20
Nominal identity52/5252/5244/52
Imported generic classes120/120120/120120/120
Recursive generic consumers288/288288/288288/288
Import/re-export graph admission2/22/20/2

Lazy source-owned definitions improve recursive controls from 112/288 to 288/288 and imported generic classes from 118/120 to 120/120. Local non-exported classes remain non-exported; transparent aliases preserve their target's origin. No fixed-depth unfolding or new permissive fallback. Inheritance, global/owner typing, broader generic forms and original graph linkage remain open (#521, #480, #487).

Fresh full timing 20260828T221149Z has lower Home means on 17/17 admitted timed workloads, with 510 complete round files and 1,530 successful samples. Apple M3 Pro, 30 interleaved runs after three warmups, shared-workstation load. Recursive generics: Home 29.7 ± 0.2 ms, native TS 7 71.8 ± 1.1 ms, TS 6 151.7 ± 2.2 ms (2.42× lower mean). Large predicates: Home 377.0 ± 100.9 ms, native TS 7 393.6 ± 94.1 ms (1.04× lower mean; noisy). Other rows also have substantial variance.

Independent confirmation 20260828T221611Z retains 60 complete rounds / 180 successful samples. Recursive generics: Home 29.6 ± 0.7 ms vs native 73.1 ± 3.9 ms (2.47×; Home faster in 30/30 paired rounds). Large predicates: Home 349.9 ± 17.0 ms vs native 392.7 ± 83.0 ms (1.12×; 28/30 paired rounds). Confirmation does not replace or get averaged into the full run. The predecessor fails recursive rejection controls and is ineligible for a before/after timing claim.

Exact-source tests: checker 4285/4285, conformance 1417/1417, Program 131/131, driver 187/187, CLI 69/69, focused source-owned 9/9, harness 66/66. Release SHA-256: 267691e33056e2c4a8b23b757bc3eed19c8b3d90fa9407120b8985834a844ccd. Reporting now rejects incomplete/extra rounds, incorrect compiler coverage/order, malformed JSON, and failed/invalid samples instead of publishing partial averages.

Full results, variance, preserved history and reproduction. Correctness details.

Not complete: these are local synthetic observations under shared-host load, not universal, real-project or cross-platform leadership. The two original graph gates and broader semantic coverage still fail or remain unverified. The tracker stays open.

Historical measured snapshot (not current admission)

Apple M3 Pro, 30 interleaved runs after three warmups; raw full-suite result 20260827T202656Z, compiler commit 9e45e10, documentation commit 75e02f3. Every observation is retained, including variance and outliers.

Workloadtsc 6.0.3native TS 7.0.2Home 0.1.0Home vs fastest competitor
startup64.4 ± 1.8 ms42.2 ± 11.3 ms3.9 ± 2.7 ms10.82× faster
many_files213.2 ± 14.8 ms53.5 ± 3.1 ms30.0 ± 1.5 ms1.78× faster
deep_types127.9 ± 2.2 ms52.6 ± 1.5 ms12.9 ± 0.3 ms4.09× faster
import_graph129.2 ± 1.9 ms45.0 ± 0.9 ms25.8 ± 1.1 msIneligible: fails semantic controls
reexport_graph94.7 ± 3.0 ms40.7 ± 0.9 ms26.8 ± 0.6 msIneligible: fails semantic controls
tsx_components159.1 ± 1.2 ms46.3 ± 0.9 ms21.3 ± 0.3 ms2.17× faster
generic_calls177.5 ± 4.4 ms55.4 ± 11.2 ms22.0 ± 1.8 ms2.52× faster
control_flow212.2 ± 127.9 ms63.1 ± 22.3 ms32.9 ± 3.8 ms1.92× faster
type_predicates237.3 ± 14.0 ms72.4 ± 10.8 ms46.5 ± 19.5 ms1.56× faster
type_predicates_large1027.8 ± 117.6 ms348.8 ± 15.7 ms322.1 ± 13.3 ms1.08× faster
null_safe_access206.3 ± 12.4 ms60.3 ± 2.4 ms40.6 ± 1.8 ms1.49× faster
destructuring144.3 ± 11.1 ms48.5 ± 1.1 ms35.3 ± 0.8 ms1.38× faster
overload_resolution222.7 ± 20.6 ms71.6 ± 10.7 ms30.9 ± 2.9 ms2.32× faster
class_hierarchy200.0 ± 17.6 ms55.8 ± 4.5 ms29.8 ± 1.1 ms1.87× faster
structural_objects197.7 ± 9.7 ms60.7 ± 2.8 ms31.5 ± 1.1 ms1.93× faster
interface_composition233.3 ± 43.6 ms76.2 ± 30.3 ms46.5 ± 3.4 ms1.64× faster
variadic_tuples268.4 ± 33.0 ms81.0 ± 8.5 ms46.2 ± 8.3 msProvisional: older admission schema
checkjs_jsdoc246.4 ± 48.7 ms63.0 ± 16.6 ms41.2 ± 6.8 ms1.53× faster

Historical snapshot: 18/18 lower Home means were recorded, with stronger primitive-return checking. This is not current acceptance: the two graph claims were subsequently withdrawn and later compiler changes have not been retimed. Large predicates have a 1.08× mean lead. The predeclared independent 30-round confirmation 20260827T203044Z records Home 342.5 ± 29.3 ms vs native TS 7 370.2 ± 22.0 ms (1.08× lead); TS 6 is 1086.3 ± 96.2 ms. Home is faster than native in 29/30 paired large-case rounds in the full run and 26/30 in the confirmation. The confirmation does not replace or get averaged into the full table. All 1,710 timed samples succeeded; exact versions, balanced ordering, published tables, preserved binary, and frozen predicate hashes were verified.

All 99 program, 4,226 checker, 173 driver, and 16 harness/report tests pass, as do all 18 shared positive projects and 13 negative controls per compiler. Five new return-contract regression groups have been independently checked with the same CLI inputs against both TS baselines (#474). Async is not an accepted workload: #475 and #476 still fail semantic controls, and #478 tracks diagnostic wording. Do not infer full TypeScript parity from these counts.

These are local synthetic results, not proof of universal, real-project, or cross-platform leadership. Several rows have substantial variance; no observations were discarded. The correctness fix is not itself claimed as a performance optimization. Full methods, history, and reproduction.

Previous 18-workload snapshot (retained)

Previous measured snapshot

Apple M3 Pro, 30 interleaved runs after three warmups; raw full-suite result 20260827T195539Z, compiler commit d330198, documentation commit 20084a4. Every observation is retained, including variance and outliers.

Workloadtsc 6.0.3native TS 7.0.2Home 0.1.0Home result
startup66.7 ± 2.5 ms42.0 ± 9.6 ms3.6 ± 0.4 ms11.73× faster
many_files215.4 ± 3.3 ms56.7 ± 7.5 ms32.4 ± 6.3 ms1.75× faster
deep_types146.0 ± 8.6 ms58.3 ± 3.7 ms14.1 ± 0.8 ms4.12× faster
import_graph139.0 ± 4.4 ms49.8 ± 4.1 ms29.3 ± 2.0 ms1.70× faster
reexport_graph111.0 ± 46.4 ms47.0 ± 19.2 ms31.1 ± 6.1 ms1.51× faster
tsx_components178.0 ± 25.2 ms50.2 ± 1.6 ms23.3 ± 4.1 ms2.16× faster
generic_calls188.4 ± 12.9 ms57.3 ± 3.9 ms23.2 ± 2.0 ms2.47× faster
control_flow190.2 ± 4.8 ms59.5 ± 2.2 ms33.6 ± 5.4 ms1.77× faster
type_predicates237.8 ± 12.7 ms70.9 ± 2.0 ms41.9 ± 0.8 ms1.69× faster
type_predicates_large1010.4 ± 25.3 ms351.4 ± 15.2 ms323.8 ± 10.2 ms1.09× faster
null_safe_access191.9 ± 4.1 ms57.2 ± 1.1 ms38.7 ± 0.5 ms1.48× faster
destructuring138.4 ± 1.6 ms47.3 ± 1.0 ms34.4 ± 0.5 ms1.37× faster
overload_resolution203.1 ± 3.4 ms64.6 ± 1.4 ms29.1 ± 0.6 ms2.22× faster
class_hierarchy186.1 ± 7.9 ms53.4 ± 7.8 ms28.1 ± 0.4 ms1.90× faster
structural_objects185.8 ± 2.9 ms58.1 ± 1.6 ms30.3 ± 0.6 ms1.92× faster
interface_composition214.9 ± 41.7 ms64.7 ± 4.3 ms45.6 ± 6.0 ms1.42× faster
variadic_tuples252.9 ± 6.3 ms77.7 ± 2.5 ms42.4 ± 1.0 ms1.83× faster
checkjs_jsdoc211.5 ± 27.6 ms54.3 ± 1.2 ms36.9 ± 0.9 ms1.47× faster

Then-reported snapshot: 18/18 lower Home means; retained as history, not current admission or a restored graph speed claim. Large predicates have a 1.09× mean lead. The predeclared independent 30-round confirmation 20260827T200026Z records Home 327.1 ± 10.4 ms vs native TS 7 359.7 ± 14.2 ms (1.10× lead). The confirmation does not replace or get averaged into the full table. All 1,710 timed samples succeeded; exact versions, balanced ordering, both tables, and frozen predicate hashes were verified. All 99 program, 4,221 checker, 173 driver, and 16 harness/report tests pass, as do all shared positive projects and negative controls.

These are local synthetic results, not proof of universal, real-project, or cross-platform leadership. Full methods, retained history, and reproduction: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md

Earlier 13-workload snapshot (retained)

Earlier snapshot

Apple M3 Pro, 30 interleaved runs after three warmups, mean wall time; raw result ID 20260827T085121Z. Background workstation activity increased absolute time and variance; the report retains every sample and rotates compiler order.

Workloadtsc 6.0.3native TS 7.0.2Home 0.1.0Home result
startup109.0 ms60.2 ms6.7 ms8.99× faster
many_files426.6 ms93.6 ms54.1 ms1.73× faster
deep_types260.0 ms90.3 ms28.9 ms3.12× faster
import_graph260.0 ms79.3 ms48.2 ms1.65× faster
reexport_graph195.6 ms72.6 ms57.9 ms1.25× faster
tsx_components364.1 ms90.5 ms44.5 ms2.03× faster
generic_calls360.5 ms93.8 ms51.7 ms1.81× faster
control_flow392.2 ms100.3 ms77.1 ms1.30× faster
overload_resolution491.2 ms132.3 ms88.5 ms1.49× faster
class_hierarchy406.5 ms96.0 ms83.7 ms1.15× faster
structural_objects361.8 ms102.3 ms69.4 ms1.47× faster
interface_composition532.6 ms133.1 ms122.8 ms1.08× faster
variadic_tuples634.1 ms160.8 ms153.5 ms1.05× faster

Then-accepted suite: 13/13 lower Home means.

Next coverage

  • Preserve actual global declaration types, merging and lexical isolation across files (#480)

  • Preserve generic imported/re-exported types and owner metadata, including source-order/cycle handling; readmit graph timing only after the unchanged controls pass (#487)

  • Add shared async/await and Promise type-propagation coverage with negative controls (#473; remaining #475, #476, #478)

  • Validate the legacy container path before claiming cross-platform results (#464)

  • Resolve independently tracked semantic/source-lifetime gaps (#467, #472)

  • Add a fair emit workload when output semantics are validated across all compilers

  • Add pinned real-world projects only where all compilers accept the same graph and configuration

  • Profile and resolve every regression or loss introduced by accepted workloads

  • Keep README and detailed performance documentation synchronized

Commit links and before/after measurements are maintained in the issue comments. The tracker remains open while coverage expands.

14 days ago

New verified correctness checkpoint #524 is on main: 0a1e5af, 33ddbef and docs/README ed12aa1. Home imported generic controls improve 62/120 → 118/120; nominal 42/52 → 44/52. TS 6.0.3 and native TS 7.0.2 are verified executable pins; tsgo and TS 7 are one competitor. Both pass all 120 generic controls. Remaining generic recursion, inheritance and imported graph failures stay documented. Full checker 4279/4279, conformance 1417/1417, Program 130/130, driver 187/187, CLI 69/69 and harness 53/53 pass. Original workload admission remains 16/18, graph admission 0/2; timing snapshot unchanged, no universal leadership claim. Results table and reproduction details: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#imported-generic-class-instantiation-untimed

14 days ago

New #524 checkpoint pushed in small conventional commits: 6576ff8 / 42452f8 fair recursive audit and calibration; c6c6c66 pool-owned generic definitions/transfer; 98e39c6 documentation and README table. Verified TS 6.0.3 and native TS 7.0.2 each pass 216/216; Home passes 96/216. All 120 failures remain, and all 24 nonrecursive calibration cases pass. Foundation release retains identical audit outcomes; shared lazy expansion remains the next implementation step. Checker 4282/4282, conformance 1417/1417, Program 130/130, driver 187/187, CLI 69/69 and harness 56/56 pass. Original admission stays 16/18, graphs 0/2; no new timing or leadership claim. https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#recursive-generic-consumer-baseline-untimed

14 days ago

Reporting-integrity gap found during the new measured checkpoint: compare.py could label an incomplete interleaved run with its requested 30-run count while printing partial averages. A fix is being verified to require the exact workload/round file set, one successful finite positive sample per compiler per round, and the recorded rotating compiler order before rendering an interleaved table. It does not discard or alter measurements. New tests cover missing/extra rounds, partial JSON, duplicated/misordered compilers, invalid samples and failed exits. The ongoing measurement remains running and no partial averages are being published as final results.

14 days ago

CommonJS graph prerequisite checkpoint landed on main: #545 closed via 0ebdba7, eaf86ee and 403385095. Untimed discovery controls improve Home 38/44 to 44/44, matching TS 6.0.3 and native TS 7.0.2; README/docs include the linked table and immutable candidate evidence. The separate #541 instance-type audit remains deliberately unchanged at Home 22/66 versus 66/66 for both competitors, so source-owned checked type transfer remains active work and no CommonJS timing workload has been admitted.

14 days ago

CommonJS checked-owner linkage #541 is complete on main. Implementation fb07dea and docs 2d5859f improve Home from 22/66 to 66/66 on the unchanged untimed audit, matching TypeScript 6.0.3 and native TypeScript/tsgo 7.0.2. All 198 compiler-case checks pass; static require discovery retains 132/132. Program 145/145, harness 78/78, checker, driver, CLI, and conformance (16/16 smoke, 86/86 named, 586/586 baseline-aware) pass. Immutable ReleaseFast SHA-256: 3cab7afcaf38cb5bef789081bc228ab0a970166c0bbd1f2bd24cd61359e823a5. README and full report are synchronized.

This is a correctness result, not a new timing claim. The parent remains open for fair new workloads, real-project coverage, other platforms, and remaining semantic gaps.

14 days ago

Checkpoint #546 is complete and closed via 31cba27, 3e1d2bc, and bf64a67 on origin/main. The new admitted checked CommonJS graph moved from an explicitly retained 1.23x loss to a 1.15x win over native TS/tsgo 7.0.2; exact correct-build Home A/B is 31.1% faster with 30/30 paired wins. The final TS 6.0.3 vs native TS 7.0.2 run records lower Home means on 20/20 admitted workloads with all 1,800 samples retained. The large-predicate lead remains narrow (1.01x, 24/30 paired wins), and real-project/cross-platform/broader semantic coverage remains open here. README and TS_PERFORMANCE.md now link and document the full evidence.

14 days ago

Linux ARM64 checkpoint complete. General compiler work landed in 4971cdb, cea9642, and 6ac9b5e: non-null object-rest bindings skip impossible source scans, resolver filesystem/package state is reused, and JSDoc diagnostic passes are gated by exact parsed tag facts. The final pinned container run 20260829T035150Z compares TS 6.0.3 with the single native TS/tsgo 7.0.2 competitor for 30 interleaved rounds after three warmups. Home records lower means on all 20/20 admitted workloads; all 600 round files and 1,800 successful finite samples are retained. CheckJS is the narrowest mean lead at 1.02x and 19/30 paired wins, so it is documented as narrow rather than universal. The full environment, image digest, compiler/tool provenance, complete table, variance, and reproduction commands are in TS_PERFORMANCE.md and linked from both READMEs. #544 and #464 are closed. This parent remains open for pinned real-project workloads, additional architectures, and broader semantic admission coverage.

14 days ago

Real-project follow-up is active in #548. Zod 4.5.2 now completes in about 4 s / 175 MB after three general checker fixes, versus an interrupted 192.89 s / 3.89 GB diagnostic run. The benchmark remains unadmitted because Home still emits 2,028 false diagnostics while TS 6.0.3 and native TS/tsgo 7.0.2 are silent; no invalid timing has been published.

14 days ago

Real-project gate update: zod@4.5.2 remains excluded from benchmark results because Home is not yet diagnostically equivalent to TS 6.0.3 / native TS 7.0.2. Current ReleaseFast completes in 6.63s at 183,287,808-byte max RSS and reports 1,929 false diagnostics, down from 2,029; TS7006 is down from 1,329 to 1,208. The crash chain exposed by the richer transferred signatures is fixed on main through a59117c. Continuing correctness admission work under #548; no timing win claimed.

14 days ago

TypeScript real-project gate update from #548:

  • c2e4ae42e relates deferred conditional structures and resolves symbolic intersection constituents.
  • fe5e45fd9 preserves inherited interfaces in the cross-file program declaration graph.
  • Exact admitted zod@4.5.2 corpus improves from 1,929 to 1,928 Home diagnostics; TypeScript 6.0.3 and native TypeScript 7.0.2 remain silent controls.
  • Checker and program suites are green.

The project is still not semantically admitted, so no timing or README benchmark claim has been published. Full evidence and the remaining $ZodRegistry rest-parameter mismatch are tracked in /home-lang/home/issues/548#issuecomment-5463498081.

14 days ago

TypeScript compatibility progress: ae8223bf6 fixes structural identity for function patterns inside deferred conditionals. On the exact unmodified Zod 4.5.2 gate, Home improves 1,928 → 1,927 diagnostics and removes the false $ZodRegistry<GlobalMeta> self-assignment error; TS 6.0.3 and native TS 7.0.2 remain silent. Full checker/program suites and negative controls pass. Timing is still withheld until the corpus is semantically admitted. Tracking detail: #548.

14 days ago

Pushed 5d3d80376: unknown index signatures now correctly accept every property value across inline object and class/interface validation. The exact Zod 4.5.2 gate improves 1,927 → 1,915 diagnostics, removing all 12 false TS2411 errors; TS 6.0.3 and native TS 7.0.2 remain silent. Full checker/program suites and negative controls pass. Details: #548.

14 days ago

Pushed 722c25230: explicit generic calls now honor and substitute omitted trailing type-parameter defaults before callback contextual typing. Exact Zod improves 1,915 → 1,914 diagnostics (TS2558 11→10); full checker/program suites and a reduced TS6/TS7/Home multi-file control pass. Next blocker is mapped-type transport for generic exported functions. Details: #548.

14 days ago

TypeScript benchmark admission update: pushed 57261e2 to origin/main.

The fair pinned Zod corpus now drops from 1,914 to 1,798 Home diagnostics (-116), with TS2339 falling from 228 to 119. TypeScript 6.0.3 and TypeScript 7.0.2 (tsgo) remain silent. Positive and negative reduced controls match both references.

No timing or README benchmark claim is published yet because semantic parity is still incomplete. Tracking continues in #548.

14 days ago

Semantic admission update: 2d9773c47 reduces the exact Zod 4.5.2 gate from 1,798 to 1,689 diagnostics. Promise instanceof parity was checked against TypeScript 6.0.3 and native tsgo 7.0.2: positive case silent in all three compilers; negative case exactly TS2322 + TS2339. Main reductions are TS2359 63 -> 4, TS2339 119 -> 92, and TS7006 1,208 -> 1,182. Timing remains withheld until Home is silent and negative controls pass across the full admitted corpus; TS7 and tso are treated as the same tsgo lane, pinned at 7.0.2.

14 days ago

TypeScript parity update: pushed 4c91fc4. Reference lanes are TypeScript 6.0.3 and tsgo 7.0.2 (the TypeScript 7 native lane). The untouched Zod 4.5.2 gate improves 1,689 -> 1,685 diagnostics; the four remaining TS2359 errors are eliminated with no other histogram movement. Focused positive and negative controls match both references, and all checker/program/CLI/build/lint checks pass. Benchmark timing and README results remain gated on semantic admission.

14 days ago

TypeScript parity update: pushed bf9ea7a. Reference lanes remain TypeScript 6.0.3 and tsgo 7.0.2. Correct Array.isArray narrowing improves the untouched Zod 4.5.2 gate from 1,685 to 1,650 diagnostics, including TS2339 92 -> 73 and TS7006 1,182 -> 1,174, with focused positive/negative controls matching both references. All checker/program/CLI/build/lint checks pass. Benchmark timing and README results remain gated on semantic admission.

14 days ago

TypeScript parity update: 71a5be5ab unifies identifier/member typeof narrowing and adds branch-local invalidation after property/root writes. Verified against TS 6.0.3 and tsgo 7.0.2 with positive, negative, and mutation controls. Untouched Zod 4.5.2 production corpus improves 1,650 -> 1,645 diagnostics: TS2339 73 -> 70 and TS2349 2 -> 0, with no new diagnostic family; both references remain silent. Full 4,296-test checker, ts_program, ts_cli, formatting/diff gates, and ReleaseFast pass. Timing/docs remain gated on semantic cleanliness.

14 days ago

TypeScript parity update: a875e2fc5 adds generic WeakSet instance modeling with exact TS 6.0.3 / tsgo 7.0.2 positive and negative control parity. The untouched Zod 4.5.2 production corpus moves 1,645 -> 1,641 diagnostics, with TS2339 70 -> 66 and no other histogram change; both references remain silent. Full 4,297-test checker, ts_program, ts_cli, format/lint/diff gates, and optimized builds pass. Timing/docs remain gated on semantic cleanliness.

14 days ago

TypeScript parity update: 376dc4335 adds structural numeric/bigint typed-array instance modeling and canonical diagnostic names, with TS 6.0.3 / tsgo 7.0.2 positive and negative control parity. Untouched Zod 4.5.2 production corpus moves 1,641 -> 1,639 diagnostics: TS2339 66 -> 65 and TS7053 3 -> 2; all other counts are unchanged and both references stay silent. Full 4,298-test checker, ts_program, ts_cli, formatting/lint/diff gates, and optimized builds pass. Timing/docs remain gated on semantic cleanliness.

14 days ago

Correctness update: f30ea1a is now on main. Cross-file generic interface heritage through Error is preserved without copying per-checker type IDs. Positive and negative controls match TypeScript 6.0.3 and tsgo 7.0.2; all full gates passed. Zod remains at the documented 1,639-diagnostic baseline, so no benchmark timing or README result is being published from this change.

14 days ago

Corpus update from 60aff02 on main: untouched Zod improves 1,639→1,637 diagnostics. Conditional literal-union preservation removes one false TS7053 and one false TS2322; repeated runs held the same total and code histogram. Reference controls match TypeScript 6.0.3 and tsgo 7.0.2, and all gates passed. Timing remains unpublished until Home is silent and the benchmark is fully admissible.

14 days ago

Correctness/performance ratchet update: https://github.com/home-lang/home/commit/6129128cdade10903f3cb1dbbd1ccbd30b42fd26 is on main.

Untouched Zod 4.5.2 production corpus improved 1,637 -> 1,633 diagnostics with no category regression: TS2304 16 -> 14, TS2352 2 -> 1, TS7053 1 -> 0. Two repeated ReleaseFast runs produced the same 1,633-code histogram in 21-22s. Controls agree with TypeScript 6.0.3 and native TypeScript 7.0.2 (tsgo).

A broad declaration-graph route caused a multi-minute slowdown and was withdrawn; the admitted implementation publishes only self-contained literal const schemas across otherwise-incomplete files. Full parser/program/checker/CLI and repository ReleaseFast gates pass. Benchmark timing docs and the README table remain deferred until the fair admitted corpus is silent.

14 days ago

Performance update: 4c6fd02 is on main. The pinned untouched Zod workload now runs in 5.47s and 5.14s in ReleaseFast, down from 21–22s, with the exact same 1,633-diagnostic histogram. This removes repeated cyclic this-type graph expansion; benchmark publication remains gated on correctness admission.

14 days ago

Zod corpus update from #548:\n\n- ac9f942 contextualizes imported generic factory callbacks from result annotations.\n- 40ff168 preserves readonly computed type members such as Symbol.toStringTag.\n\nOn the untouched strict Zod 4.5.2 corpus, TypeScript 6.0.3 and tsgo 7.0.2 remain clean. Home now reports 1,582 diagnostics, down from the parser-admitted 2,038 and below the prior parse-blocked 1,633 baseline. Two Home runs were 23.75s and 23.37s with identical diagnostic positions/codes. Full optimized build and focused gates pass; benchmark docs/README remain gated on a clean admitted corpus.

14 days ago

TypeScript benchmark/parity update: 9ccedba is on main. It correctly resolves callable parameter inference for explicit generic mapped object methods and passes the focused/broad negative controls plus full ReleaseFast build. The untouched Zod 4.5.2 corpus remains at 1,582 diagnostics (25.12s), with positions/codes identical to the prior baseline, because core.$constructor<zodtype> is still seen as untyped after the .js namespace re-export chain (TS2347), preventing the new contextual typing from activating in the real corpus. TS6.0.3 versus tsgo7.0.2 remains the fair reference comparison; benchmark docs/README stay gated until Home is clean and negative controls pass.

14 days ago

Benchmark/parity update: 016f12c is on main. On the unchanged strict 106-file Zod 4.5.2 corpus, Home improved 1,582 -> 1,573 diagnostics by preserving generic namespace-export facts through .js barrels; TS2347 fell 10 -> 1 with no new codes. Two runs were 24.38s and 24.72s and had identical diagnostic position/code sets. The fair references remain TypeScript 6.0.3 and tsgo 7.0.2. README/docs publication remains gated because Home is not yet clean; next blocker is transferring the actual imported generic callable signature so the 937 TS7006 callbacks receive context.

14 days ago

Parity update: 767ce34 is on main. Home now parses readonly(): T as a valid interface method name instead of a readonly modifier plus malformed members. This is a correctness prerequisite for the Zod signature-transfer work; benchmark docs remain gated, and the broader recovered-export change remains unpushed until its newly exposed false diagnostics are fixed.

14 days ago

TypeScript correctness update: 1662fd7 fixes valid any-indexed access without weakening invalid-index diagnostics. The full 4,303-test checker suite passes; the ongoing untouched Zod audit drops from 1,545 to 1,464 diagnostics with this fix on top of recovered export traversal.

14 days ago

TypeScript correctness update: e959b85 fixes optional-parameter widening across Program-transported signatures. Strict controls accept undefined and reject null, and the Zod audit no longer reports the false TS2345 at classic/errors.ts:80 after export recovery.

14 days ago

TypeScript audit update: 497311e recovers healthy exports through star barrels even when an unrelated target has syntax diagnostics, without ignoring recovered name conflicts. Full ReleaseFast passes; untouched Zod runs are deterministic at 1,464 diagnostics, down 109 from the earlier 1,573 baseline.

14 days ago

TypeScript parity update: b3f2d9e removes a composite/object elaboration false positive. The untouched Zod 4.5.2 corpus is deterministically down from 1,464 to 1,449 diagnostics (15 removed, zero new); all 4,304 checker tests and the full ReleaseFast build pass. Fair TS 6.0.3 vs tsgo 7.0.2 benchmark publication remains gated on Home reaching a clean corpus and passing negative controls.

14 days ago

Parity update: 2c9bc2d removes 8 deterministic false TS2345 T-to-T errors from the untouched Zod corpus, taking Home from 1,449 to 1,441 diagnostics with zero new positions. All 4,305 checker tests and the full ReleaseFast build pass. Timing variance was too high to claim a speed change; final benchmark docs remain gated on a clean Home run and fair TS 6.0.3 vs tsgo 7.0.2 measurements.

14 days ago

Parity update: 94b4321 removes the full 10-position $ZodRegistry TS2558 cluster by preserving imported generic default existence even when the default value is outside the transferable subset. Untouched Zod is deterministically down from 1,441 to 1,431 diagnostics with zero new positions; all 4,306 checker tests and the full ReleaseFast build pass. Benchmark publication remains gated on Home becoming silent and fair TS 6.0.3 vs tsgo 7.0.2 runs.

14 days ago

Parity update: 26baa12 fixes contextual out expression parsing. Untouched Zod is deterministically down from 1,431 to 1,394 diagnostics: 39 parser diagnostics removed, 2 downstream definite-assignment diagnostics exposed, net -37. Lexer/parser suites, all 4,306 checker tests, and the full ReleaseFast build pass. Final benchmark publication remains gated on a silent Home run and fair TS 6.0.3 vs tsgo 7.0.2 measurements.

14 days ago

Correctness update from #548: 5edee8012 is on main. It fixes definite-assignment joins after a try whose catch always exits.

The untouched Zod 4.5.2 corpus moves 1,394 → 1,387 diagnostics (seven TS2454 removals, zero additions), with TypeScript 6.0.3 / tsgo 7.0.2 positive and negative controls and a passing full ReleaseFast build. This is still correctness work; no benchmark timing is being published to the README until the corpus is fully admitted.

14 days ago

Correctness update from #548: 3a0f4282b is on main. Nested definite-assignment joins now compose assignments with exiting alternatives, matching TypeScript 6.0.3 and tsgo 7.0.2 while retaining a fallthrough negative control.

Untouched Zod 4.5.2 improves 1,387 → 1,386 diagnostics (one exact TS2454 removal, zero additions), leaving no TS2454s. All 4,308 checker tests and the full ReleaseFast build pass. Benchmark publication remains gated on full corpus admission.

14 days ago

Correctness progress landed in d5735a7fb: override is now treated contextually as a legal interface member name while invalid modifier use still reports TS1070, matching TypeScript 6.0.3 and TypeScript 7.0.2 (tsgo).

Untouched Zod 4.5.2 production corpus: 1,386 -> 1,382 diagnostics, reproduced exactly twice (32.27s and 28.04s), with four removals and zero additions. Parser suite (896/896) and full ReleaseFast build passed. This is diagnostic-correctness evidence only, not a benchmark-table timing result.

14 days ago

Correctness progress landed in 3e821c72c: contextual generic value bindings and async conditional return signatures now match TypeScript 6.0.3 and TypeScript 7.0.2 (tsgo), while genuine bare typeof T still emits TS2693.

Untouched Zod 4.5.2 production corpus: 1,382 -> 1,375 diagnostics, reproduced exactly twice (25.27s and 24.91s), with seven removals and zero additions. Checker suite (4,310/4,310) and full ReleaseFast build passed. These are validation timings, not benchmark-table results.

14 days ago

Correctness progress: shipped 266013349, aligning any | Constructor new-expression behavior with TypeScript 6.0.3 and TypeScript 7.0.2. On the untouched Zod 4.5.2 corpus this removes exactly two duplicate TS2351 diagnostics (1,375 → 1,373), adds none, and is stable across repeated optimized runs (24.73s / 24.38s). Focused checker suite and full ReleaseFast build pass.

14 days ago

Correctness progress: shipped ce58c1bc7, matching TypeScript 6.0.3 / 7.0.2 value-namespace resolution for computed type keys when a runtime value and type alias share a name. Untouched Zod 4.5.2 drops exactly one TS2693 (1,373 → 1,372), adds none, and repeats at 24.51s / 24.38s. Focused checker suite and full ReleaseFast build pass.

14 days ago

Correctness progress: shipped bd2f40ba0, recognizing explicit type arguments on calls to later hoisted generic function declarations, matching TypeScript 6.0.3 / 7.0.2 for TS2347. Untouched Zod 4.5.2 drops exactly one diagnostic (1,372 → 1,371), adds none, and repeats at 25.00s / 25.02s. Focused checker suite and full ReleaseFast build pass.

14 days ago

Correctness progress: shipped 5efb18a03, matching TypeScript 6.0.3 / 7.0.2 assertion-overlap behavior for fully dependent T[K] targets while retaining TS2352 for concrete indexed keys and generic arrays. Untouched Zod 4.5.2 drops exactly one diagnostic (1,371 → 1,370), adds none, and repeats at 24.48s / 25.51s. Focused checker suite and full ReleaseFast build pass.

14 days ago

Compatibility update: feda46e narrows recursive-return TS7023/TS7024 detection to real return-dependent inference cycles, matching TypeScript 6.0.3 and tsgo 7.0.2 controls.

On the accepted Zod 4.5.2 production corpus this moves 1,370 → 1,368 diagnostics with zero additions (exact removals: TS7023 at core/to-json-schema.ts:571, TS7024 at core/api.ts:1802). Repeated ReleaseFast wall times: 24.86s / 24.99s. Checker tests and full ReleaseFast build pass. Commit: feda46e.

14 days ago

Compatibility update: 421d783 faithfully models generic Proxy handlers, Reflect arities/returns, and Object prototype PropertyKey inputs. TypeScript 6.0.3, tsgo 7.0.2, and Home agree exactly on the expanded positive/negative control.

On the accepted Zod 4.5.2 production corpus this moves 1,368 → 1,349 diagnostics: 17 TS7006 and 2 TS2554 removals, zero additions. Stable ReleaseFast corpus runs were 25.34s / 24.20s / 25.03s. Checker suite and a clean full ReleaseFast build pass.

14 days ago

Documentation update: 860a706 adds the requested concise benchmark table to the root README and links it to docs/docs/TS_PERFORMANCE.md.

The table summarizes the existing admitted 30-round snapshots only: TypeScript 6.0.3 vs the single native TypeScript 7.0.2 (tsgo) vs Home, identical strict/noEmit projects, three warmups, all successful finite samples retained. It reports Home with the lowest mean on 20/20 workloads on both Apple M3 Pro macOS ARM64 and Debian Bookworm Linux ARM64, while explicitly preserving the narrowest margins (1.01× and 1.02×) rather than overstating them.

14 days ago

Fresh TS 6.x vs native TS 7.x benchmark evidence is now on main.

  • Harness fix: 50d8f2a88 now makes the default report select only complete, provenance-verified timing directories.
  • Result docs: 0746499d1 publishes raw-result 20260830T220926Z and refreshes the README summary.

Fair full-suite result on macOS ARM64: TS 6.0.3 vs native TS 7.0.2 vs Home 0.1.0, 30 interleaved rounds after 3 warmups. Admission and negative controls passed; compiler/tool provenance stayed unchanged; all 600 round files / 1,800 successful finite samples were retained. Home has the lowest mean on 20/20 workloads and wins 597/600 paired rounds. Nineteen workloads win 30/30; type_predicates_large wins 27/30 and remains the narrowest mean lead at 1.04×.

The detailed table, fairness rules, limitations, raw-result ID, and reproduction steps are in docs/docs/TS_PERFORMANCE.md.

14 days ago

Source-marker profiling optimization and its full confirmation are now on main.

  • Implementation: 7d5208e53 reuses the existing one-pass source marker index for JSDoc and triple-slash reference presence, avoiding redundant whole-file scans without changing exact substring semantics.
  • Results/docs: 8cb78bbdf publishes full raw-result 20260830T223857Z and refreshes the README summary.

Validation: complete ts_checker test artifact passed, focused source-marker tests passed, Zig formatting passed, all 95 benchmark-harness tests passed, and the exact final ReleaseFast binary passed predicate admission/negative controls.

Full macOS ARM64 result (TS 6.0.3 vs native TS 7.0.2 vs Home, 30 interleaved rounds after 3 warmups): Home has the lowest mean on 20/20 workloads and wins 599/600 paired rounds. type_predicates_large improved from 343.6 ± 12.7 ms / 1.04× / 27-of-30 paired wins to 337.5 ± 6.4 ms / 1.07× / 30-of-30. All 600 round files and 1,800 successful finite samples were retained, and compiler/tool provenance remained unchanged.

14 days ago

Follow-up docs consistency fix: f49ab15de updates the benchmark README qualified macOS checkpoint to raw-result 20260830T223857Z.

14 days ago

Published another general TypeScript frontend optimization and refreshed the qualified macOS benchmark.

  • acaa9ad21 (perf(ts): reuse program source markers (#416)) routes exported-namespace interface prechecks through the existing exact per-file marker index instead of rescanning each source four times.
  • Same-host A/B on the frozen type_predicates_large corpus: 338.524 ± 3.226 ms before vs 331.880 ± 3.698 ms after (1.020×, 30/30 paired wins).
  • Full qualified result: 20260830T225711Z, TS 6.0.3 vs the single native TS 7.0.2 (tsgo) vs Home, 30 rounds after 3 warmups, provenance verified before/after, all 600 round files and 1,800 finite samples retained.
  • Home has the lowest mean on 20/20 workloads and wins 600/600 paired rounds. type_predicates_large is now 334.8 ± 9.3 ms vs native TS 7.0.2 at 368.0 ± 11.4 ms (1.10×).
  • dbc9e4852 updates the detailed report and concise README tables.

The benchmark manifest was already updated to compilers.tsgo.version = "7.0.2" in 69d3f9e84; the old 7.0.0-dev.20260707.2 string remains only in explicit rejection tests and historical notes.

Sign in to comment on this issue.