ReviewOS

home-lang/home

bench(ts): add type-predicate narrowing workload

#450
Closed chris opened this 8 days ago ·
8 days ago

Goal

Add a deterministic frontend workload for user-defined type predicates and assertion functions, then optimize Home only through general compiler improvements until it beats both pinned competitors.

Fair workload

Generate one dependency-free TypeScript module containing independent type-predicate families. Each family will exercise:

  • a discriminated input union;
  • a value is T predicate;
  • an asserts value is T assertion function;
  • call-driven narrowing and nested property reads;
  • typed result consumption.

All three compilers must receive the identical generated project and shared --noEmit configuration. The project must validate successfully and silently in Home, JavaScript TypeScript 6.0.3, and native TypeScript 7.0.2 before timing. A preserved invalid mutation must also be rejected consistently before the workload is accepted.

Process

  1. Land and freeze the valid workload before performance tuning.
  2. Record an interleaved red baseline if Home loses.
  3. Profile and fix general compiler behavior without corpus-specific shortcuts.
  4. Run the full interleaved suite with all samples retained.
  5. Publish detailed methodology/results and synchronize the root README table.

Tracks #416.

8 days ago

The validity-gated workload is frozen on main in 9fab1ff. Generated source: 13,312 lines, 369,048 bytes, SHA-256 8b04ff5b9ae889c940facb652e5c925c7ff2aa6350058292208ca29020192350. Home, tsc 6.0.3, and native TS 7.0.2 accept the exact project successfully and silently; all three reject the preserved invalid copy with TS2322 and the same string-to-number mismatch. Initial interleaved five-run baseline after three warmups (20260827T133314Z): Home 132.3 ± 1.8 ms, native TS 7.0.2 79.8 ± 1.4 ms, tsc 6.0.3 277.6 ± 13.5 ms. Home is 1.66× slower than the fastest competitor, so the corpus remains unchanged while the general bottleneck is profiled.

8 days ago

Completed on main in three small commits:

  • 9fab1ff adds the deterministic 256-family type-predicate/assertion workload.
  • 12cbc5b adds general checker indexes, token-aware source facts, binder-backed lookups, and successful named-shape caches.
  • c185044 publishes the README and detailed benchmark record.

Validity evidence:

  • Frozen source: 13,312 lines, 369,048 bytes, SHA-256 8b04ff5b9ae889c940facb652e5c925c7ff2aa6350058292208ca29020192350.
  • Home, tsc 6.0.3, and native TypeScript 7.0.2 accept the valid project silently.
  • All three reject the preserved string-to-number variant with TS2322.
  • Full ts_checker suite passes (4,191/4,191).

Red baseline 20260827T133314Z: Home 132.3 ± 1.8 ms vs native TS 7.0.2 79.8 ± 1.4 ms. Authoritative 30-run snapshot 20260827T151143Z: Home 69.0 ± 4.9 ms vs native TS 7.0.2 69.6 ± 5.5 ms and tsc 6.0.3 229.6 ± 12.3 ms, a 1.01× win over the fastest competitor. All 15 suite workloads are Home wins.

Sign in to comment on this issue.