ReviewOS

home-lang/home

bench(ts): add class hierarchy workload

#442
Closed chris opened this 24 days ago · 4 comments
24 days ago

Context

The accepted TypeScript frontend suite now leads across startup, file scaling, deep types, import/re-export graphs, TSX, generic calls, control flow, and overload resolution. It does not isolate declaration-heavy class checking: inheritance, overrides, generic base substitution, constructors, instance member access, and structural assignment.

Proposed fair workload

Add one deterministic, dependency-free TypeScript module containing repeated independent class families with:

  • a generic base class and concrete derived classes;
  • valid property and method overrides with explicit override;
  • constructor/super argument checking;
  • inherited and overridden instance member reads and calls;
  • structural interface assignments that validate the resulting instance shapes; and
  • typed exports that force every family to be checked and consumed.

All three compilers must receive the exact same source and tsconfig, accept it successfully, and produce no output under --noEmit before timing. Compiler versions remain tsc 6.0.3 and native TypeScript 7.0.2.

Acceptance criteria

  • Commit the workload before any optimization and do not change it in response to results.
  • Keep the shape deterministic, auditable, dependency-free, and distinct from existing workloads.
  • Record an interleaved red/green baseline.
  • If Home loses, profile the unchanged workload and make only general compiler improvements.
  • Run focused tests plus the complete affected package suites.
  • Publish a 30-run result in the detailed performance document and README summary.
  • Update #416 with commits and results.

Refs #416

24 days ago

Workload shape cross-validated before timing:

  • one deterministic 4,224-line, 136,478-byte module plus the shared minimal declaration file;
  • 128 independent families, each with a generic base class, concrete derived class, structural interface, valid property/method overrides, constructor/super checks, inherited member calls, and a typed tuple export;
  • generated source SHA-256 bd9c36cf2abd624632d6f9317f89ecf8939e26c2a2eceea92220d6c454e16835, stable across regeneration; and
  • tsc 6.0.3, native TypeScript 7.0.2, and Home accept the identical project silently under --noEmit.

The workload is now frozen; baseline timing follows only after its workload-first commit.

24 days ago

Frozen five-run red baseline (20260826T232221Z):

CompilerMean ± sample SD
tsc 6.0.3210.7 ± 4.7 ms
native TypeScript 7.0.259.6 ± 1.5 ms
Home 0.1.079.4 ± 2.8 ms

Home is 1.33× slower than the fastest competitor. The previous nine workloads remain green. The committed workload and compiler options are frozen; profiling and general compiler work begin from this result.

24 days ago

Validation gate passed on the optimized implementation now in 71833644d:

  • full TypeScript checker suite: 4,190/4,190 passed
  • focused coverage passed for heritage resolution, overrides, protected access, JS prototype replacement, function expandos, local enums, and virtual-section annotation lookup
  • five-run gate (20260826T233556Z): Home 52.9 ± 1.8 ms, TypeScript 7.0.2 57.2 ± 1.1 ms — Home 1.08× faster
  • all ten accepted workloads were green in the gate

Next gate: rebuild from current main, then run the authoritative 30-run / 3-warmup publication benchmark.

24 days ago

Completed on main.

  • frozen workload: b493ea55b
  • optimized implementation: 71833644d
  • published docs: df67841bd
  • full checker suite: 4,190/4,190 passed
  • authoritative 30-run / 3-warmup result: 20260826T234231Z
  • class hierarchy: tsc 6.0.3 194.1 ± 57.8 ms, native TS 7.0.2 53.6 ± 9.8 ms, Home 50.7 ± 15.9 ms — Home 1.06× faster
  • complete accepted suite: 10/10 Home wins

The source and compiler options were unchanged from the recorded red baseline; the win comes from general successful heritage-resolution caching and conservative elimination of impossible source/HIR scans.

Sign in to comment on this issue.