ReviewOS

home-lang/home

bench(ts): add interface composition workload

#444
Closed chris opened this 24 days ago · 3 comments
24 days ago

Context

The accepted TypeScript frontend suite now leads across eleven workloads, including structural plain-object compatibility and class inheritance. It does not isolate declaration composition: generic interface extends chains, repeated interface declarations merged into one symbol, namespace/type merging, inherited member aggregation, and structural consumption of the composed result.

Proposed fair workload

Add one deterministic, dependency-free TypeScript module containing repeated independent interface families with:

  • generic base interfaces and multi-base extends;
  • repeated declarations of the same interface that contribute distinct compatible members;
  • an interface merged with a namespace containing a related type and value helper;
  • concrete object values checked against the fully composed interface;
  • inherited and merged members consumed through typed functions; and
  • typed exports retaining every family.

All three compilers must receive the exact same source and tsconfig, accept it successfully, and produce no output under --noEmit before timing. Compiler versions remain JavaScript tsc 6.0.3 and native TypeScript 7.0.2.

Acceptance criteria

  • Commit the workload before any optimization and do not change it in response to results.
  • Keep the shape deterministic, auditable, dependency-free, and distinct from existing workloads.
  • Record source size/hash and an interleaved red/green baseline.
  • If Home loses, profile the unchanged workload and make only general compiler improvements.
  • Run focused tests plus the complete affected package suites.
  • Publish a 30-run result in the detailed performance document and README summary.
  • Update #416 with commits and results.

Refs #416

24 days ago

Pre-optimization workload shape is fixed:

  • one deterministic module
  • 128 independent generic interface families
  • 5,760 lines, 194,012 bytes
  • SHA-256 84123a04d7d9e3adeb8e1089f068be7524890cbd6872466b83ba2bced0c322be
  • two generic base interfaces, multi-base extends, repeated compatible interface declarations, interface/namespace merging, a generic namespace helper, concrete structural values, and typed export consumption
  • tsc 6.0.3, native TS 7.0.2, and Home 0.1.0 all accept the exact generated project silently
  • hash is stable across regeneration

The workload will be committed before the first timed baseline and will remain unchanged afterward.

24 days ago

Frozen pre-optimization baseline recorded from 788192d95.

Result ID: 20260827T000848Z (5 interleaved runs after 1 warmup)

  • tsc 6.0.3: 208.5 ± 8.1 ms
  • native TS 7.0.2: 65.3 ± 0.8 ms
  • Home 0.1.0: 117.3 ± 3.0 ms
  • outcome: Home is 1.80× slower than the fastest competitor
  • the previously accepted eleven workloads remain Home wins

The committed source and configuration are frozen. Next step is profiling and general binder/checker optimization against this unchanged workload.

24 days ago

Interface-composition benchmark completed and published.

  • frozen workload: 788192d
  • source: 5,760 lines, 194,012 bytes, SHA-256 84123a04d7d9e3adeb8e1089f068be7524890cbd6872466b83ba2bced0c322be
  • named-type reverse index: 026ab0e
  • interface/namespace/source lookup indexes: ca0b8df
  • README and detailed report: d5076fa
  • full checker validation: 4,190/4,190 passed
  • red five-run baseline 20260827T000848Z: Home 117.3 ± 3.0 ms, native TS 7.0.2 65.3 ± 0.8 ms
  • authoritative 30-run result 20260827T014124Z: Home 74.6 ± 3.0 ms, native TS 7.0.2 82.0 ± 2.3 ms, tsc 6.0.3 294.9 ± 33.5 ms
  • outcome: Home is 1.10× faster than the fastest competitor and 1.57× faster than its frozen baseline; the accepted suite is now 12/12 Home wins

The generated source and compiler options remained unchanged after the workload commit. Every compiler validated every workload successfully and silently before timing; compiler order rotated within every measured round.

Sign in to comment on this issue.