ReviewOS

home-lang/home

bench(ts): add admitted checked CommonJS graph workload

#546
Closed chris opened this 9 days ago ·
9 days ago

Parent: #416.

Now that #541 passes all 66 unchanged CommonJS linkage controls, add one logical timed workload that exercises real cross-file checked-JavaScript CommonJS consumption. Use identical explicit JS roots and strict allowJs/checkJs settings for TypeScript 6.0.3, native TypeScript/tsgo 7.0.2, and Home. Add unchanged positive validation plus wrong-type and missing-member admission controls before timing. Include reassignment/inferred-field shapes representative of the audited semantics without constructing a benchmark-specific shortcut. Retain any loss, document complete results and update the README table/link. Do not admit timing unless every compiler passes every control.

9 days ago

Completed and pushed to origin/main in small commits:

  • 31cba27 bench(ts): add checked CommonJS graph workload (#546)
  • 3e1d2bc perf(ts): index CommonJS graph lookups (#546)
  • bf64a67 docs(ts): record admitted CommonJS graph win (#546)

Fairness: TS 6.0.3 and native TS/tsgo 7.0.2 are the two pinned competitors. The 128-owner checked-JavaScript graph requires identical positive admission plus exactly 3 TS2322 and 3 TS2339 controls in every compiler before timing.

The initial admitted loss is retained: Home 62.8 ± 1.2 ms vs tsgo 51.1 ± 2.1 ms (1.23x slower). The exact correct-build A/B improved Home from 62.33 ± 1.49 ms to 42.96 ± 0.92 ms, 31.1% less time with 30/30 paired wins. Final full run 20260829T013535Z: CommonJS graph TS6 148.9 ± 2.5 ms, tsgo 48.3 ± 1.4 ms, Home 42.1 ± 0.7 ms (1.15x faster, 30/30 paired wins). Home has lower means on all 20/20 admitted workloads; all 600 round files and 1,800 successful finite samples are retained. The type_predicates_large margin is explicitly narrow at 1.01x and 24/30 paired wins.

Verification: CommonJS audit 198/198; static discovery 132/132; harness 82/82; Program 145/145; checker, driver, CLI green; conformance 16/16 smoke, 86/86 named, 586/586 baseline-aware. README and detailed docs contain the complete table, controls, hashes, evidence locations, and reproduction commands.

Broader real-project, cross-platform, and semantic coverage remains tracked by #416.

Sign in to comment on this issue.