ReviewOS

home-lang/home

perf(ts): improve large type-predicate scaling beyond the current 1.46x lead

#733
Open glennmichael123 opened this 9 days ago ·
9 days ago

Parent: #416. Build/execution capacity: #732.

The current-source focused confirmation 20260911T033015Z admitted and measured type_predicates_large with the pinned schema-3 harness: 30 fresh processes after three warmups, round-robin compiler order, every sample retained, and compiler/tool provenance verified unchanged before admission and after timing.

On Mac14,9 (16 GiB, 10 cores, macOS 26.3.1 arm64):

CompilerMean ± sample SD
TypeScript 6.0.31257.2 ± 104.6 ms
Native TypeScript 7.0.2421.8 ± 20.1 ms
Home 0.1.0288.0 ± 15.3 ms

Home leads the fastest competitor by 1.46×, but this is the narrower of the two current-source focused rows and therefore the next profiling target. The previous 2026-09-05 session reported a 1.60× ratio; cross-session absolute or ratio differences are not a regression proof because host state, OS, and executable shape differ.

Profile the frozen generated workload without changing its source, count, compiler options, admission controls, or timing schedule. Accept only general checker/parser improvements with focused semantic controls and no workload-name/source-shape special case. Require an exact-parent A/B screen, full relevant tests, then an independent 30-round confirmation with all samples and provenance retained. Update TS_PERFORMANCE.md and the README only from verified evidence.

9 days ago

Published build capacity and benchmark documentation to origin/main as glennmichael123 <gtorregosa@gmail.com>, without coauthors:

  • 11a38aa95 adds -Dhome-tsc-strip=true, uses it in the benchmark workflow and documented reproduction, and retains ReleaseFast semantics. The current compiler built with one job at 2,942 MB peak supervised tree footprint, down from the unstripped attempts that crossed 3,584 MB and 3,840 MB ceilings.
  • aa60cdbdf publishes focused result 20260911T033015Z in TS_PERFORMANCE.md and the README.

The focused run admitted unchanged checkjs_jsdoc and type_predicates_large, used 30 fresh processes after three warmups, rotated compiler order, retained all 180 samples, and verified artifact provenance unchanged before admission and after measurement. Mac14,9, 16 GiB, 10 cores, macOS 26.3.1 arm64. Benchmark peak was 452 MB.

WorkloadTypeScript 6.0.3Native TypeScript 7.0.2Home 0.1.0Home lead
checkjs_jsdoc270.0 ± 54.1 ms69.2 ± 3.7 ms39.0 ± 1.4 ms1.77×
type_predicates_large1257.2 ± 104.6 ms421.8 ± 20.1 ms288.0 ± 15.3 ms1.46×

This focused confirmation supplements the prior full 20-workload snapshot. It is not a cross-platform or full-suite claim. #733 tracks the narrower large-predicate scaling lead. #732 remains open because GitHub Actions is disabled and the Linux x64 full-suite runner still cannot be dispatched.

9 days ago

Rejected and fully reverted a target-lowering reuse probe; no production change remains.

The probe reused the asserted target type already lowered by TS2677 compatibility checking when recording explicit predicate metadata. Positive type_predicates_large output remained byte-identical and silent for the baseline and candidate, and both binaries emitted the exact expected 2×TS2322 + 2×TS2339 automatic negative controls.

Fair scaled screen: unchanged generator at 32,768 families (50,252,408 bytes; SHA-256 d924932d25ebc8b6f8e78c89e0da6922ef8436f588d01d1879c5b7c16af0c9af), identical --noEmit -p, alternating process order, all samples retained. The 10-pair screen favored the candidate (1.0398× wall, 1.0228× CPU; 7/10 wins), but the independent 20-pair confirmation did not: wall was 4.9651s baseline vs 4.9895s candidate (0.9951×, 11/20 wins), CPU 4.9299s vs 4.9046s (1.0052×, 12/20 wins). Paired uncertainty crosses zero. Peak supervised tree footprint was 1,005 MB and host low-water 36%.

This does not meet the acceptance bar, so the patch was removed rather than committed. Raw samples are retained in the temporary local A/B workspace for this session.

9 days ago

Two additional general optimization probes were measured against the same preserved baseline and fully reverted; no production change remains.

  1. Replacing per-predicate intern("this") with exact canonical-byte comparison preserved the positive project and exact 2×TS2322 + 2×TS2339 controls, but the 10-pair 32,768-family screen was slower: wall 4.8786s baseline vs 4.9450s candidate (0.9866×; 6/10 wins), CPU 4.8397s vs 4.9031s (0.9871×; 6/10).
  2. Counting explicit predicates in the declaration batch and reserving fn_predicates / signature_predicates capacity also preserved all controls, but was flat/slower: wall 4.7156s vs 4.7396s (0.9949×; 5/10), CPU 4.6555s vs 4.6595s (0.9991×; 3/10).

Both use the unchanged 50,252,408-byte, 32,768-family source (SHA-256 d924932d25ebc8b6f8e78c89e0da6922ef8436f588d01d1879c5b7c16af0c9af), alternating order, all samples retained. The reservation run peaked at 998 MB supervised footprint with host low-water 48%. Neither meets the acceptance bar, so both patches are absent from main.

9 days ago

Two additional exact-parent probes were tested and rejected; the tracked tree was restored after each.

  1. Use the lexer's authoritative escape flag in Parser.internToken

    • Candidate: replace a second byte scan for \\ with tok.flags.has_escape.
    • Semantic escape corpus: clean under parent and candidate.
    • Frozen diagnostic corpus: 32,768 predicate families, 50,252,408 bytes, SHA-256 d924932d25ebc8b6f8e78c89e0da6922ef8436f588d01d1879c5b7c16af0c9af.
    • 20 alternating AB/BA fresh-process pairs after 3 warmups each:
      • parent mean 5.528101 s, median 5.224933 s
      • candidate mean 5.275753 s, median 5.151260 s
      • paired delta mean -1.359%, median +0.164%, 10/20 wins, 15.752 pp SD
    • Verdict: inconclusive/flat; reverted.
  2. Gate import-binding probes with the source-level import marker

    • Candidate: early-return from localNameIsImportBinding and localNameIsNamespaceImportBinding when an immutable source marker proves the file contains no import declaration.
    • Focused checker unit test passed; the frozen predicate corpus and import-heavy benchmark both remained clean.
    • 20 alternating AB/BA fresh-process pairs after 3 warmups each:
      • parent mean 5.272022 s, median 4.927757 s
      • candidate mean 5.225073 s, median 4.910759 s
      • paired delta mean +0.041%, median +0.305%, 10/20 wins, 12.438 pp SD
    • Verdict: flat; reverted.

Both runs stayed under the fixed 3,840 MB hard ceiling (observed peaks about 1.0 GB). No benchmark or source change is being claimed from these results.

9 days ago

The current-source profile independently rediscovered the previously rejected typeOfIdentifier declaration-slot reuse probe, so I reran it at the larger frozen scale rather than assuming the old result no longer applied.

  • Exact parent: b164b747e; parent binary SHA-256 bfba4f9e94731d4e9639094e53fea1e707fd1c4c160157474ee2de86ba1275b3
  • Candidate binary SHA-256: 94e3c313d86c8c9d68b63523d3ffd068f066510454354a6840690caab8dfc9be
  • Generated-code check: total isDeclNameSlot call relocations fell from 49 to 13, eliminating the expected 36 calls from typeOfIdentifier.
  • Frozen workload: 32,768 predicate families, 50,252,408 bytes, SHA-256 d924932d25ebc8b6f8e78c89e0da6922ef8436f588d01d1879c5b7c16af0c9af; both binaries exited zero silently.
  • 20 alternating AB/BA fresh-process pairs after three warmups each, with every sample retained:
    • parent: 6.207927 ± 1.853128 s (median 5.525843 s)
    • candidate: 5.625798 ± 0.978428 s (median 5.265798 s)
    • 15/20 candidate wins; paired mean improvement 0.582129 s
    • paired 95% CI: -0.055687 to +1.219945 s
  • Guard: 996 MB observed peak, 35% host low-water, fixed 3,840 MB ceiling.

The interval still crosses zero. Combined with the existing official 2,048-family rejection (0.973× wall, 4/10 wins), this does not clear the documented admission rule. The candidate was fully reverted and no speedup is claimed. README and the detailed performance document now record this scale retest so it is not repeated.

Documentation commit b55516fc9 is pushed to origin/main under the configured repository identity, with no coauthors.

9 days ago

Fresh current-source confirmation completed; no production or benchmark-source change was made.\n\nThe first run in this session is explicitly invalid for publication: its metadata identified Home SHA-256 94e3c313d86c8c9d68b63523d3ffd068f066510454354a6840690caab8dfc9be, the previously rejected declaration-slot candidate binary whose source had already been reverted.\n\nI rebuilt the documented stripped ReleaseFast Home compiler from current main under the fixed guard:\n- command: zig build home-tsc -Doptimize=ReleaseFast -Dhome-tsc-strip=true -j1\n- resulting Home SHA-256: bfba4f9e94731d4e9639094e53fea1e707fd1c4c160157474ee2de86ba1275b3\n- build peak: 2,854 MB; fixed ceiling 3,840 MB\n\nThen reran the unchanged canonical type_predicates_large benchmark:\n- result id: 20260912T141609Z\n- 30 fresh processes after 3 warmups, round-robin interleaved\n- schema 3 admission passed\n- compiler/tool provenance verified unchanged before and after\n- TypeScript 6.0.3: 1194.9 +/- 14.6 ms\n- Native TypeScript 7.0.2: 417.8 +/- 5.7 ms\n- Home 0.1.0: 284.9 +/- 4.7 ms\n- Home lead over fastest competitor: 1.47x\n- benchmark peak: 469 MB; host low-water 57%\n\nThis independently confirms the currently documented 1.46x result. I am not replacing README/TS_PERFORMANCE.md for a 0.01x cross-session fluctuation, and I am not claiming the invalid stale-candidate run. Raw ignored results remain at bench/vs_tsgo/results/20260912T141609Z.

Sign in to comment on this issue.