ReviewOS

home-lang/home

perf(ts): remove exported-declaration query scaling regression

#537
Closed chris opened this 8 days ago ·
8 days ago

Tracker #416; follows source-owned factory typing #534 (de8fe28). Complete fair checkpoint 20260828T224220Z passes all 19 admission gates and validates all 570 round files / 1710 successful samples. Home has lower means on 18/19 rows, but the unchanged 2048-family type_predicates_large workload is 541.3 ± 50.7 ms versus native TS7.0.2 380.6 ± 44.8 ms: 1.42x slower, with Home losing all 30 paired rounds. TS6.0.3 is 1071.8 ± 87.3 ms. Prior frozen Home 3b7f0ea had lower historical timings, but a same-host interleaved before/after test is required to attribute the difference.

Profile the source-owned exported-declaration collection/query work and checker costs, prove the cause, then remove repeated work using bound ownership/indexes or demand-driven semantics. No benchmark-specific shortcuts, weakening type checks or removing failing rows. Both old and new binaries must first pass this workload and its unchanged predicate rejection controls before their performance is compared. Preserve the 240 factory controls, both original graph gates, broader generic/owner/static audits and full regressions. Retain measurements and publish the losing checkpoint and subsequent results in docs/README.

8 days ago

Controlled same-host comparison confirms the regression: frozen3b7f0ea93 350.96±16.88ms versus de8fe28 533.76±14.86ms, 1.5208x slowdown. Both binaries pass this same workload and its unchanged predicate rejection controls before timing. Three warmups then30 alternating before/after rounds retain all60 finite successful samples; after loses0/30 wins. Evidence graph-types.4EsbvV/predicate-ab-v3/{metadata.json,round-*.json} and predicate-ab-v3.log. This is separate from, not averaged into, the full3-compiler table. Source inspection: declaration_graph.collect calls Query.resolve for every export; export_origins.Query.exported scans the whole root block and checks declaration ancestry per exported declaration, and local also scans all imports/root statements. The large workload has2048 exported variables, making repeated whole-block traversal a concrete quadratic-cost candidate. A late Debug sample landed in ordinary checking and does not itself attribute the regression; profile earlier collection or instrument visit counts, then implement owner-bound indexing and rerun identical controls/measurements. No cause claim from the historical timing comparison alone.

8 days ago

Deterministic operation-count regression added: 128 exports in a256-statement source require65,536 root-statement visits in the existing Query (expected one256-statement traversal); Program baseline133/134 passes with only this new scaling test failing. Evidence export-index.NVcIdu/program-before.log. The index implementation is now under test: one owner-bound import/export edge index per prepared Compilation, with lexical edge ordering, explicit-export precedence, per-query type-only visibility and cycle state preserved. It caches syntax/binding edges, not potentially context-dependent semantic answers.

8 days ago

Implementation b7b9dd9 is pushed, with separate parser ownership fix d6878e8 (#538). All19 workload admission gates and existing audits retain their scores. Checker4286, conformance1417, Program137, parser891, driver187, CLI69, emitter499 and harness72 tests pass. Controlled same-correctness large-predicate comparison: de8fe28 525.56±12.44ms versus b7b9dd9 353.54±10.53ms, after/before0.67269 (32.7% less time),30/30 paired wins. All30 alternating rounds/60 successful finite samples retained; three warmups per binary; both pass the unchanged positive and predicate rejection gates. Evidence export-index.NVcIdu/predicate-ab-v3/ and predicate-ab-v3.log. Deterministic root visits fall65,536→256 for128 independent exports. The complete19-workload TS6/TS7 run is now in progress; no new all-workload speed claim yet. New export-list audit improves0→80/96 and retains16 diagnostic-code mismatches under #540.

8 days ago

Implementation b7b9dd9 (#537) and parser ownership fix d6878e8 (#538), with verified README/report 24a34f4, are pushed to main. Compiler versions are verified TS 6.0.3 and native TS 7.0.2; TS 7/tsgo is one competitor.

All 19/19 unchanged workload admission gates pass. Full measurement 20260828T231439Z validates 570 rounds / 1,710 successful finite samples; fresh generation matches 513 measured input files. Home has lower means on 19/19 workloads, but large predicates have only a noisy 1.02x lead: 371.6 ± 125.5 ms versus TS 7 377.7 ± 66.4 ms, 25/30 paired wins. Independent 30-round confirmation 20260828T231856Z retains all 90 samples: Home 320.4 ± 2.8 ms versus TS 7 331.4 ± 4.6 ms (1.03x; 29/30 paired wins). Confirmation is separate, not substituted into the full table.

The same-correctness alternating before/after comparison for #537 improves 525.6 ± 12.4 ms to 353.5 ± 10.5 ms, 32.7% less time and 30/30 paired wins. Deterministic source visits drop 65,536 to 256 for 128 independent exports. Bound-owner edge indexing preserves semantic query work; no control or failed observation was removed.

New exported-variable-list audit improves 0/96 to 80/96. All 32 positives, 32 wrong-type controls and 16 barrel hidden-name controls pass; 16 direct-hidden diagnostics remain TS2305 instead of TS2459 (#540). Existing audit scores remain: factory 240/240, recursive 288/288, generic classes 120/120, static 84/84, bound classes 52/52, callable identity 56/56, callable unions 256/256, graph discovery 28/28, export origins 32/32, owners 12/20, nominal origins 44/52, globals 32/56, bound globals 44/56. Both competitors pass every audit.

Verified suites: checker 4286, conformance 1417, Program 137, parser 891, driver 187, CLI 69, emitter 499 and benchmark harness 72 tests all pass. Pickier reports no errors and one pre-existing README fragment warning. Frozen evidence export-index.NVcIdu, SHA-256 615e9fa1f78c2cad17a0b8911b897ede73c00422138d6ecbb2b74801d4564f14.

Next: #536 removes redundant CommonJS export-query compilation; #540 adds external owner-local diagnostic facts. Correctness gaps, expanded rejection coverage, real projects and cross-platform measurements keep the overall goal open. Shared-host synthetic results are not universal benchmark leadership. Full results and reproduction.

Sign in to comment on this issue.