ReviewOS

home-lang/home

perf(ts): reduce measured declaration-space and whole-file JSDoc scanning

#853
Closed chris opened this 8 days ago ·
8 days ago

Current evidence

Follow-up to #416/#852: complete protocol4 matrix20261008T213919Z passes145 admissions/600 rounds/3000 samples on the byte-pinned accepted c16c8ca compiler. Home still leads1/20 and checkedJS/JSDoc trails the fastest competitor by2.14x.

A diagnostic-only native sample of the unchanged checkedJS generator at8192 families passes58 official image admissions, preserves identical original/renamed instruction-section bytes and stable input/source/binary fingerprints, and never supplies timing samples. The one-second stack sample has428 leaf observations in Checker.checkSourceFile,348 in checkDeclarationSpaceDiagnosticsImpl,18 in jsdoc.parse and5 in string interning. These are exploratory leaf counts, not whole-run CPU percentages or causal speedup evidence.

Scope

Inspect the actual loops within declaration-space checks and the sampled late checkSourceFile work, map sampled code offsets where needed, and replace repeated scans with complete syntax/declaration facts under natural source/pass ownership. Select the optimization from evidence, not library/benchmark filenames. Preserve all duplicate/merge/namespace/virtual-section/ordering/source-location and JSDoc diagnostic behavior, and full allocation-failure fallback/rollback. No numeric semantic limits, diagnostic suppression, approximate types, body omission or alignment tweaks.

Acceptance

  • Proven source-level cause tied to current profile; retain the exploratory raw samples and code/input/image fingerprints.
  • Exact scanner/oracle parity across representative syntax and source changes, plus complete OOM controls.
  • Full checker/Program/driver/CLI gates, all protocol4 checkedJS negative controls and pinned Zod diagnostics.
  • Frozen exact-parent official checkedJS and unchanged-generator scales, unfiltered rotating-order paired confirmation; all20 families and independent adverse-risk rechecks.
  • Publish accepted or rejected outcomes with checksum-covered evidence, detailed docs and a brief README table in small issue-linked Chris commits without coauthors.
8 days ago

Current profile evidence is published in6004a2634 on origin/main. Report: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#untimed-admission-and-diagnostic-profile

Accepted native c16c8caaf52d35180a7fcd53b87cac0dd71ff7b86526ef285e09bb4e088335f4, exact cached object bd955e66b15f59c4b457819f9f6be965bb005e27b8901bdc1d3092b056b2442c. Original and renamed instruction bytes match at SHA1c9d4cd471ae2c71e2e73dff7a4deb0205bb81ef02f4d921fb48976a8da505bb. Both images pass all58 revised official admissions. Scale8192 generated by unchanged run.generate_checkjs_jsdoc exits0 silently, input digestbfb2fb1bef4dcb4ad7df9bffa647b5d44bd35c6895b32772fb4044ce0ced829e stable before/after. Diagnostic image never timed; one-second leaf counts428 checkSourceFile,348 checkDeclarationSpaceDiagnosticsImpl,18 jsdoc.parse,5 interning are exploratory, not whole-run percentages. Raw samples, exact scale, both instruction sections, runner/source/input fingerprints and guard log are archived at SHA db598c898ebbd9ef69ad3b3e5a043049780086ac51e237c150cb9c812aa9d5c0,50 verified checksums. Next: map the actual sampled loop offsets and establish the source-level repeated work before implementing. No optimization, speed gain or source-level cause claimed yet.

8 days ago

Mapped the accepted diagnostic symbol start0x1007b41bc + sampled offset12656 to instruction0x1007b732c. Its loop advances the full statement ordinal, unwraps export kind13, filters interface kind7, then compares virtual section and declaration name: this is the inlined inner loop of checkClassInterfaceMemberModifierMerges. Source confirms one complete statement scan for each class, even with no interfaces. The local draft builds a complete ordered interface bucket index once per statement container, then visits only same-name/same-section interfaces; no persistent Checker field, filename handling, cap, skipped diagnostic rule or approximation. Any construction OOM releases partial groups and falls back to the original scanner. Complete checker testing is compiling, with ordering/scope parity, construction OOM cleanup, fallback and no-interface/no-allocation controls. The late checkSourceFile sampled loop is captured separately and not claimed fixed. No performance claim or compiler push yet.

8 days ago

Mapped cause and draft status: exact sampled declaration offset12656 is the inlined full-statement inner loop of checkClassInterfaceMemberModifierMerges (export-kind13 -> interface-kind7 -> section/name checks). Complete ordered interface buckets replace that repeated scan while retaining the original scanner on construction OOM and no persistent Checker fields. Draft supported native build passes (peak2951 MB), and corrected full checker4531/4531 passes (peak3113 MB). Initial full run passed4530 and failed one OOM control because its hardcoded range reached diagnostic allocation rather than construction; that failure/patch/log is retained. The control now measures and fails every construction allocation; no tests removed or filtered. Ordering/scope parity, partial cleanup, actual OOM fallback and complete no-interface absence/no-allocation controls pass. Exact parent6004a2634/nativec16c8caa, frozen candidate9e9cc2a1d108640694c066b7afb36e2b286a73a03d92271d30b3fa657c54f8ea, patchc83cf3f0e2fa3fcf80c9be499e43f94c6786e0bb9077b98c1ad1a9e82ab51422. Program/driver/CLI/entrypoint and pinned Zod gates are currently running/queued behind the shared lock. No compiler push or timing samples yet. Predeclared A/B will require66 official/scale admissions per phase, official checkedJS20/30 pairs, unchanged generator512/8192 family scales6/30 pairs, all20 family screens10 pairs, independent30-pair >3%-slower/negative-interval rechecks plus carried-forward large-predicate and reexport risks. Late checkSourceFile work is not claimed fixed.

8 days ago

All final draft correctness gates pass before timing: checker4531, Program247, driver200, CLI74, entrypoint35 and supported stripped native build. Parent/candidate full pinned Zod messages/positions are identical195TS+3HM9002 (normalized6bf2feb5...), not production admission. Native candidate9e9cc2a1... and patchc83cf3f0... are frozen against parent6004a2634/c16c8ca. Maximum current test footprint3113 MB, build2951 MB, unchanged3840 guard/disk floors. Official checkedJS/512-/8192-family A/B screen is now running and requires66 admission records (58 revised official +8 actual first/middle/last scale negative/positive records) before all unfiltered timing. Class-interface index is complete and ordered; no persistent checker layout change, cap, benchmark recognition or skipped merge diagnostic. No speedup or source push claimed until independent confirmation and complete family/recheck phases.

8 days ago

Initial screen completed with66 successful admission records, unchanged fingerprints and all observations retained. OfficialJS31.074->31.055 ms, saving CI[-0.414,+0.342]ms; scale512 161.638->159.119 ms,5/6,[-0.195,+7.932]ms; scale8192 20893.870->20300.335 ms,4/6,[-2184.677,+1717.159]ms: all inconclusive, not speedups/equivalence. Large8192 checks take~20s per fresh process. Continuous30-pair confirmation would exceed the fixed600s execution guard, so before confirmation I declared five separate6-pair batches totaling all30 pairs, each retaining all66 admissions/3 warmups/order/fingerprints. No guard increase, omitted observations, smaller substitute workload, best-of batch or removal of the initial screen. Official/512 confirmation and all20-family screen are running/queued; first8192 batch queued. Original runner revision is preserved; batch metadata will remain explicit rather than relabeled as one continuous run. No source push or performance acceptance yet.

8 days ago

Independent official/512 confirmation and complete20-family screen finished, all66 admissions/phase and fingerprints unchanged. OfficialJS29.197->29.176 ms,16/30, saving CI[-0.086,+0.566]ms is inconclusive. Scale512 156.808->155.196 ms,22/30, CI[+0.268,+2.385]ms is a measured small scale gain. No >3%-slower-median or wholly negative20-family interval selects extra families; carried-forward predicate/reexport30-pair rechecks are now queued/running. First explicitly separate8192 confirmation batch (6 of required30) is18.267->18.952s,1/6,CI[-1.061,+0.251]s, adverse median but inconclusive. It is retained, not replaced. Batch1 of remaining4 is queued. Full30 large observations across all5 declared batches are still required before any acceptance, aggregation will retain every batch and its warmups/admissions/order/fingerprints. Source remains local/uncommitted; no production push, universal speedup, equivalence, or new matrix claim.

8 days ago

Both carried-forward independent30-pair regression checks finish with unchanged66-admission gates/fingerprints and intervals spanning zero: large predicates244.421->243.934ms, CI[-2.647,+0.748]ms; reexports11.802->11.680ms, CI[-0.539,+0.966]ms. No equivalence claim. Separate late checkSourceFile hotspot is now mapped precisely: instruction kind8/0x34 are class_decl/class_expr (not functions), and the strict comment containment/minimum-span loop matches reportClassLocalTypedefTemplateReferences called once per typed typedef tag by checkJSDocTypedefTypeTags. Assembly/raw original source and enum mapping are retained. This whole-HIR scan per typedef remains untouched by the frozen interface-index candidate; later optimization will get separate provenance and complete equal-width/boundary/virtual-section/OOM controls. Remaining large batches still required before accepting the present draft.

8 days ago

Large confirmation batch1 (second of the five declared batches) completed:18.529->18.091s median,6/6 candidate faster, paired saving CI[+0.278,+0.575]s. The preceding batch0 remains adverse/inconclusive and retained. Batch2 is running and batch3 queued; batch4 will follow. The verifier independently recomputes raw rotating-order records, sample arrays, medians, intervals, exact negative diagnostic codes, all66 admissions per phase, unchanged input/binary/harness/patch fingerprints and matching original/current runner revisions. A supplemental five-batch interval will show variation across separate warmup sets, in addition to the pooled30 paired differences; no phase is dropped. Full benchmark-harness136-test gate is queued. No acceptance/source push/new ranking yet.

8 days ago

Source review identifies a natural regression risk that the original >3%-median screen threshold does not select: eagerly building interface groups allocates even when a container has no eligible class. The full interface-composition screen is33.553->33.746ms (4/10), CI[-0.392,+0.009]ms, small and inconclusive; class hierarchy23.214->23.071ms (7/10), CI[-0.009,+0.344]ms also inconclusive. Before acceptance I am adding independent30-pair confirmation of BOTH declaration families, same66 admission gates/three warmups/alternating order/frozen binaries and patch. This supplements rather than replaces the existing screen/confirmation/carried-risk checks or any of the five large batches. No source changes during measurement. A source-derived lazy first-class build, if warranted, will be a separately frozen candidate with all current evidence preserved.

8 days ago

Independent declaration-risk checks completed with all 66 admissions and unchanged fingerprints. Class hierarchy: 23.198 -> 22.970 ms, 22/30 candidate faster, paired mean-saving CI [+0.074, +0.447] ms (confirmed small gain). Interface composition: 33.957 -> 33.823 ms, 20/30, CI [-0.087, +0.522] ms (inconclusive, not equivalence). Its initial adverse 10-pair screen remains retained. Full benchmark harness passes 136/136. Large batch3 completed 18.382 -> 18.063 s, 6/6, CI [+0.205, +0.523] s; final batch4 is now running. All five batches, including adverse batch0, remain required. Only that final batch and final audit/publication remain for this frozen candidate; no source push or new matrix claim yet.

8 days ago

Landed the scoped class/interface modifier-merge index in bbabc2f and published all evidence/docs/README tables in b513c85. Both commits are sole-authored/committed by Chris, without coauthor trailers, pushed directly to origin/main. Complete audit: 660 admissions, 442 paired rounds / 884 timing samples, 1,563 verified archive checksums. Confirmed small gains: 512-family JSDoc 156.808 -> 155.196 ms, CI [+0.268,+2.385] ms; class hierarchy 23.198 -> 22.970 ms, CI [+0.074,+0.447] ms. Official checked JS is inconclusive. All five large batches, including adverse first batch, are retained: combined 18.407 -> 18.069 s, 25/30; pooled saving CI [+0.085,+0.425] s, but supplemental five-batch CI [-0.212,+0.721] s spans zero. Independent interface/predicate/reexport risk checks are inconclusive, not equivalence. Full checker/Program/driver/CLI/entrypoint pass 4531/247/200/74/35; harness136/136; scoped Pickier/Zig formatting/diff checks pass. Zod remains unchanged at 195 TS + 3 HM9002, not production admission. Report: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#classinterface-modifier-merge-index-2026-10-08-utc . Issue stays open: mapped per-typedef whole-HIR class-owner scan remains untouched; no fresh competitor matrix, universal leadership or other-platform claim.

8 days ago

Beginning the separate class-local typedef owner optimization from exact landed parent b513c85 (native 9e9cc2a). This addresses the mapped shortest strictly containing class scan, not the already-landed class/interface merge index. Plan: lazily build one complete class-span index per typedef pass; preserve strict boundaries, smallest spans and lowest NodeId ties; use the complete scalar scan after allocation failure or crossing/malformed nesting, freeing all partial data. No persistent TypeIds/source-name conditions or semantic limits. Differential full-pass diagnostic tests, all allocation failures and nested/export/class-expression/virtual-section/range controls precede frozen exact-parent timing. Prior #853 evidence remains immutable and separate; no speed acceptance yet.

8 days ago

The class-owner draft is now in full checker/build validation, not timing acceptance. Five controls cover every source byte, 93,750 synthetic strict-containment queries (all ordered three-range assignments on five coordinates), equal-range NodeId ties, empty/reversed/crossing ranges, all actual index allocation failures and complete indexed/forced-slow/failed-index typedef-pass diagnostics across exports, class expressions, nested classes and virtual sections. The scalar reference uses actual owner presence instead of treating the widest u32 span as an unset-width sentinel; a widest-representable-range control verifies this logical boundary correction on both paths. The initial full diff is frozen separately from the prior landed candidate; no semantic cap or sample omission is introduced.

8 days ago

Initial full checker: 4,535 passed / one new allocation-control failure; original diff and complete failed log/command are retained. The control depended on allocator growth taking new allocations, but successful in-place remaps can change that path. Corrected the TEST allocator to refuse remaps deterministically, retaining the >1 growth-allocation assertion and failing every actual growth allocation, checking all byte-position owners, complete scalar fallback, zero retained capacity and balanced allocated/freed bytes. Production code is unchanged by this correction. The corrected full checker is queued; Program 247/247 passes, with native/driver/CLI/entrypoint validation still running/queued. No timing acceptance or source push.

8 days ago

Native build passed (peak 2,919 MB). Driver, entrypoint, corrected checker and harness then failed closed before execution: disk admission reported 565–566 MB free, below the unchanged 1,024 MB floor. Their complete refusal logs/commands are retained separately, not relabeled as semantic failures or discarded. After confirming terminal jobs and no open handles, removed only 50 explicitly inventoried old compiler/unit-test cache directories in this managed task worktree (100 rebuildable output files, 1,297,717,280 bytes). Source, logs, evidence, benchmark binaries and diagnostic profile objects remain; removal manifest is retained and outputs are reproducible from source/toolchain pins. Disk returned above 2 GB. Retrying the refused gates under identical limits; pinned parent/candidate Zod checks are queued. No source push or speed acceptance.

8 days ago

Final correctness validation is complete for the frozen class-typedef-owner candidate: checker 4,536; Program 247; driver 200; CLI 74; entrypoint 35; harness 136 all pass. Native candidate SHA-256 fb77a18ed4b9b14a7c0ef29366963166754de8904602c2516ea2d223389cbb82, exact accepted parent 9e9cc2a at source b513c85. Complete pinned Zod primary message/position/code multiset remains 195 TS + 3 HM9002 (zero added/removed), not production admission. Declaring timing BEFORE execution: original official/512/8192 screen, independent official/512 30-pair confirmation, all 20 families, all >3%-slower or wholly-negative screen risks plus carried predicates/reexports, independent class/interface checks, and ALL five six-pair 8192 confirmation batches with separate admission/warmup metadata. Every phase requires 66 positive/negative admissions, alternating fresh processes and stable fingerprints; all observations retained. Large processes of the frozen parent remain ~18s, so separate batches keep the unchanged 600s execution guard. No old candidate observations are merged into this new experiment.

8 days ago

The initial frozen class-owner screen completes with all 66 admissions and unchanged fingerprints. Official checked JS: 29.612 -> 28.146 ms; paired saving interval [+1.326,+1.722] ms. Unchanged 512-family generator: 157.622 -> 130.732 ms, 6/6, [+23.734,+30.319] ms. Unchanged 8192-family generator: 18.589 -> 12.295 s, 6/6, [+6.251,+6.313] s. All observations retained; these are screening results, not acceptance or a new competitor ranking. Independent official/512 confirmation is running, and all five large confirmation batches plus complete family/risk checks remain required. Source stays frozen and uncommitted.

8 days ago

Independent smaller-input confirmation completed, all 66 admissions and fingerprints unchanged: official checked JS 29.662 -> 28.097 ms, 30/30 candidate faster, paired saving CI [+1.339,+1.753] ms; unchanged 512-family generator 157.129 -> 131.756 ms, 30/30, [+23.146,+33.404] ms. These support about 5.3% / 16.1% lower medians in this exact-parent experiment. Large confirmation batch0 is running; batch1/2 and full class/interface risks are queued, the complete 20-family screen is queued, and dynamic/carried-risk rechecks plus remaining large batches3/4 will still be required. The verifier labels pending phases and unknown risk outcomes explicitly, rather than interpreting not-yet-run checks as passing. No source commit, omitted observation, new matrix, or universal leadership claim.

8 days ago

First of the five declared large confirmation batches completes: 21.232 -> 15.187 s medians, 6/6 candidate faster, paired saving CI [+3.440,+7.025] s. This batch has substantial absolute variation (all arrays retained), unlike the initial screen; no filtering or replacement. It is only six of the required 30 independent large observations, not a complete scaling verdict. The source/binary/input/harness fingerprints and all 66 admissions match. Remaining batches and full family/risk checks still required before source acceptance. The fresh five-compiler matrix is prepared but will run only after accepted source publication, not infer a new rank from the A/B.

8 days ago

Large confirmation batch1 completed: 18.255 -> 11.942 s, 6/6 candidate faster, paired saving CI [+6.148,+6.444] s. This is 12 of the required 30 large observations across two separately admitted batches; all first-batch variation remains retained. Source/input/binary/harness fingerprints match and all 66 admissions pass. Class/interface checks and full 20-family screen are next in the shared queue, with remaining large batches and dynamic/carried-risk rechecks still required. No source push or new ranking yet.

8 days ago

Complete 20-family screen and independent declaration-risk checks now finish with all 66 admissions per phase and stable fingerprints. No >3%-slower median or wholly negative screen interval selects an extra family; the predeclared carried large-predicate/reexport 30-pair checks are nevertheless queued. Independent class hierarchy: 22.427 -> 22.432 ms, CI [-0.144,+0.181] ms; interface composition: 33.055 -> 33.042 ms, CI [-0.081,+0.340] ms. Both are inconclusive, not equivalence. All full screen rows remain. Large batches2/3 and final batch4 are now running/queued; no source edits, omitted rounds, acceptance or new competitor rank yet.

8 days ago

Third large confirmation batch completes: 17.908 -> 11.409 s, 6/6 faster, paired saving CI [+6.308,+6.701] s. All 66 admissions and fingerprints pass unchanged. Eighteen of 30 required large observations are now retained across three batches, including the first variable batch. Batches3/4 and the carried predicates/reexports recheck remain running/queued. No source edit, source push or matrix ranking claim yet.

8 days ago

Fourth large confirmation batch completed: 18.872 -> 12.671 s medians, 6/6 faster, paired saving CI [+5.767,+7.127] s, all 66 admissions and stable fingerprints. All 24 completed independent large pairs favor the candidate, with every observation retained. The carried-risk recheck is next/running, followed by final batch4. Full audit will still require all 30 large pairs and independently recomputed pooled/five-batch intervals. No acceptance or source push yet.

8 days ago

Carried-risk 30-pair checks complete with all 66 admissions and matching fingerprints: large predicates 242.898 -> 245.442 ms (11/30), saving CI [-7.941,+7.407] ms; reexports 11.459 -> 11.479 ms (13/30), [-0.806,+0.301] ms. Both have adverse medians but inconclusive intervals, not equivalence or proven regression freedom; every pair remains retained. Only the fifth/final large batch remains. Current complete audit covers 436 paired rounds and 594 admissions; full acceptance still waits for all 442 rounds / 660 admissions and the five-batch calculation.

8 days ago

Landed the class-local typedef owner index in ac34883 and complete docs/README/evidence in 9629578, both sole Chris author/committer without coauthors and pushed directly to origin/main. Full audit: 442 paired rounds / 884 timing samples, 660 successful admissions, 1,570 verified archive checksums. Independent official/512/8192 confirmation medians are 5.3%/16.1%/34.0% lower; all 30 pairs per confirmation favor the candidate. Large pooled saving CI [+5.665,+6.661] s and five-batch CI [+5.509,+6.817] s are positive, with every variable/outlier observation retained. Full checker4536/Program247/driver200/CLI74/entrypoint35/harness136 pass. Widest-span/93,750 range-query/all-growth-OOM controls pass. Original one-test failure, every disk refusal, exact cache-removal manifest and successful retries remain in the archive. Zod remains 195 TS + 3 HM9002, not production admission. Predicate/reexport adverse medians and inconclusive intervals are explicit; no equivalence or universal regression-free claim. Report: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#class-local-typedef-owner-index-2026-10-09-utc . The fresh five-compiler matrix is starting next; the old 1/20 ranking is not changed from A/Bs. Both initially mapped loops now have source-derived indexes, while the overall #416 leadership goal remains incomplete.

8 days ago

The scoped source-derived optimizations for both originally mapped repeated scans are now landed and fully documented: bbabc2f/b513c85 for modifier-merge candidates, ac34883/962957813 for class-local typedef ownership. Complete correctness/OOM/admission and exact-parent family/scaling evidence is published with all failures and inconclusive/adverse observations retained. Closing this scoped investigation, not the overall leadership goal. Fresh matrix and remaining landed-source JSDoc hotpath profiling continue under #416/#838/#854; Zod remains open under #548, and no universal/C++/other-platform claim is made.

Sign in to comment on this issue.