ReviewOS

home-lang/home

perf(ts): avoid allocating module resolution keys on cache hits

#851
Open chris opened this 8 days ago ·
8 days ago

Evidence and scope

Follow-up to #416/#838 and the fresh verified 20261008T202953Z matrix. Home leads only 1/20; repeated resolver work remains a candidate shared cost for module-heavy workloads. Current Resolver.resolve constructs and arena-allocates the full containing-file + NUL + specifier string before cache.get, including repeated hits. The CLI adapter invokes this path under its shared resolver lock. Allocation count and retained arena memory therefore grow with hits even though the key/result are already present.

Proposed work

Use a borrowed/adapted lookup of the same byte-exact concatenated key, allocating owned key storage only for actual insertion. Keep the existing cache representation and lifetime where practical. No benchmark recognition, approximated resolution, suppressed tracing/diagnostics, foreign TypeIds, numeric semantic limits, or bypassed importer/mode conditions. Preserve trace banners, the per-call ambiguous_root reset, bundler importer-extension/module conditions, non-caching of TS2209/2210 ambiguity failures, negative cache behavior and complete resolution on allocation failure.

Acceptance

  • Prove borrowed hash/equality matches the current full key across empty/mixed/binary segments and delimiter boundaries.
  • Allocation/arena-growth controls for repeated positive and negative hits; source-key ownership and every insertion allocation-failure path.
  • Full resolver/checker/Program/driver/CLI gates, semantic source/option controls and pinned Zod diagnostic identity comparison.
  • Exact-parent unfiltered paired confirmation plus the complete family screen and independent adverse-family rechecks; include failure/inconclusive outcomes.
  • Publish source and evidence in small issue-linked Chris commits without coauthors, detailed docs and a brief README table. Do not claim a new competitor ranking without a complete admitted matrix.
8 days ago

Draft implemented locally, not committed: adapted borrowed lookup preserves the existing StringHashMap representation and exact containing-file+NUL+specifier bytes. Streaming hashes/equality pass 16900 binary/empty/length-boundary cases, including embedded-delimiter contracts. All 124 resolver tests pass, including 8192 repeated positive/negative hits with zero allocator calls/arena growth/additional traces, stale-ambiguity reset, owned caller keys, every insertion-allocation failure until success/retry, importer MTS/CTS conditions and repeated non-cacheable ambiguity. An initial context-initializer syntax error failed before execution and is retained; no tests were removed. Supported stripped native build and full Program gate are running. Exact-parent frozen native is c16c8ca, compiler inputs equal to ac8af9c. Predeclared A/B: official imports/reexports/CommonJS (20 screen, 30 independent confirmation pairs), existing CommonJS1024 and reexport4096 generators with first/middle/last negative controls (6 screen, 30 confirmation), all20 family screens (10 pairs), independent30-pair rechecks for >3% slower medians/negative intervals plus carried-forward large-predicate risk. No speedup or production change claimed yet.

8 days ago

Final draft gates now pass: resolver124, Program247, checker4527, driver200, CLI74, entrypoint35 and supported stripped ReleaseFast build. Native build peak2906 MB, Program3074 MB, entrypoint2978 MB under unchanged3840 guard. The 105 complete primary diagnostic oracle cases match exact-parent Home and nativeTS7; all86 shared-config TS6 cases match, with19 documented CommonJS/node10 option-dialect differences preserved. Pinned unmodified Zod106 files has identical195 TS+3HM9002 full primary message/position multiset, normalized SHA6bf2feb527d91c651631642e523da6d28cd2b0c7ae4898cac0888aa7b420874f (not production admission). Frozen candidate SHA cab3dc5a3c355208bd213f810b38423c990719fa826a77bfad2679cb6548cae1; parent c16c8caaf52d35180a7fcd53b87cac0dd71ff7b86526ef285e09bb4e088335f4; exact patch SHA7890fb1ccba6e2cc52677cec6e71be21134dba480269619f48c407d14e3a479c. Exact-parent screen is running with64 required official/scale admissions. Code remains local/uncommitted; no performance claim until independent confirmation and adverse-family rechecks.

8 days ago

First unfiltered exact-parent screen is adverse/inconclusive despite proven allocation removal. All64 official/scale admissions and before/after fingerprints pass; peak872 MB. Import19.902->20.451 ms, 8/20, saving CI[-0.948,+0.066]ms; reexport12.000->12.725 ms,4/20, CI[-2.392,-0.176]ms (clear adverse screen); CommonJS24.045->24.374 ms,8/20,[-0.578,+0.284]ms; CommonJS1024 281.128->285.276 ms,2/6,[-6.841,+2.839]ms; reexport4096 1323.521->1282.795 ms,3/6,[-49.041,+111.533]ms. Every observation is retained. The predeclared independent30-pair confirmation on all five targets and complete20-family screen are running/queued. No source push or speed claim; memory/allocation removal by itself does not justify accepting a latency tradeoff.

8 days ago

Independent confirmation establishes an unacceptable latency tradeoff; the draft is rejected, not pushed. All64 admissions and fingerprints pass in every phase. 30-pair confirmation: imports20.513->20.713 ms,11/30,CI[-0.700,+0.067]ms; reexports12.569->12.783 ms,13/30,CI[-1.300,-0.046]ms; CommonJS25.177->25.226 ms,11/30,[-0.371,+0.232]ms; CommonJS1024 291.316->293.459 ms,13/30,[-12.614,-0.817]ms; reexport4096 1261.839->1298.532 ms,4/30,[-73.518,-20.605]ms. Allocation-free hits are proven but do not compensate for these latency regressions. All20 family screens are retained; no >3%-slower-median or wholly negative family interval selects an extra family, and the predeclared carried-forward large-predicate30-pair recheck is running. I will restore the production resolver and retain only general ownership/importer-condition/ambiguity controls, then publish checksum-covered rejection evidence and README/docs scope. The streaming adapted-hash cost is a hypothesis, not an established cause; no alignment tweak, timing filter or favorable replacement run will be used.

8 days ago

Published on origin/main as Chris Breuer, without coauthors:

  • db28b2e: test(ts): retain resolver key ownership and condition controls (#851).
  • dce5796: docs(ts): preserve rejected resolver lookup evidence (#851).

The optimization did NOT land. Production resolve/cache insertion is restored; only two general ownership/importer-condition/repeated-ambiguity controls and corrected key documentation remain. Restored resolver122/122 and supported native build pass. Rebuilt native bytes EXACTLY equal accepted parent c16c8caaf52d35180a7fcd53b87cac0dd71ff7b86526ef285e09bb4e088335f4, so this does not change the current1/20 matrix runtime.

Independent30-pair confirmation is adverse for official reexports (1.7% slower median; mean-saving CI -1.300 to -0.046 ms), CommonJS1024 (0.7% slower; -12.614 to -0.817 ms), and reexports4096 (2.9% slower; -73.518 to -20.605 ms). Import/CommonJS official intervals and carried-forward predicate recheck span zero; no equivalence claim and no favorable phase replaces an adverse one. Zero-allocation/arena-growth hits are proven but do not justify this latency tradeoff. The cause remains unproven.

README table and report: https://github.com/home-lang/home/blob/main/docs/docs/TS_PERFORMANCE.md#borrowed-resolver-key-lookup-rejected-2026-10-08-utc

Archive: bench/vs_tsgo/evidence/20261008-resolver-borrowed-keys-rejected-851.tar.gz; SHA-256 306b81723e6c9b342e86a04b90634f5caf7981aeedad0fffd0d78651b14dd10b; all13078 payload checksums verify. It retains452 raw rounds,256 admissions, every20-family screen, independent risk recheck, exact scale/oracle inputs,105 primary diagnostic cases, unchanged full Zod195TS+3HM9002 messages/positions, tests/builds, syntax-failure record, rejected and retained patches/snapshots and verifiers. Limits/lock/disk floors are unchanged. #851 remains OPEN for a different measured design; the overall #416/#838/#548 goals are not complete.

8 days ago

Current-main bounded baseline check (2026-10-19): attempted pantry/.bin/zig build home-tsc -j1 -Doptimize=ReleaseFast -Denable_jsc=false --summary all through the existing 3,840 MB run_bounded guard. The supervisor stopped it at measured tree footprint 3,877 MB (largest sampled compiler process 3,875.4 MB; host low-water 31%), exit 125. This is a guard refusal, not a proven OS OOM or a benchmark result. No #851 source candidate, timing claim, assertion change, or new optimization was pushed. The previously rejected segmented-Wyhash design remains absent from main; a different design still requires an exact-parent build and full qualification.

Sign in to comment on this issue.