ReviewOS

home-lang/home

fix(ts): bound generic substitution on real React declaration graphs

#842
Open chris opened this 8 days ago ·
8 days ago

Faithfully mounting TypeScript's real tests/lib declarations exposes excessive generic-substitution work in Home's checker. This is a remaining full-corpus prerequisite, linked to #66, #832 and the compiler-input audit #841. Do not restore simplified declaration inputs or raise the memory ceiling.

Observed in the active compiler-input batch based on 0234bc598:

  • Real /.lib files are mounted byte-for-byte from the pinned nested TypeScript tree. Missing absolute package declarations remain missing instead of being synthesized as any.
  • The ordinary zig build test -Dfilter=ts_conformance --summary all run was stopped by the normal supervisor with exit 125 at the unchanged 3,840 MB runaway ceiling. The run did not produce a complete test aggregate.
  • A native process sample during the baseline-aware type-relationship survey shows Program.compileAll -> Checker.checkInterfaceDecl -> reportInterfaceExtendsNonObject -> repeated substituteType/substituteTypeUncached calls.
  • The driver currently evaluates declaration semantics before filtering their diagnostics when skipLibCheck is requested. The conformance program path already forwarded direct skip directives, but selected-variant/configuration handling and the legacy route are being corrected in the same batch.

Pins: typescript-go 89d5d5b2849a0db0957065889ca58536fa6d2e4a; nested TypeScript 5848bc5157b22ff7f4e3369f4645a514a433b15f; Zig 0.17.0-dev.2163+89ff10d56.

Acceptance:

  • Identify and retain the exact original fixture and declaration graph causing excessive work.
  • Resolve shared/cyclic generic substitution with correct binder identity, constraints, metadata and deterministic results; do not replace types, skip unrelated checks or alter baselines.
  • Implement the reference compiler's declaration-check policy faithfully, preserving application diagnostics and declaration type metadata.
  • Re-run the real declarations and complete baseline survey under the normal lock, original test limits and 3,840 MB ceiling.
  • Retain the unsuccessful run and process sample alongside corrected evidence.
8 days ago

Published cc1128f62 directly to origin/main as Chris Breuer without coauthors.

Compiler inputs are now independent of expected diagnostic contents. The pinned TS7 GetStrictOptionValue returns true when strict is unset (Strict != TSFalse); this is applied consistently, then overridden by actual tsconfig, harness directives and explicitly selected variant options. No diagnostic-code scanning supplies strictness, and no expected header supplies the actual filename. Boolean variant parsing is limited to exact keys inside the selected filename's option list.

The fabricated any-exporting absolute package declarations are removed. Missing targets remain missing. Real nested TypeScript tests/lib bytes are mounted at /.lib; libFiles creates real program roots. Library imports, absolute package types and globals retain their declaration types. Actual skipLibCheck settings are preserved through configuration/variant selection and both compile paths. Direct skip directives were already supported in the program route; the corrected handling extends the missing paths and removes the raw text-based config approximation.

Observed results:

  • final input controls 6/6, including mutated-header invariance, physical library/package/global type preservation, missing input rejection and application errors under skipLibCheck;
  • real survey-gate controls 10/10;
  • unchanged type-only regression fixtures 2/2 exact;
  • corrected leading exact prefix 166/200, 34 actual failures, zero skips, exits 1;
  • full normal conformance attempt stopped with exit 125 at the unchanged 3,840 MB ceiling; no complete aggregate. This attempt preceded final library-policy wiring. The driver still eagerly evaluates declaration semantics before suppressing their diagnostics. A retained process sample shows repeated generic substitutions in the baseline survey on the real React graph.

Raw evidence and source/binary/library hashes include successful controls, compilation failures, the aborted run and process sample. No upstream source, baseline, skip, TODO, deadline or memory/ratchet limit was changed. The patch only changes the conformance harness, so no new standalone compiler or Zod result is claimed.

#842 now tracks the real declaration-graph substitution and checking cost. #841 remains open for complete variant execution, root-file inclusion and real default-library/compiler API inputs. #832 and full Bun acceptance in #66 remain open. These intermediate survey outcomes do not prove full TypeScript or Bun parity; controls earn no original Bun case credit.

8 days ago

Published source e2c643d68 and integrated evidence through 348e36e92 to origin/main. Main includes the concurrently published #839 export-owner improvements; no work was overwritten or force-pushed.

The source batch fixes five related behaviors:

  • Member-predicate metadata follows the actual rebuilt signature's binder. It no longer re-instantiates a member or its already substituted signature predicate a second time. Receiver-specific predicates preserve their separate substitution path.
  • Signature constraints/defaults share the original memo while their local substitution environment is unchanged; separate environments still produce independent binders.
  • Explicit skipLibCheck in declaration files avoids diagnostic-only interface inheritance/index relation passes while retaining inherited shapes, checked metadata and syntax diagnostics. Application validation remains active; other declaration processing still runs.
  • Program checking and cached emit use each file's .ts/.d.ts/.tsx syntax mode. A JSX root no longer forces JSX parsing onto its declaration dependencies.
  • Contextual parameter names such as object work in ordinary/assertion predicates, preserving parameter indices and regular non-predicate returns.

Observed gates: substitution controls 4/4; declaration-policy controls 3/3; complete checker 4,486/4,486; driver 198/198; latest program 232/232; parser/precedence 905/905; real survey controls 10/10. Latest stripped compiler builds under the 3,840 MB ceiling. Pinned 106-file Zod stays at 196 TS diagnostics and three HM9002 recoveries with the ceiling unchanged. Its diagnostic text changed from the prior checkpoint, and the normalized diff is retained; latest integrated output matches the first substitution build. This is a recovery ratchet, not a passing Zod project.

Real original-case evidence is deliberately not green:

  • Before the contextual-predicate parser fix, correcting per-file JSX mode reduced the two original React cases' peak from 3,597 MB to 154 MB, but both still failed exact comparison. This isolated real object-is-ReactElement syntax errors instead of fabricated JSX errors inside declaration comments.
  • Restoring that real predicate syntax exposes additional semantic work. The original React selection stopped at 3,851 MB against the unchanged 3,840 MB ceiling while intraExpressionInferencesJsx was active; its second case was unstarted.
  • The fresh integrated leading 200-case attempt stopped at the same ceiling (3,842 MB) on multiline.tsx and produced no complete aggregate. The previous 166/200 result remains historical to pre-predicate syntax and is not a latest-tree passing count.
  • A retained sample shows collectProgramCommonJsExports -> moduleExportAssignmentInfo -> compileSource -> full declaration checking. Export discovery is still recompiling/checking an owner instead of consuming the retained bound source. This is the next concrete #842 target, alongside correct generic/declaration resolution.

Evidence, failures, samples and hashes are committed. Disk-critical/admission failures and a known-bug intermediate interruption are retained. Only inactive generated test/build caches and regenerable download/HTTP caches were reclaimed; source, pinned Bun build objects and pinned WebKit libraries were preserved. No fixture, baseline, original skip/TODO, deadline or memory/ratchet ceiling was modified. No unfinished, interrupted or skipped test earns passing-case credit.

#842 remains open for these real owner/type-graph costs. #841 retains complete input/variant/default-library fidelity, #832 full TS corpus evidence, and #66 full Bun acceptance. The persistent full-port goal remains active.

8 days ago

Published source 51db032e9 and evidence 1602e3d63 to origin/main.

Declaration export-assignment discovery now inspects the retained compilation instead of reparsing and fully checking it. The independent source query prepares syntax, and conformance resolver callbacks reuse their bound owner. Normal compilation still checks declaration semantics: a control verifies the same owner later reports a genuinely missing type.

Validation:

  • Complete Program suite: 234/234.
  • Retained resolver controls: 4/4; opt-in runner controls: 3/3; survey admission/mismatch controls: 10/10.
  • Standalone compiler builds under the unchanged 3,840 MB ceiling, with 2,866 MB observed peak.
  • Pinned 106-file Zod recovery check retains 196 TS diagnostics and three HM9002 recoveries, ceiling three, at 596 MB. Diagnostics match the preceding checkpoint after normalizing only its temporary directory; this is not a passing Zod project.

The two original React cases were requested with unchanged declarations, baselines and the original 900-second deadline. The first, intraExpressionInferencesJsx, stops at 3,844 MB against the unchanged 3,840 MB ceiling (exit 125). The second, contextuallyTypedStringLiteralsInJsxAttributes02, is unstarted. No complete aggregate or passing-case credit exists. The retained process sample now places the work in normal declaration member lowering/generic substitution, rather than the removed export-discovery check.

An initial compiler admission attempt expired while waiting for the shared lock before compilation began; the same-limit retry succeeds. Both attempts are retained. Pins, source hashes, successful controls, resource failure and sample are in docs/docs/bun-port-evidence/2026-10-07-ts-export-assignment-owners.

Remaining work is correct shared/cyclic type substitution and faithful declaration-check policy in #842, with input fidelity in #841 and complete corpus/Bun acceptance in #832/#66. Full parity remains incomplete; previously fabricated corpus claims remain withdrawn (#840).

8 days ago

Published source b1242be3a and evidence 0b0339a21 to origin/main.

Generic signatures now share a substitution memo once their local binder map is fixed, across parameter, return, receiver and predicate types. The enclosing memo is restored afterward, so separate binder environments remain independent. Two new controls fail on the preceding implementation (excessive graph rebuilding and different parameter/receiver identities) and pass without changing their expectations. Nested binders/defaults and assertion-predicate identity are checked.

Validation: substitution controls 6/6; complete checker 4,488/4,488; driver 198/198; Program 234/234; opt-in runner controls 3/3; survey admission/mismatch controls 10/10. The standalone compiler builds at 3,024 MB observed peak. Pinned 106-file Zod retains 196 TS diagnostics and three HM9002 recoveries, ceiling unchanged, at 617 MB. Diagnostic text matches the preceding checkpoint after normalizing only its temporary directory; this is a recovery check, not a passing project.

The original React retry still stops at the unchanged 3,840 MB ceiling, observed 3,841 MB, exit 125. intraExpressionInferencesJsx started; contextuallyTypedStringLiteralsInJsxAttributes02 is unstarted. No complete aggregate or passing-case credit exists. Original declarations, expected diagnostics, skips, TODOs and the 900-second deadline are unchanged. A retained sample from the earlier disk-stopped attempt on the same source shows declaration member lowering and recursive inheritance-base metadata substitution (decl_single_base). Signature-map sharing does not solve these remaining recursive costs.

Disk admission refusals, the critical-disk interruption, and successful retries are retained separately. Inactive generated test/build caches and older registry-cache copies were reclaimed while preserving source, installed runtimes and pinned Bun/WebKit artifacts. All retries used unchanged resource limits. Pins, source/artifact hashes, negative controls and raw results are in docs/docs/bun-port-evidence/2026-10-08-ts-signature-memo-scopes.

#842 remains open for correct recursive/shared substitution and faithful declaration-check policy; #841 tracks input fidelity and #832/#66 full corpus/Bun acceptance. Full parity remains incomplete. Previously fabricated corpus claims remain withdrawn (#840).

8 days ago

Published source 0b8d584c1 and evidence 53167b7e5 to origin/main.

Receiver predicates now share their callable's rebuilt target when their resolved original targets match, while retaining receiver-owned annotation locations, parameter indices and assertion kinds. Unknown targets with different annotation nodes remain independently resolvable. This prevents creating another local binder when only a source location or receiver override differs.

Negative controls are retained: the preceding implementation fails the location/binder control (2 pass / 1 fail); the initial correction then fails an added receiver-override control (3 pass / 1 fail). Final controls pass 4/4 without weakening their assertions, including guard/assertion variants, independent string/number environments and different unresolved targets.

Complete checker 4,491/4,491, driver 198/198, Program 234/234, opt-in runner controls 3/3 and survey admission/mismatch controls 10/10 pass. The standalone compiler builds at 2,993 MB peak. Pinned 106-file Zod retains 196 TS diagnostics and three HM9002 recoveries, ceiling unchanged, at 615 MB. Its diagnostics match the preceding checkpoint after normalizing only its temporary directory; this recovery check does not establish a passing project.

The unchanged original React attempt stops at 3,844 MB against the unchanged 3,840 MB ceiling (exit 125). intraExpressionInferencesJsx starts; contextuallyTypedStringLiteralsInJsxAttributes02 is unstarted. No complete aggregate or passing-case credit exists. Original fixtures, declarations, baselines, skips, TODOs and the 900-second deadline remain unchanged.

The retained sample pins the remaining repeated work to decl_single_base inheritance-base metadata substitution at check.zig:166393, reached during declaration member lowering. Earlier signature-memo notes and issue comments are corrected to name this inheritance path rather than receiver predicates. The reproduced predicate defects are fixed; recursive inheritance metadata remains unresolved.

Pins, source/artifact hashes, all controls, failure and sample are in docs/docs/bun-port-evidence/2026-10-08-ts-receiver-predicate-targets. #842 remains open for recursive/shared substitution and faithful declaration-check policy; #841 tracks input fidelity and #832/#66 complete corpus/Bun acceptance. Full parity remains incomplete, and fabricated corpus claims remain withdrawn (#840).

8 days ago

Mapped inheritance-base references and lifetime fixes are published in 4656fcaf3, with retained evidence.

  • Exact source bodies, declaration binders and mapped arguments remain pool-owned; consumers resolve the base surface on demand. Non-parameter rewrites retain general substitution. No mapper escapes its scope.
  • Controls cover independent arguments, eight-level metadata deferral, own/inherited ordering, normalization, ownership transfer and forced member-pool relocation. Original negative failures and the initial crash are retained.
  • Tested source stage ab5d774e4: checker 4,501/4,501, driver 199/199, Program 238/238, opt-in 3/3; compiler build completes. Pinned Zod reports 195 TS diagnostics / three HM9002 recoveries, with the ceiling unchanged. Its normalized output differs from the preceding 196-diagnostic checkpoint; the diff is retained.
  • Later fixture-harness stage 773ca888f: opt-in 3/3, survey admission/mismatch controls 10/10. These are control results, not original corpus-case credit; subsequent fixture-library changes are not covered by these aggregates.
  • Original React on 4656fcaf3 exits 125 at 3,842 MB against the unchanged 3,840 MB ceiling; diagnostic replay exits 125 at 3,844 MB. The first fixture is unfinished and the second unstarted. No complete aggregate. The replay sample now shows structural member/union substitution in generic interface instantiation as the remaining dominant work.

Shared/cyclic substitution and faithful declaration-check policy remain open in #842; input fidelity and complete corpus acceptance remain open in #841/#832/#66. Fixtures, expectations, original skips/TODOs, deadline and resource ceilings were not weakened.

8 days ago

Owned cyclic substitution references and lexical binder scope are on main in 9e98f427d and 774f72f16, with retained evidence.

Back edges preserve exact source bodies, declaration-parameter identities and mapped arguments. Completed shared ancestors can memoize without retaining an unsubstituted recursive edge. Consumers use the generic resolver for structural assignability. A subsequent regression caught loss of a recursive generic method binder; lexical scope now preserves that binder, constraints and defaults. Non-parameter rewrite maps retain the unresolved guarded path; full graph/declaration-policy acceptance remains open.

Corrected source 774f72f16 passes 4,508/4,508 checker, 200/200 driver, 241/241 Program, 6/6 focused tests, 3/3 opt-in controls and 10/10 survey admission/mismatch controls. The compiler builds at 2,896 MB. Pinned 106-file Zod retains 195 TS diagnostics / three HM9002 recoveries, ceiling three, at 512 MB. Normalized diagnostics are unchanged from the preceding checkpoint; Zod is not a passing project.

The unchanged original React selection exits 124 at the original 900-second deadline, under the normal machine lock and 3,840 MB ceiling. intraExpressionInferencesJsx is unfinished; contextuallyTypedStringLiteralsInJsxAttributes02 is unstarted. No complete aggregate or passing-case credit. The timeout path does not report its final memory peak, and this run does not establish that the earlier memory failure is solved or a controlled performance improvement across fixture-input changes.

Retained samples distinguish phases: owner-specific transfer at about one minute; interface/base refresh and UMD source scans at about seven minutes; generic constraint comparison dominated by relation.Engine.substituteTpDeepLimit at about ten minutes. The engine's separate structural substitution is the next graph-correctness target. Initial negative failures, the binder regression, an unsuccessful scope compile and deliberately interrupted first React attempt remain in the evidence. Original fixtures, declarations, expectations, skips/TODOs and limits were not weakened.

Remaining work stays linked in #842, #841, #832 and #66.

8 days ago

The relation substitution service is published in d1b893abf, with source-qualified evidence.

Checked-source comparisons retain callable, receiver-predicate and declaration metadata through the owning checker. The standalone graph walk memoizes a fixed mapper, preserves owned recursive references and maps tuple/reference arguments without expanding their bodies. Positional maps apply simultaneously, matching the pinned reference compiler. Generic binders retain constraints, defaults, variance, constness and declaration origins; callback destruction detaches only its active owner.

Negative controls exposed repeated shared-graph work, lost receiver/predicate metadata, unmapped tuple/reference arguments, transitive positional renaming, a recursive-edge failure and partial completion-cache entries. Allocation-failure injection also exposed a pre-existing 32,768-byte L1 leak when L2 construction failed; cleanup now releases it. Those failures, an initial compile error and disk admission rejection are retained. The original admission/resource floors were not bypassed.

After preserving the concurrent CommonJS-index integration, source d1b893abf passes 4,519/4,519 checker, 200/200 driver, 242/242 Program, 8/8 graph tests, 4/4 metadata/ownership/recovery tests, 3/3 opt-in controls and 10/10 survey admission/mismatch controls. Compiler builds at 2,901 MB. Pinned 106-file Zod retains 195 TS diagnostics / three HM9002 recoveries, ceiling three, at 478 MB; normalized diagnostics are unchanged. Zod is not a passing project.

The original React selection is running at the retained 2026-10-08 18:06:59 UTC snapshot, under the normal lock, unchanged 900-second deadline and 3,840 MB ceiling. intraExpressionInferencesJsx is unfinished and contextuallyTypedStringLiteralsInJsxAttributes02 unstarted. No terminal result, complete aggregate or passing-case credit is claimed. Its four-minute sample shows repeated UMD namespace source scans during interface checking; sampled footprint is not a final peak. The earlier pre-integration attempt was deliberately interrupted to validate the combined source and is retained as exit 130.

The running measurement still needs its terminal evidence. Earlier timeout/memory failures are not proved solved. Full graph/declaration-policy, input fidelity and corpus/Bun acceptance remain open in #842/#841/#832/#66. Original fixtures, declarations, expectations, skips/TODOs and limits were not weakened.

8 days ago

Parsed UMD namespace facts and exact indexed lookups are published in 30ab18c8c, with retained evidence.

The parser retains decoded names, source nodes, top-level scope and external-module eligibility alongside the erased declaration. Checker and Program use these facts instead of independently scanning raw text. Comments, strings and nested invalid declarations do not create UMD globals; token trivia and escaped names remain valid. Exact lookup caching preserves root/virtual-file restrictions and noImplicitReferences, and invalidates on source attachment, declaration mode and importer-path changes. Dynamic import expressions do not promote declaration-only scripts into modules.

Retained negative controls show two checker failures and one Program failure against the old scanners. Focused controls pass 6/6, parser selection 1,065/1,065, and the combined source after preserving require-binding integration passes 4,526/4,526 checker, 200/200 driver, 243/243 Program, 3/3 opt-in controls and 10/10 survey admission/mismatch controls. The compiler builds at 2,915 MB. Pinned 106-file Zod retains 195 TS diagnostics / three HM9002 recoveries, ceiling three, at 545 MB, with normalized output unchanged. Zod is not a passing project.

The unchanged original React selection on 30ab18c8c ends with exit 125 at observed 3,874 MB against the unchanged 3,840 MB ceiling. intraExpressionInferencesJsx is unfinished and contextuallyTypedStringLiteralsInJsxAttributes02 unstarted. No complete aggregate or passing-case credit. Its sample at about six minutes shows Program.importProgramGlobals through owner import, checked transfer and type_transfer.prepare/materialization, including repeated mapped argument allocation/free. The sampled 2.1 GB footprint is not the final peak; a later sample attempt found the process already terminal and produced no sample. These are phase observations, not a controlled performance claim.

The preceding relation-service measurement is no longer merely a running checkpoint: its terminal 900-second timeout and late UMD scan sample are published in bea9ab4ec. The first UMD-source probe was deliberately interrupted for combined-source validation and is retained as exit 130. Original fixtures, declarations, expectations, skips/TODOs, deadline and resource limits were not weakened.

Correct owner-specific graph transfer and declaration policy remain open in #842; input fidelity and complete corpus/Bun acceptance remain open in #841/#832/#66.

8 days ago

Owner-transfer scratch reuse and member batching are published in bdb366c03, with retained evidence.

The builder reuses separate bounded buffers for mapped type arguments, template text and tuple elements. Interner consumers copy those borrowed results before reuse; signature parameters are consumed before generic binder mapping uses the same buffer. Object finalization reserves the member batch once. Every source type, source owner, declaration identity, metadata table and validation remains represented, and existing unpublished-pool rollback remains active on allocation/callback failure.

A 64-row control alternates widths and exercises receiver/constructor signatures, generic binders, tuple optionality, template text and nested references. It destroys the source and changes callbacks before validating destination contents. Both transfer selections pass 11/11, including exhaustive allocation-failure rollback, cancellation, malformed-input rejection and distinct owner identities. On this exact workload, measured destination allocations fall 256 → 81. This is an allocation count for the control, not a whole-compiler improvement or completion claim.

Tested source passes 4,527/4,527 checker, 200/200 driver, 243/243 Program, 3/3 opt-in controls and 10/10 survey admission/mismatch controls. Compiler builds at 3,067 MB. Pinned 106-file Zod retains 195 TS diagnostics / three HM9002 recoveries, ceiling three, at 490 MB; normalized diagnostics are unchanged. Zod is not a passing project. The concurrent source rebase included documentation only; recorded code hashes match the published source.

The unchanged original React selection exits 125 at observed 3,849 MB against the unchanged 3,840 MB ceiling. intraExpressionInferencesJsx is unfinished; contextuallyTypedStringLiteralsInJsxAttributes02 is unstarted. No complete aggregate or passing-case credit. A process sample attempt found the process already terminal and produced no sample. Scratch allocation reduction has not completed the owner-transfer graph or original corpus.

Correct graph transfer/declaration policy and complete input/corpus/Bun acceptance remain open in #842/#841/#832/#66. Original fixtures, declarations, expectations, skips/TODOs, 900-second deadline, resource ceilings and normal lock were not weakened.

8 days ago

Related guarded observation in #908: a one-virtual-JS exact conformance fixture passed 1/1 but took about two minutes on the debug test path. Two five-second live samples showed Program.loadImportClosure -> compileAll -> checkInterfaceDecl -> lowererLowerWithTypeParams/substituteType throughout, while peak tree footprint was only 349 MB. This reinforces that declaration semantic work and generic lowering need a correct underlying optimization; it does not justify skipping checks or replacing library inputs.

Sign in to comment on this issue.