ReviewOS

home-lang/home

fix(ts): preserve qualified issue arrays through imported generic parse contexts

#688
Closed chris opened this 8 days ago ·
8 days ago

Parent

Evidence

After #687 correctly transfers and narrows T | Promise<T>, four Zod 4.5.2 src/v4/core/parse.ts callbacks remain TS7006 at lines 30, 54, 81, and 101. Focused controls show the Promise union and terminating/awaited branches are now exact; the remaining opacity is the nested imported element type behind ParsePayload.issues: errors.$ZodRawIssue[].

A draft whole-constraint projection was rejected because it left these four errors and added 12 unrelated Zod diagnostic identities.

Fair oracle

Use separate owner/consumer modules with a namespace-qualified issue type, an imported payload containing issues: errors.RawIssue[], and both synchronous/async generic parse paths. TypeScript 6.0.3 and tsgo 7 preview are the references. Include a deliberately wrong callback assignment so exact context is proven rather than inferred from silence.

Acceptance

  • TypeScript 6.0.3, tsgo, and Home agree on the focused valid/invalid multisets.
  • No declaration-name, file-name, or benchmark special case.
  • Exact Zod diagnostic identity A/B has zero additions.
  • Full Program/checker suites and optimized builds pass.
  • Evidence is documented and linked from the README.
8 days ago

Progress update after a guarded production-corpus reduction:

  • The standalone positive instanceof assignment-flow gap discovered here is now fixed by #738 in f29b2a0dc.
  • The qualified imported issue-array objective remains open: all four src/v4/core/parse.ts TS7006 identities still reproduce.
  • A broad “nested contextual read is reachable” projection was rejected. On the same 16-file Zod core shard, one candidate bus-faulted at a guarded 3,178 MB peak; restoring the old member fallback completed at 2,629 MB.
  • A bounded variance-aware version completed but produced 164 diagnostics versus the independent origin/main baseline’s 150: 14 additions and no removals. Narrower variants either retained the same addition or left all four target TS7006 sites unchanged.
  • All rejected projection code was removed before commit. No diagnostic suppression, library special case, raised memory cap, or workaround landed.

The next viable direction must be demand-driven projection of the actually read member/result path, not graph-wide reachability across Zod’s densely connected declarations. The full 106-file graph remains unsafe under the fixed 3,840 MB ceiling, so this investigation used the pinned src/v4/core/**/*.ts shard and exact path/line/column/code identity comparisons.

8 days ago

Rejected the contextual-origin projection candidate after the required source-matched production gate.

Evidence:

  • Frozen parent compiler SHA-256: bfba4f9e94731d4e9639094e53fea1e707fd1c4c160157474ee2de86ba1275b3.
  • Standalone oracle improved from exactly 3 TS7006 diagnostics to exactly 3 deliberate TS2322 diagnostics, matching TypeScript 6.0.3 and native TypeScript 7.0.2.
  • Pinned Zod 4.5.2 core gate regressed: parent had 145 unique diagnostic identities; candidate had 157.
  • It removed only the two targeted TS7006 diagnostics in src/v4/core/parse.ts (30:63 and 54:62), but added 14 TS2345 diagnostics across src/v4/core/api.ts, schemas.ts, and util.ts.
  • Peak measured footprints were 567 MB for the frozen parent and 2,637 MB for the candidate, both below the fixed 3,840 MB guard.

The implementation and all candidate tests were fully reverted; no commit was made. The broad design was unsound because projected contextual origins escaped into ordinary assignability/call sites. A future fix must retain the imported generic callback context without publishing those projections globally. Keeping this issue open.

8 days ago

Root cause located and confirmed by instrumentation. Recording it before the trail goes cold; nothing committed, worktree restored to the inherited candidate (277 lines, packages/runtime/test/test.txt untouched).

Mechanism

<T extends schemas.Schema> lowers its constraint to any (types.Primitive.any = 1). With T extends any, schema._zod is any, result.issues is any, and the issue callback parameter becomes an implicit any — the TS7006.

The creating site is not any of the constraint-lowering sites that match the obvious pattern. Captured by panicking in Interner.internFreshTypeParameterWithFlags when the id/constraint matched:

interner.zig:828  internFreshTypeParameterWithFlags
check.zig:78817   lowererLowerWithTypeParamsInner     <-- here
check.zig:76884   lowererLowerWithTypeParams
check.zig:72321   checkTypeAliasDecl

check.zig:78817 is a fifth copy of the const constraint: TypeId = if (tpp.constraint != none) lowererLowerWithTypeParams(...) shape, indented four spaces deeper than the four at ~37049 / ~41735 / ~66072 (+ placeholder pairs 37033/41710/66047). A textual patch of the other four does not touch it.

Why no resolver handles it

Schema is projection_only in the Program graph. Verified against the oracle's own four-module graph:

entry export=ParsePayload     target=/proj/schemas.ts contextual=false projection=true
entry export=SchemaInternals  target=/proj/schemas.ts contextual=false projection=true
entry export=Schema           target=/proj/schemas.ts contextual=false projection=true
entry export=Parse            target=/proj/consumer.ts contextual=true  projection=false

Both resolvers skip exactly that case:

programExportedTypeForImportPath:            if (entry.contextual_only or entry.projection_only) continue;
programContextualExportedTypeForImportPath:  if (entry.projection_only or !entry.contextual_only) continue;

resolveQualifiedTypeRef returns null for the node as well. A projection_only exported type is consumed nowhere in check.zig — every reference to the flag is a skip.

Measured result of letting the constraint use a projection entry

Routing check.zig:78817 through a helper that resolves a projection_only entry (both programDeclarationTypeReference and programContextualExportedDeclarationType give the same outcome):

  • constraint becomes a real Schema type instead of any
  • TS2322 goes 1 -> 2 of the required 3
  • but it introduces TS2339, so it fails the "zero new diagnostic identities" bar and was reverted
#688 diagnostic 2322 x2
#688 diagnostic 2339: Property 'issues' does not exist on type 'ParsePayload<unknown> | Promise<T>'
#688 diagnostic 7006: Parameter 'issue' implicitly has an 'any' type

The next blocker is that union: MaybeAsync<ParsePayload> should be ParsePayload<unknown> | Promise<ParsePayload<unknown>>, and the Promise arm keeps an unsubstituted T, so the instanceof Promise narrowing in parse cannot remove it. A probe on the .parameter substitution fallback shows no MaybeAsync miss, but 17 misses for decl=Parse decl_params=0 args=0 — Parse's own function-level T has no declaration-level parameter to match against, so it falls through to programExpressionParameter. That is the thread to pull next.

Experiment saved at /tmp/688-experiment.patch (243 lines) rather than committed, since it does not meet the acceptance bar.

Suite is 207/208 throughout — unchanged by any of the above.

8 days ago

Continued from the previous comment. The full chain is now mapped end to end. Still nothing committed; worktree restored to the inherited candidate (277 lines), packages/runtime/test/test.txt untouched, suite 207/208 throughout.

Layer 3 — resolving the constraint works, and exposes the next layer

Routing check.zig:78817 through a helper that accepts a projection_only entry gives the constraint a real Schema type instead of any, and moves the oracle TS2322 1 -> 2 of 3. It also introduces TS2339, so it fails the acceptance bar on its own:

2322 x2
2339: Property 'issues' does not exist on type 'ParsePayload<unknown> | Promise<T>'
7006: Parameter 'issue' implicitly has an 'any' type

Layer 4 — why the union does not narrow

MaybeAsync<T> = T | Promise<T>. The Promise arm is lowered by lowerProgramBuiltinReference to buildStructuralPromise (check.zig:~83841) — an object with then/catch/finally plus a __home_promise_value marker — not the nominal Promise class instance. instanceof narrowing discriminates nominally, so the union survives the guard.

promisePayloadType (check.zig:~175699) already recognises that marker, but is used only by promisePayloadArgumentAssignable.

Layer 5 — where an instanceof Promise partition does and does not apply

I added a Promise partition beside the existing rhs_is_array one in the instanceof narrowing (check.zig:~131660), using promisePayloadType to split the union. Instrumentation shows it works correctly where it is reached:

PROBE instanceof-Promise current=46 union=true obj=true
   member=20 obj=true payload=null     <- ParsePayload, subtracted
   member=44 obj=true payload=21       <- structural Promise, kept

but it never fires for the oracle. The oracle's guard is

if (result instanceof Promise) throw new Error("async");
result.issues.map(...)

which narrows the flow after the if via the throwing branch, not either branch of the condition. That is a different mechanism from the when_true/when_false path the Array/Promise partitions live on.

So the remaining work is: make the post-if-throw flow narrowing subtract a structurally-modelled Promise constituent. parseAsync is the sibling case — await on the same union leaves the payload unresolved, producing the TS7006.

Summary of the chain

  1. constraint at check.zig:78817 lowers to any (fifth copy of the pattern, deeper indent than the four at ~37049/~41735/~66072)
  2. because Schema is projection_only, and no resolver accepts a projection_only exported type
  3. resolving it -> TS2322 1 -> 2, but exposes
  4. Promise modelled structurally, so instanceof cannot discriminate it, and
  5. the oracle's guard narrows post-if flow, which the branch partitions do not cover

Each layer is verified by instrumentation rather than inferred. Nothing in layers 3-5 is shippable alone; a fix needs at least 1+2+4+5 together, which is why the piecemeal attempts each measured worse or no better.

8 days ago

Layer 5 resolved, and it rules out the approach I was pursuing. Still nothing committed; worktree restored to the inherited candidate (277 lines), test.txt untouched, 207/208.

Correcting my previous comment: the oracle's if (result instanceof Promise) throw does reach applyTypeGuard. check.zig:9844 calls applyTypeGuard(i.cond, false) when the then-branch definitely transfers control, so post-if flow narrowing runs on the same path as the branch partitions. My earlier statement that it does not was wrong — the probe had simply not been in that build.

With a Promise partition added beside the existing rhs_is_array one, instrumentation shows the whole mechanism working:

PROBE promise-guard current=449 union=true when_true=false
   member=440 obj=true payload=null     <- ParsePayload, subtracted
   member=448 obj=true payload=440      <- structural Promise, kept
PROBE narrowing 449 -> 440 (when_true=false)

The union is correctly identified, the structural promise is correctly recognised via promisePayloadType, and recordNarrow is reached with the right type. And the diagnostic does not change — result.issues still reports ParsePayload<unknown> | Promise<T>, so the member access does not observe the recorded narrowing.

That is the real remaining question, and it is a different one from where I started: not "can the guard narrow this union" (it can) but "why does the member-access path not read the narrowed binding for this const".

Net effect, measured

Layers 1-3 plus the Promise partition together move the oracle from

1x 2322, 1x 7006            (inherited candidate)

to

2x 2322, 2x 2339, 1x 7006   (with all of the above)

More diagnostics, not fewer, and two of them are new identities. It fails the acceptance bar on both counts, so none of it is committed.

What I would tell the next attempt

  • The constraint fix (layer 1-2) is correct and necessary, but only pays off once the downstream union is consumable.
  • Do not spend time on the instanceof narrowing — it demonstrably works; the gap is between recordNarrow and the member-access type lookup for a const binding.
  • The parseAsync half needs its own answer regardless: await on the same union, with no guard to narrow it.
8 days ago

Sixth and final layer for this session. Worktree restored to the inherited candidate (277 lines), test.txt untouched, 207/208, nothing committed.

The TS7006 source, captured by stack trace

The un-narrowed read of result comes from the contextual-typing probe for the inner (issue) => ... callback:

checkExpression                  check.zig:113625
contextualSignatureForCall       check.zig:31645
contextualTargetTypeForFunction  check.zig:30915
checkFnSignatureOnly             check.zig:37300
checkFnDecl                      check.zig:18770
checkFnDeclWithFlowBoundary      check.zig:19872

To type issue, the checker asks for .map's signature, which re-evaluates the receiver result.issues. Instrumenting every access shows the split plainly — most passes see the narrowed binding, exactly one does not, and that one emits the TS2339/TS7006:

PROBE access result.issues obj_t=440 narrow=440    <- narrowed, correct
PROBE access result.issues obj_t=449 narrow=null   <- the probe pass

Worth noting for anyone repeating this: the existing instanceof narrowing already works on this union. 449 -> 440 happens without any change. A Promise partition added beside the rhs_is_array one fires and narrows correctly, and changes nothing, because the failing pass is not the one doing the narrowing. That was a dead end.

Not a visibility problem

checkFnDeclWithFlowBoundary raises narrow_lookup_floor so a nested function cannot see enclosing narrowings. That looked like the cause, and lookupNarrow already has a below-floor escape hatch (this, super, type parameters). I extended it to look below the floor while a contextual_call_probe counter is set around the receiver evaluation in contextualSignatureForCall.

No effect. Which means the narrowing is not merely hidden at that moment — it has not been recorded yet. checkFnSignatureOnly runs the contextual probe during signature checking, before the body's statements (including if (result instanceof Promise) throw) are walked. So the probe is ordered before the flow fact exists, and no lookup change can fix it.

Full chain

  1. constraint at check.zig:78817 lowers to any — a fifth copy of the pattern, deeper-indented than the four the brief names
  2. because Schema is projection_only, which every resolver skips
  3. resolving it: TS2322 1 -> 2, exposes TS2339
  4. Promise is modelled structurally (buildStructuralPromise), not nominally
  5. the instanceof narrowing nonetheless already works — layer 4 is a red herring
  6. the failing read is the contextual-signature probe, which is ordered before the narrowing is recorded

Layers 1-2 remain correct and necessary. Layer 6 is the blocker and is an ordering question in checkFnSignatureOnly, not a resolver or narrowing question. That is where the next attempt should start, and it should establish whether the contextual probe can be deferred until after the enclosing body's flow is walked, or whether the callback's contextual type must be recomputed at the use site.

8 days ago

Landed on main, without closing the issue:

  • bb5e069 fix(ts): distribute await over union constituents
  • 71c4a1a fix(ts): resolve qualified Program constraints for issue-array callbacks
  • cfdec2e docs(ts): describe the shared constraint lowering path

Full write-up: Qualified Program constraints and contextual issue arrays (untimed), linked from the README.

Acceptance

CheckResult
Four-module oracle (/private/tmp/home-goal-688-oracle)TypeScript 6.0.3, native TypeScript 7.0.2 (bench/vs_tsgo/.tools), and ReleaseSafe Home all report exactly 3× TS2322 at consumer.ts 7:9, 18:11, 31:11, with no TS7006. main reports 3× TS7006.
Function-declaration/class constraints, valid raw-issue pushes, await over unionsExact three-engine file:line:column:code parity on each control graph
Zod 4.5.2 core (tsconfig.core.json) vs main 65f80a525150 → 146 diagnostics, 145 → 141 identities: 4 removed (core/parse.ts 30:63, 54:62, 81:66, 101:44 TS7006), 0 added
SuitesProgram 210/210, checker 4,371/4,371 on the exact commit tree
BuildsReleaseSafe and ReleaseFast home-tsc, zig fmt --check, git diff --check

Commit bb5e069 was verified on its own first: checker 4,371, Program 205, ReleaseSafe, and an unchanged Zod core (150/145, none added or removed).

What the fix is

  1. Constraints. All five type-parameter declaration forms (function types, function declarations, classes, interfaces, type aliases) lower extends through typeParameterConstraintType. When plain lowering yields any for a qualified reference, the helper resolves it or its projection_only Program export.
  2. Awaited unions. Awaited<T> distributes, so A | Promise<A> awaits to A.
  3. Projection scope. The issue-array element is a contextual read projection. It now types array-method callbacks on that member and is never substituted as the member's value type, the same scope main gives programImportedClassContextualMemberType.

Corrections to my earlier comments

  • The narrowing-floor visibility change I called the "layer 6" blocker is not part of the fix. Once constraints resolved it was unnecessary, and it leaked narrowings into nested functions (+1 identity, core/schemas.ts:2002:9 TS2322).
  • The inherited candidate's value-type substitution was the source of twelve TS2345 additions (core/schemas.ts payload.issues.push({ code: "invalid_format", ... })). The displayed projection keeps path/message required where $ZodRawIssue makes them optional. This issue's body also records a rejected draft that added "12 unrelated" identities. The count matches, but I have not verified that the two sets are identical.
  • The inherited class_schema.zig alone adds three TS2801. Without the distributive await the combined candidate still adds core/checks.ts:821:9; with it, none remain.

Follow-up

The same interface written as a direct annotation or explicit type argument (schema: schemas.Schema, Holder<schemas.Schema>) still yields TS7006 where both engines report the deliberate TS2322. It is tracked with a five-row isolation table in #751.

8 days ago

Closing as completed by bb5e069, 71c4a1a, and cfdec2e.

The acceptance evidence already recorded above proves:

  • TypeScript 6.0.3, native TypeScript 7.0.2, and Home agree on the focused valid/invalid multisets
  • all four targeted Zod src/v4/core/parse.ts TS7006 identities were removed
  • the pinned Zod core A/B had zero additions
  • Program 210/210 and checker 4,371/4,371 passed
  • ReleaseSafe and ReleaseFast builds passed
  • the implementation is general to qualified Program constraints, without declaration/file special cases

The direct-annotation and explicit-type-argument follow-up is independently tracked by #751 and should not keep this narrower constraint issue open.

Sign in to comment on this issue.