JIT: implement OSR exits, deoptimization, and precise GC

Parent: #146 Blocks: #429 Depends on: #431 (completed) Related: #132, #133 Children: #435, #436, #437, #438, #439 (completed)

Make guarded optimized execution resumable and precisely collectable. Side exits must reconstruct exact bytecode state; exceptions, traps, and GC safepoints must not lose or invent observable work.

Acceptance

• Define OSR entry eligibility and immutable frame-state metadata at every guard, call, exception edge, and safepoint.
• Reconstruct locals, operand stack, accumulator, handlers, and exact step/IP state on deoptimization.
• Emit precise stack maps for movable GC values and validate relocation while optimized frames are live.
• Preserve catch/finally edges, termination, watchdog, debugger, and interpreter/baseline fallback behavior.
• Support invalidation-at-poll resume without executing stale optimized facts past the invalidation point.
• Pass focused deopt/OSR/exception/GC tests, normal/no-GIL differential runs, and relevant sanitizer gates.

Landed checkpoints

• #435: exact loop OSR entry, cycle-safe backedge copies, multi-iteration execution, budget/checkpoint accounting, and generation invalidation resume.
• #436: exact handler-aware recovery for throws, abrupt completions, all call/construct shapes, and interpreter-owned effects.
• #437: complete at 057a2ef/ae26011 with deopt-indexed frame/scratch maps, fail-closed active-native publication, live moving compaction, and normal/no-GIL/TSan relocation evidence.
• #438: complete through 48bfe60/b80e81f/3d78eaf with general reducible regions, fused nested loops, exact internal block checks, edge-owned copies, and per-header moving safepoints; irreducible two-entry control rejects before publication.
• #439: complete through 628b262/b6d1b62 with executable runtime effects and five-way native finally dispatch, exact nested propagation, fail-closed malformed completion handling, and focused/broad TSan evidence.

Completion checkpoint

Exact reconstruction, live moving-GC integration, runtime effects, and completion dispatch are complete. Recovery-only object locals avoid Number guards, occupy marked loop scratch slots, move during real optimized execution, and resume deterministically in single-mutator and no-GIL TSan gates. Native finally dispatch preserves exact state and consumes normal, throw, return, break, and continue completions once, including remaining outer catch/finally handlers.

The final #439 batch passes 7/7 focused full-root native-finally tests and the broad JIT gate at 164/164 under TSan, with zero failures, skips, or leaks. Exact cooperative-stop, expired-watchdog, and shell-timeout delivery after optimizer checkpoint exits is covered through 89fc073. Backend and broad differential completion remains #434.

The #431 dependency is complete at e3f411a. The optimizer now has immutable published SSA artifacts, guarded AArch64 execution, exact pre-accounting side exits, and one equal-cost two-way control shape. Unequal paths deliberately remain rejected because dynamic checkpoint/IP reconstruction belongs here.

The next implementation slice can now define frame-state/deoptimization metadata without conflating it with the completed optimizer-foundation contract.

Two #432 foundation checkpoints are now on main:

• d0f8aae publishes deterministic SSA frame states as artifact-owned recovery tables.
• 67e878a adds exact VM reconstruction and a three-way native outcome (miss, complete, deoptimized), so a partial side exit cannot fall through into baseline and restart executed work. Restart-only direct calls explicitly reject artifacts that can side exit.

Normal and TSan focused JIT pass 110/110; normal and TSan optimizer filters pass 21/21 with zero leaks. All acceptance boxes remain open pending generated exits, dynamic step/checkpoint state, handlers/exceptions, OSR, invalidation, and precise stack maps.

Landed the first generated partial optimizer exit in 3cbc26a. An unequal-cost guarded numeric branch now runs its common prefix natively, charges exactly that prefix, selects the successor's immutable frame state, reconstructs the activation, and resumes bytecode without restarting. Entry is declined before work if the prefix would cross the budget or a 1,024-step checkpoint.

Evidence: normal/TSan test-jit 111/111 and normal/TSan optimizer filters 23/23, including both branch directions, hot ordinary-call activation routing, identical baseline/deopt step totals, and checkpoint-boundary fallback. Acceptance stays open for handlers/exceptions, OSR, invalidation polling, general control, and precise stack maps.

Checkpoint ea669989 is on main.

• Added owned, deterministic edge-specific locals/operand-stack states to optimizer SSA.
• Preserved the exact incoming SSA values for every CFG edge after compaction.
• Added a loop regression proving the preheader and backedge have the same frame shape but distinct loop-carried values at one header.
• Corrected the optimizer documentation to describe the already-landed asymmetric side exit and the new loop-OSR boundary.

Focused zig build test-jit: 111/111 passed, zero leaks. Benchmarks were not rerun because this metadata-only slice does not affect benchmark execution.

Checkpoint bc8c997f is on main.

• Added immutable loop OSR entry/import metadata.
• Entry lookup now requires an exact IP, local count, operand-stack depth, handler depth, and accumulator value.
• Scratch preparation imports VM locals/stack into the target SSA block-argument slots transactionally; malformed metadata cannot partially mutate scratch.
• The optimizer derives one contract for the tested loop header and keeps backend publication disabled until that loop is executable.

Focused JIT verification: 112/112 passed, zero leaks. The exact OSR-contract filter also passed after the transactional hardening. Benchmark scores remain unchanged because no benchmark execution path changed.

Checkpoint 6fba2c4e is on main.

• Added the first executable AArch64 loop OSR region for a guarded numeric header with one Boolean branch.
• Hot backedges import the exact VM frame into SSA scratch, execute/account for the header, then reconstruct and resume the selected body or exit edge without replay.
• Entry-only paths reject OSR-only artifacts; stack/handler/accumulator mismatches, escaped frames, unsupported recoveries, and budget/checkpoint crossings remain in bytecode.
• Added compiler and end-to-end VM tests, including exact step-delta convergence and checkpoint fallback.
• Updated the concise README limitation and optimizer documentation.

Normal focused JIT: 113/113 passed, zero leaks. TSan focused JIT: 113/113 passed, zero leaks. Benchmarks were not rerun because the published benchmark workloads do not enter this optimizer OSR subset, so README scores remain unchanged.

Invalidation checkpoints 70e6cdae and 51bfef3e are on main.

• clear() immediately closes all later native-entry and OSR polls while execution leases retain mapping lifetime.
• Optimizer code rechecks an owner generation before its first operation, closing the race after the VM-side poll.
• Both rejection paths perform zero bytecode-step accounting and leave VM state untouched.
• Concurrent owner invalidation, loop OSR refusal, and generated-code generation mismatch are covered in normal and TSan focused tests.

This does not close the acceptance checkbox yet: future long-running native loop bodies still require resumable polls inside the generated loop. Benchmark scores remain unchanged because benchmark execution paths are unaffected.

Checkpoint 254415a1 is on main.

Every optimizer recovery point now publishes an immutable, deopt-indexed precise stack map. The executable subset's guarded/constructed values are primitive, so both frame and scratch movable-root masks are proven empty; compiler and artifact tests keep map indexes aligned with recovery records.

Focused JIT: 113/113 passed with zero leaks. The TSan optimizer loop filter also passed. Full pointer relocation remains open in child #437; loop bodies/internal polling moved to #435 and handler/exception deoptimization to #436. Benchmark scores are unchanged because execution paths are unaffected.

Child #435 advanced with c152d97a: the optimizer now executes and accounts one complete numeric loop body through OSR, explicitly copies header-to-body SSA arguments, and recovers updated loop-carried locals at the next header. Tier arbitration now guarantees OSR-only artifacts reach bytecode backedges instead of losing to baseline entry. Normal and TSan focused JIT passed 113/113 with zero leaks. Multi-iteration native backedges/internal polls remain open.

Child #435 advanced with e65880ca: supported numeric OSR loops now keep backedges native across multiple iterations and poll invalidation plus budget/checkpoint distance at every header. Poll exits reconstruct exact current header state without replay. Normal and TSan focused JIT passed 113/113 with zero leaks. #435 remains open for general parallel-copy cycles and broader/nested loop coverage.

Child #435 advanced with e9d0dac0: loop edges now use cycle-safe parallel-copy scheduling, and a swapped four-local backedge verifies exact exit/checkpoint reconstruction. Non-number OSR guard refusal is also explicit. Normal and TSan focused JIT passed 115/115 with zero leaks. #435 remains open for nested/broader loop control and deterministic active-loop invalidation-race evidence.

Child #435 checkpoint 9bf10349 keeps one equal-cost inner branch diamond native across loop OSR iterations, with checked path-cost proof, shared-latch SSA recovery, and exact checkpoint fallback. Normal and TSan loop-OSR filters passed 4/4. Active-loop invalidation-race evidence and broader unequal/deeper control remain.

Child #435 is acceptance-complete through 6852f5f2: a deterministic post-backedge generation race proves active native loops observe invalidation at the next header and reconstruct all completed work exactly. Full normal and TSan JIT passed 117/117. The invalidation-at-poll checkbox is now complete; broader reducible loops continue in #438, while #436/#437 retain exceptions/handlers and movable roots.

Child #436 started with a8d094a5: deoptimization records now own ordered catch/finally stacks, and VM reconstruction transactionally restores locals, operand stack, accumulator, handlers, and IP. Malformed handler metadata refuses without partial mutation. Optimizer lowering still emits empty handler slices, so #432 handler/exception acceptance remains open.

Child #436 checkpoint 321b1593 wires per-point handler slices through optimizer frame/edge states and published artifacts. Focused normal/TSan checks pass. Bytecode handler dataflow and exceptional edges are still absent, so #432 exception acceptance remains open.

Child #436 advanced with 1ee7e18a: active catch stacks now propagate through normal optimizer CFG state and survive real asymmetric side exits. Invalid merges, unbalanced pops, calls, and throws fail closed. Focused normal/TSan handler gates pass; explicit exception edges remain open.

#436 advanced in 3c5f856: optimized explicit-throw prefixes now preserve two-level catch/finally stacks and resume nested throw-through-finally in bytecode with exact step parity under normal and TSan tests. Unsupported handler-only blocks may remain bytecode after the mandatory side exit; any unsupported operation reachable before it still rejects. Remaining #432 exception work is tracked in #436.

#436 checkpoint 668bead adds exact return-through-finally deoptimization: the optimized primitive prefix restores the completion value and handler stack, then bytecode executes abrupt_return and every enclosing finally once. Focused normal and TSan differentials pass.

#436 checkpoint 96d74a1 models direct calls as exact pre-effect side exits. Function-valued callees and arguments are reconstructed without irrelevant Number guards; bytecode performs the call, preserving both normal results and caught call exceptions. Full focused JIT is 126/126 in normal and TSan modes; the call-specific TSan differential is 5/5.

Batch aa3854d advances #436 and #437 together: tail calls publish exact pre-effect recovery, while object-valued call inputs stay in rooted frame slots and receive deopt-indexed candidate root masks. Focused normal/TSan optimizer and end-to-end filters pass.

Batch f8c3a06 advances #436/#437 together: named property lookup now side-exits before observable effects with its receiver and active handlers reconstructed, and the receiver is covered by the exact deopt frame-root map. Focused normal/TSan filters pass.

e86a9978 completes the next #436/#437 batch: all current call/construct shapes and common property/coercion/brand effects now leave optimized code before observable work with exact rooted recovery inputs and handler state. End-to-end computed read/write recovery passes normal and TSan gates with repeat-run step parity.

Remaining #432 work is now concentrated in explicit exceptional CFG successors/native catch-finally behavior, collector exposure plus live movable-reference rewriting (#437), and broader reducible loop regions (#438).

Second batched checkpoint 40f52de3 advances both remaining safety axes: all explicit abrupt completions now reconstruct through finally in bytecode, and active native invocations expose validated frame/scratch maps to precise tracing and relocation. Focused normal and TSan gates are green for lowering, end-to-end labeled-break unwinding, and selected native-root movement.

The next precise-GC slice is generated publication of the current safepoint index plus an actual in-native moving checkpoint; explicit exceptional CFG successors and native catch/finally execution remain on the exception side.

612340d6 extends handler-aware, precisely rooted side exits across the numeric/string/property-key coercion and bitwise families. This reduces #436's remaining implicit-effect surface to environment, allocation, and iteration operations while preserving #437's canonical frame-root policy.

Authoritative gate update: exact main SHA 612340d6 passed CI run 29877847695 with 42 successful jobs, one expected skipped lane, and zero failures. #435 is now closed; broader loop CFG work remains isolated in #438.

Checkpoint 125349c, documented by a5c2c6a, completes the next batched terminal-effect slice: all emitted call/construct shapes, abrupt completions, and interpreter-owned environment/allocation/object/iteration/property/coercion/scope effects now reconstruct exact pre-opcode state. The issue body and #134 child map are current. Focused lowering is 3/3 normal plus TSan; focused VM recovery is 5/5 normal plus TSan. The broad matrix is delegated to the exact main CI run.

Checkpoint 561a516 advances #438 without weakening deoptimization boundaries: unequal inner-loop arms now retain exact per-path costs, while the header conservatively reserves the larger checkpoint/budget quantum and all polls recover the immutable header state. Both cost directions pass normal/TSan lowering and VM convergence tests. Exact main CI will provide the next broad gate.

Checkpoint 057a2ef/11e4b24 completes live relocation for the current optimizer loop region. The generated safepoint publishes the exact loop-header map after backedge copies, and a real compaction rewrites a recovery-only object scratch slot before native resume and bytecode property access. Normal and TSan safepoint batches are 9/9. Precise stack-map acceptance is now checked; no-GIL stress and broader exception/CFG work remain.

#436 advanced at 1b812e2/c3833eb: the optimizer now owns an explicit exceptional CFG successor for throw→catch and executes deterministic nested catch chains natively. Exact unwound stack/locals/outer handlers are edge state; uncaught propagation remains an exact side exit. The combined optimizer TSan gate is now 83/83. Remaining #432 exception work is native finally dispatch and call/effect exceptional successors in #436; no #432 checklist is being closed prematurely.

#436 checkpoint e3cb3bb/bd94239 adds explicit finally exceptional SSA edges and native deterministic finally-body execution. Pending [value, kind] completions recover exactly at end_finally; returns inside finally can override a throw entirely in native code. Focused finally evidence is 9/9 and the optimizer TSan gate is 85/85. Remaining #432 exception scope is dynamic native completion dispatch and call/effect exceptional successors; checklists remain open until the full matrix lands.

#436 checkpoint 0cb719d/80281cc completes the optimizer-model portion for calls/effects: all handler-active pre-effect exits own exact exceptional catch/finally destinations, unwind/target depths, and outer-handler state. Combined optimizer TSan evidence is 87/87. Remaining #432 exception work is executable call/effect lowering, native dynamic finally dispatch, and the final differential matrix.

#439 started at e66f4f4/5a227e3 with a backend-neutral runtime-operation descriptor and distinct value/exception/termination/watchdog/debugger/host/OOM/invalidation outcomes appended to NativeFrame. Existing offsets stay stable. Focused normal+TSan ABI evidence passes 3/3; descriptor ownership and executable lowering are next.

#439 checkpoint 6c1d86a/443f6f0 carries immutable native-operation descriptors end to end: exact staged operands, exceptional-target link, published artifact ownership, and destruction/invalidation cleanup. Optimizer TSan passes 87/87. VM callback dispatch and executable lowering remain next.

#439 checkpoint 2e401c0/2dcc3c9 installs the validated VM-side native-operation dispatcher for managed entry and OSR. to_numeric is the first implemented descriptor; malformed ids/ranges/opcodes fail closed. Focused normal+TSan dispatch passes 3/3. Generated invocation/root maps remain next.

#439 advanced through 749c2aa: in, instanceof, and private-brand in now execute as precisely rooted runtime operations with Boolean SSA results and exact exceptional continuations. Proxy has and custom Symbol.hasInstance witnesses prove observable callbacks execute once; private names are artifact-owned and malformed names/arity fail closed.

The batched optimizer gate passes 156/156 under TSan, with docs current at d7773bf. #432 remains open for the unfinished #439 completion-dispatch/effect work and final differential gates. Latest main CI: https://github.com/zig-utils/zig-js/actions/runs/29893818314

#439 advanced through 7ce8b63: nine unary/coercion bytecodes now produce executable rooted SSA values, void is a pure undefined value, throwing conversion resumes the exact catch once, and runtime operations can own deterministic common-prefix steps before a branch. Strict terminal proofs remain separate, so existing throw/finally recovery is unchanged.

The batched broad optimizer gate passes 157/157 under TSan, with docs current at 4f8ffa0. #432 remains open for unfinished #439 completion dispatch/effects and final differential gates. Latest main CI: https://github.com/zig-utils/zig-js/actions/runs/29894681886

#439 checkpoint: edae6d74/b169dbf5 make exponentiation and every bitwise/shift coercion executable inside precisely recoverable native exception regions. Both operands are stack-map roots, left-to-right coercion throws resume the owned catch target once without replay, and malformed arity fails closed. The combined optimizer TSan gate is now 158/158 with zero skips, failures, or leaks. Latest main CI: https://github.com/zig-utils/zig-js/actions/runs/29895064566

#439 checkpoint at bb28a32b/a5981afb: all thirteen arithmetic/relational/equality sites can now select precisely rooted runtime semantics from race-safe per-bytecode operand profiles while numeric sites retain their original inline Number guards. Dynamic left-to-right coercion resumes an owned catch target once without replay. Focused dynamic paths pass 6/6 normally and under TSan; the broad optimizer TSan gate passes 161/161 with zero skips, failures, or leaks. Latest main CI: https://github.com/zig-utils/zig-js/actions/runs/29895820403

Progress on #439: 3a32d9a executes all remaining non-tail invocation forms as native optimizer runtime values, including direct eval, named methods, plain/eval/receiver spread, and spread construction. It also fixes method-getter-before-arguments order and direct eval through spread. 979d541 updates the concise README and optimizer status. Focused invocation evidence passes 2/2; the batched broad JIT gate passes 162/162 under TSan with zero failures, skips, or leaks. Tail-call control remains a separate terminal-flow tranche.

Further #439 progress: fd687e3 executes all four proper-tail opcodes natively. Generated JS-to-JS tails replace the current heap activation, named tails fetch their getter once, direct-eval tails preserve scope, and every form owns exact roots and exceptional targets. 06c962f removes tail calls from the concise README missing list and updates optimizer docs. Focused evidence passes 2/2 and the broad TSan JIT gate remains 162/162.

Further #439 progress: ef1c0f8 executes all eleven object/array literal construction opcodes natively—allocation, property/prototype/spread/accessor initialization, append, holes, and iterable spread—with precise roots and exact exception targets. A throwing iterator proves partial mutation is retained and the effect is not replayed. 6aa623a updates concise README/docs. Focused construction evidence is 2/2; broad JIT TSan is 163/163.

#432 is complete now that its final executable-effect/completion child #439 has landed. The child chain #435–#439 covers exact OSR entry/exit, transactional deoptimization, handler-aware exception recovery, live moving-GC stack maps, reducible loop regions, runtime effects, and five-way native finally dispatch. Final evidence for the last batch: 164/164 broad JIT tests under TSan plus 7/7 focused full-root native-finally tests under TSan, with zero failures, skips, or leaks. Remaining architecture and broad differential work stays in #434; IC/artifact lifetime work stays in #433.