ReviewOS

zig-utils/zig-js

Optimizer: preserve exact checkpoints after runtime re-entry

#1097
Closed chris opened this 9 days ago ·
9 days ago

Parent: #467; blocks the budget/termination acceptance of #1096. Related #466 and #491.

Executed against engine source archived from main commit 2eeae7a and against the #1096 draft. This is a pre-existing native callback checkpoint defect, exposed again by broader environment admission.

A warmed, fully native function budgetWrite(rhs) { var result = rhs(); return result; } calls an RHS that increments a global and returns 7. Drive the same function through required bytecode with optimizer disabled/enabled, starting the interpreter counter at each step from 992 through 1023. With a 1024-step budget, main's optimizer returns 7 where bytecode throws RangeError at step 1025. With cooperative stop requested, it returns where bytecode stops at step 1024. The exact-parent control has 8 mismatches out of 64 cells. One example: start 1010, budget 1024, bytecode throws at 1025; native returns 7 at 1025.

The retained-with writer in #1096 has 12 mismatches out of 64 cells. At start 1011, budget 1024, bytecode throws at 1025 before assignment; native reaches 1029 and writes the property. This is an externally visible effect difference, not just accounting.

The entry reservation excludes dispatch consumed by runtime re-entry. After a successful callback, generated code continues using that earlier reservation and may skip the next exact watchdog/cooperative checkpoint.

Definition of done:

  • Successful native callbacks publish an exact post-operation continuation, including locals, operand stack, handlers, environments/References, result/base projections and roots; no callback or mutation is replayed.
  • Continuation revalidates the remaining native budget/checkpoint window after re-entry and keeps cached loop budget counters consistent with the live interpreter counter.
  • The 64-cell exact-parent callback control and 64-cell retained-with control match forced bytecode in result/exception, dispatch count and visible effects; extend to branch/loop, accessor and caught-callback cases.
  • Recovery/verifier/OOM, moving-GC, shared-realm and unsuppressed TSan qualification, affected exact-parent conformance and production gates pass; land a small conventional fix on main with evidence.
9 days ago

The exact-parent control has been extended to six caller shapes: ordinary calls, both branch arms, getters, native loop regions, and caught callbacks. It compares 384 optimizer-off/on cells in result/exception, final dispatch count, and visible callback effects.

Main at 2eeae7a has 83 mismatches; the current fix has zero. Successful callbacks now retain a post-operation recovery state and revalidate the residual native window when child dispatch changes the live counter. A boundary miss resumes at the next bytecode, preserving the completed effect. Loop countdowns are refreshed without counting the consumed block prefix twice.

The full optimizer integration cohort is running. ReleaseSafe/TSan, GC/shared-realm stress, exact-parent corpus and production gates remain pending; nothing has landed yet.

9 days ago

Performance review found that the new continuation-exit flag excluded previously admitted direct calls. Executed unary/property controls had 1,000 direct native hits on main and zero with the initial fix.

The updated implementation restores 1,000 hits in both controls through a general read-only entry proof. It resolves each operation's inputs from existing recovery metadata before execution and applies the canonical primitive/data-only guards. Mutations, calls and scratch-dependent inputs retain ordinary activations. Per-operation guards also protect stale read-only assumptions; a refused pure prefix does not charge dispatch twice.

The pinned controls include strict/sloppy unary calls, own-data reads, and rejection of getters, object coercions and writes before effects. The 384-cell callback boundary matrix still has zero mismatches. The updated full Debug cohort passes all 209 tests; ReleaseSafe and unsuppressed TSan are being rerun. Production, corpus and timing controls will be refreshed for this final source before landing.

9 days ago

The canonical checkpoint fix landed on main in ed872e60.

  • Six-shape exact-parent boundary control: 83/384 mismatches → 0/384, comparing result/exception, dispatch count and visible effects.
  • All 209 optimizer tests pass in Debug, ReleaseSafe and unsuppressed TSan. Instrumentation was checked in the linked artifact.
  • Completed-result relocation changes the object address and preserves its value in serialized and no-GIL modes; four shared-realm workers pass.
  • Read-only entry controls recover 1,000 direct hits for unary/property reads. Getters, object coercions and writes are refused before direct effects or dispatch charges.
  • Production JIT: 149 pass, one platform skip. VM/concurrency: four cases each. Frontend: 79. Library, relocation inventory, documentation links and checked 51-page build pass.
  • Exact parent: all 4,582 test262 cases have identical failing/diverging paths. Default failures remain zero; the six existing TCO tree/VM distinctions are unchanged. Flips 0 cases.
  • Threadfuzz: 400 default and 300 broad programs, zero failures.

The timing controls remain development evidence and show residual checkpoint/preflight cost; this commit makes no throughput claim. Hosted CI is queued in run 38037582489.

This issue remains open until #1096's retained-with writer and WithBaseObject projection controls are integrated and verified. The original #1096 draft is preserved; its merge preview includes the required receiver-publication ordering and logical-diamond residual windows.

9 days ago

The retained-Reference and result/base-projection integration is now complete in 136178ae, following ed872e60.

The original six-shape callback matrix is 83/384 → 0/384. The retained-with writer is 12/64 → 0/64, and the separate callable-getter/WithBaseObject matrix matches all 64 budget/stop cells in result/error, receiver, side-effect counts and steps. All 225 combined optimizer tests pass in Debug, ReleaseSafe and unsuppressed TSan, including actual completed-result/Reference/native-loop relocation and shared workers.

Exact-parent 4,582-case path comparison, production gates, 400 default + 300 broad fuzz programs, relocation inventory and checked docs pass. The detailed integration evidence is on #1096. Hosted CI remains pending; no claim of complete hosted integration or throughput parity is made.

Sign in to comment on this issue.