VM: interleave captured runtime environments with direct-eval activation views

Parent: #771 Related: #776

Problem

A VM closure created inside a live block, class, loop-head, disposal, or with Environment Record captures that runtime record through Function.closure while its statically resolved outer function bindings remain in Frame slots. Direct eval needs the exact lexical order current frame -> captured runtime records -> parent frame. The current DirectEvalPlan rejects any nonzero parent_environment_depth because materialization can only place every activation view on one side of Function.closure.

This is the final explicit runtime-Environment boundary in #771. Admission must stay fail-closed until the engine can represent the interleaving without changing published Environment.parent links or copying binding values.

Scope

• Retain each ordinary Frame defining closure Environment as a precise, relocating edge so every child-to-parent frame boundary can recover its captured runtime segment.
• Extend immutable direct-eval plans with validated boundary metadata for the exact number and ordering of intervening Environment Records, including named-function-expression records.
• Materialize identity-preserving forwarding records whose own-binding operations target the original declarative or object Environment Record while their parent links interleave activation views in spec order.
• Preserve TDZ, const/function-name state, deletability, aliases, with HasBinding and Symbol.unscopables observability, catch Annex B rules, per-iteration binding identity, and closures produced by eval.
• Keep publication transactional under deterministic OOM, exact under moving GC, and race-free for escaped frames and shared no-GIL execution.

No-workaround rules

• No binding-value snapshots, source rewriting, published-parent mutation, global fallback, tree-walker callback, broad admission, or eager materialization for calls that do not resolve to the realm eval intrinsic.
• No stacks, queues, jobs, Promise/microtask, Worker lifecycle, or Map/Set iterator implementation changes.

Acceptance

• Required bytecode and forced tree-walker agree for direct eval inside nested block/class/catch/loop and with scopes, including shadowing, TDZ, mutation, deletion, named function-expression self binding, and eval-created escaping closures.
• Runtime records remain exact live identities: per-iteration captures stay distinct, with traps and Symbol.unscopables run in the specified order, and later initialization or mutation is visible through both original and interleaved chains.
• Frame closure edges and forwarding-record target edges trace and relocate precisely; deterministic allocation-failure sweeps publish no partial frame cache or vm.env chain.
• Shared-realm no-GIL and TSan witnesses show no parent-edge or binding-table race, including concurrent first materialization and escaped defining frames.
• The parent_environment_depth and current environment_depth barriers are removed only for fully validated plans; malformed or unsupported boundaries fail closed without fallback.
• Focused compiler/VM/frontend, relevant exact Test262 eval/function/with/class accounting, full units, threadfuzz, debugger, watchdog, moving-GC, TSan, and no-GIL gates pass with zero regressions.
• A conventional commit lands directly on main and cross-links #771 and #776; exact Test262 flips are recorded without an unmeasured performance claim.

Completed integration

CI run 33113309062 completed successfully on main commit 8a1398ffc04e9e30987f968693667c7271ca3ce1, a descendant containing this implementation. All 53 executed jobs succeeded (unit and TSan-unit shards, Linux no-GIL TSan, functional/no-GIL corpus gates, fuzz profiles, ABI, Wasm, Test262 parallel, and docs); the scheduled-only nightly job was skipped. Together with the measured local evidence in the comments, this completes this issue's acceptance.

The independently scoped provisional root-registration/preparation defect remains open in #787. This closure covers registered caller-root relocation and exact eval environment identity, not that outstanding activation-driver ownership work. The exposed capture-metadata race was fixed and verified in #789.

The precise Frame anchor layer landed in d2b4c494. Every running ordinary activation now retains its exact defining Environment as an immutable Frame edge; captured-function, suspended-generator, and active-interpreter roots trace and relocate it, and recyclable activations clear it before reuse. The relocation witnesses cover all three root categories. Formatter and AST checks pass; compiler admission and evaluation semantics remain unchanged, so this layer flips 0 Test262 cases. The issue remains open for boundary-plan metadata and identity-preserving interleaving.

Immutable boundary metadata landed in 62f9385b. DirectEvalPlan now records the exact runtime Environment count at each child-to-parent Frame boundary, including current call-site records and named-function-expression self-name records. The existing causal barrier remains: any nonzero boundary still rejects compilation until materialization consumes it exactly. Focused planner tests pass 2/2; adjacent Frame relocation categories pass 3/3, with zero failures/leaks. This layer flips 0 Test262 cases because the newly represented cases remain fail-closed.

The identity-preserving forwarding layer landed in 2b46afb0. A forwarding Environment owns only its replacement parent edge; all declarative/object own-binding queries, reads, writes, tags, aliases, References, with operations, conflict checks, and deletion target the original live record. The target edge traces/relocates precisely, reuse clears it, and quick binding caches reject the synthetic chain in favor of the exact ordinary VM path. Environment clusters pass 13/13 normally and 13/13 under TSan; focused forwarding witnesses pass 2/2 in both modes, with zero failures/leaks/sanitizer reports. Admission remains fail-closed, so this layer flips 0 Test262 cases. Next is transactional materializer integration.

Captured-boundary admission landed in 6e212e82.

The materializer validates each exact Frame closure segment, then places identity-preserving forwarding records at each lexical scope's runtime depth. This preserves block/catch shadowing inside nested with records instead of flattening all frame scopes to one side. Environment-backed cells are excluded from slot views; original captured parent links are never changed. Target pointers are reloaded from precise roots after allocation, and the complete candidate chain is allocated before cache publication.

Measured coverage:

• Required bytecode versus forced tree-walker: captured with and loop cells, named-function self binding, block/catch shadowing, repeated eval, two captured Frame boundaries, and eval-created escaping closures through compaction.
• Synthetic cached-parent/current reuse, malformed boundary/depth rejection, and deterministic OOM sweeps leave no partial caches or changed original parents.
• Four no-GIL workers materialize against a shared escaped defining frame.
• Direct-eval cluster: 32/32 normal and 32/32 TSan; final expanded interleaving witnesses: 3/3 TSan. The rebased landed tree also passes 32/32 normal. Zero failures/leaks/sanitizer reports in these runs; formatter, AST, and diff checks pass.

Exact ReleaseFast Test262 accounting, baseline 2b46afb -> landed 6e212e8:

• eval-code: 347 -> 347 pass
• function-code: 217 -> 217
• expressions/function: 264 -> 264
• statements/function: 451 -> 451
• statements/with: 181 -> 181
• expressions/class: 4059 -> 4059
• statements/class: 4367 -> 4367

Total: 9,886 -> 9,886 passing, 0 failures in either run, 0 flips and 0 regressions across those exact subtrees. No performance claim.

This remains open: current-frame runtime eval still fails closed at environment_depth, because returning from eval must preserve the VM's active runtime chain for block/with/loop unwind. Full required gates and the rest of the acceptance matrix are not yet claimed. Continues #771 and #776.

The current-frame implementation now passes required-bytecode/tree-walker scope, abrupt-with, and no-GIL witnesses. Exact ReleaseFast accounting remains 9,886/9,886 across the same seven eval/function/with/class subtrees, with 0 flips. A new test that requests a moving nursery safepoint from inside eval exposed #786: nested VM calls restore an unrooted saved caller Environment. That root-ownership defect is being fixed before current-frame admission lands; no compaction or execution-mode workaround is being used.

Landed fix(vm): preserve runtime eval scopes and caller roots (#785, #786) on main.

Current-frame runtime records now interleave with activation views in exact lexical order. Semantic eval and runtime-unwind projections forward to original binding identities; handler prefixes preserve eval-created variables across abrupt completion. Direct-eval-only loop captures, simple-catch Annex B rules, class TDZ, with/unscopables, nested captured/current boundaries, labeled jumps, and generator suspension have forced-tree-walker/required-bytecode witnesses. Publication/OOM and actual moving-safepoint witnesses pass without fallback.

Final-source validation: full units 1988 passed, 1 platform skip, 0 failures/leaks; focused TSan direct eval 36/36 and root relocation 1/1 with zero reports; threadfuzz default 400 and exact-result verify 300 seeded programs passed. Focused eval/compiler/with/debugger/activation/termination and shell-timeout probes passed.

Exact ReleaseFast parent 6e212e8 -> candidate Test262: eval-code 347 -> 347, function-code 217 -> 217, expressions/function 264 -> 264, statements/function 451 -> 451, statements/with 181 -> 181, expressions/class 4059 -> 4059, statements/class 4367 -> 4367. Total 9886 -> 9886 passed, zero failures, flips, or regressions in these subtrees. No performance claim.

The nested moving witness exposed #786, fixed in the same commit. Pushed required integration remains pending; the separate provisional-root publication/lifecycle audit remains #787. Keeping this issue open while those boundaries are tracked.

Pushed Linux integration found an engine-state race in the new current-runtime direct-eval witness: tsan-parallel-js job. Environment.markCaptured writes private_activation while a peer lockBindingsForRead reads it. The job's functional tests are 60 pass / 8 skip / 0 fail / 0 leak, but one TSan warning causes exit 66.

Tracked as #789. Local focused TSan did not expose this interleaving; this issue's integration gate remains open. The fix will atomically publish/read capture and private-ownership state, preserving the existing lock-elision proof and captured-frame identity rather than suppressing the report.

Landed fix(threads): publish environment capture state atomically (#789) on main. This addresses the Linux TSan read/write race in the direct-eval forwarding/capture path: both capture/private flags now have atomic publication/read semantics, and escaped-frame marking precedes captured publication. No suppression or disabled reuse was introduced.

Local evidence: Environment 14/14 and direct eval 36/36 in both normal and TSan builds (zero reports); full normal units 1,993 pass / 1 skip / 0 fail / 0 leak; 2,711 Test262 passes unchanged across eight relevant subtrees; seed-1 default/verify threadfuzz 400/300 programs with zero failures. #789 retains the outstanding pushed Linux required-integration gate; these local results do not close the parent integration requirement.

Linux confirmation for #789: tsan-parallel-js on cfef054f passed, including the original current-runtime direct-eval regression: 60 pass / 8 platform skips / 0 fail / 0 leak and no TSan report. The broader workflow is still running, so this confirms the specific race gate rather than full integration.