ReviewOS

zig-utils/zig-js

VM: resolve ordinary identifier references through with environments

#732
Closed chris opened this 9 days ago ·
9 days ago

Parents: #465, #493 Related: #731

Problem

Plain frame-mode functions can lower a with statement, but ordinary identifier reads/calls and simple/compound/update assignments still select load_local/load_upval/store_local statically. A captured Object Environment Record must participate before the frame boundary, including observable Proxy has traps and Symbol.unscopables; calls also require the exact WithBaseObject for this. The existing comment that any function using with falls back is not true.

#731 adds activation-owned identifier References for destructuring and simple var initialization only. Reusing that primitive for the general expression surface is a separate complete lowering.

Scope

  • Resolve ordinary IdentifierReference reads and calls through activation-local and captured with Environment Records before static local/upvalue/global fallback.
  • Preserve WithBaseObject call this, direct-eval identity, typeof, strict/sloppy assignment, compound/logical assignment, prefix/postfix update, and evaluation order.
  • Bound environment walks at exact frame/upvalue boundaries and retain direct bytecodes when no dynamic record can intervene.
  • Extend native-tier admission only when guards/fallback preserve the same resolution contract.
  • Add nested-function/captured-with, Proxy/@@unscopables, recursion, suspension, moving/concurrent GC, and no-GIL witnesses.

No-workaround rules

No delayed re-resolution, duplicated HasBinding, source rewriting, synthesized globals, heuristic syntax refusal presented as implementation, or unsafe raw Environment/Object pointers. No stacks, queues, jobs, Promise, microtask, Worker lifecycle, or Map/Set iterator changes.

Acceptance

  • Tree-walker and required bytecode agree on exact observable lookup/GetValue/PutValue/call order and destinations.
  • Static no-with functions retain their current direct bytecodes and native-tier eligibility.
  • Focused compiler/frontend/VM, full units, TSan/no-GIL, and relevant nonzero Test262 coverage pass with exact accounting.
  • Landing commit cross-links #465, #493, and #731.
9 days ago

Initial lowering audit is complete.

The existing static-slot divergence spans five distinct contracts:

  • value reads and typeof need bounded dynamic resolution before local/upvalue fallback;
  • simple assignment must capture before the RHS;
  • compound, logical, and update expressions must retain one Reference across GetValue, coercion/RHS/short-circuit, and PutValue;
  • ordinary/optional/tail/tagged identifier calls need an explicit WithBaseObject receiver without repeating HasBinding;
  • direct eval must remain identity-based when the syntactic callee is eval, including a with-provided non-intrinsic callee receiving the correct this.

I am separating compile-time with depth from ordinary declarative-environment depth, including parent-frame crossings. That keeps no-with local/upvalue reads and stores on their existing direct bytecodes while using the #731 activation-owned Reference slots only where an Object Environment Record can actually intervene. The runtime design adds exact GetValue plus explicit consume/retain/clear modes so short-circuited logical assignments do not retain stale roots. Tree-walker identifier assignment/RMW paths will use the same captured-Reference contract rather than delayed re-resolution.

9 days ago

Implementation and validation update:

  • ordinary reads, calls/tags with WithBaseObject, direct-eval identity, typeof, simple/compound/logical/update assignment, nested closures, recursion, and suspended VM activations now retain one exact binding Reference
  • parser logical assignment now keeps one Reference for identifier targets instead of desugaring into two observable resolutions
  • dynamic References are activation/tree roots traced and relocated by precise GC; no-with slot loads/stores retain direct bytecodes and native eligibility
  • focused differential: green in tree-walker, required VM, and required parallel/no-GIL modes across Proxy, Symbol.unscopables, strict/sloppy deletion, recursion, suspension, and forced GC
  • focused TSan: 1 passed, 0 failed/leaked/races
  • optimizer regression set: 153 passed
  • full unit suite: 1,864 passed, 1 platform skip, 0 failed, 0 leaked
  • Test262 statements/with: 180/181 -> 181/181; fixed set-mutable-binding-binding-deleted-with-typed-array-in-proto-chain-strict-mode.js
  • Test262 logical-assignment: 78/78 -> 78/78
  • Test262 compound-assignment: 454/454 -> 454/454

I am doing the final diff/invariant review before landing.

9 days ago

Landed on main as feat(vm): resolve identifiers through with (#732).

The commit body records the exact unit, TSan, optimizer, and Test262 evidence. CI run: https://github.com/zig-utils/zig-js/actions/runs/32680496969

9 days ago

Post-landing CI follow-up: the pushed candidate repeated both GIL corpus failures on a failed-job rerun: sync/atomics-futex-lock.js returned not-equal in round 5, and api/condition-async-wait.js reached 4/5 completions. The exact first parent shard-3 control was rerun under current CI conditions and passed, so I am treating this as a #732 regression rather than dismissing it as scheduling noise.

The scoped correction under validation keeps retained References for assignment/update/destructuring, but makes immediate GetValue resolution a single environment-table walk that returns the exact WithBaseObject from that same walk. It changes no Atomics, Condition, queue/job, or corpus code. Local build/corpus validation is waiting for the user-owned Home suite to release the host.

9 days ago

Follow-up landed on main: 441565b4 (fix(interpreter): preserve identifier checkpoints (#732)).

Root cause: specialized bare-call/tag/direct-eval Reference resolution bypassed the AST evaluator checkpoint that owns debugger service, step accounting, VM traps, cooperative GIL yield, and GC safepoints. That made the pushed candidate repeatedly lose arm 2 of condition-async-wait (4/5 completions) and changed the futex wake schedule. The exact parent control passed under the same CI conditions. The fix factors that checkpoint and calls it from every specialized identifier path; immediate GetValue also uses one exact environment walk, while retained write References remain two-phase.

Final validation:

  • authoritative condition-async-wait and atomics-futex-lock: 5/5 each;
  • full warmed GIL shards: 63/63 and 61/61;
  • focused semantic TSan plus both TSan corpus cases: green, zero failure/leak diagnostics;
  • full units: 1,864 pass / 1 platform skip / 0 fail / 0 leak;
  • Test262: with 181/181, logical assignment 78/78, compound assignment 454/454; zero flips.

No Condition, Atomics, queue/job, or corpus source changed.

9 days ago

Follow-up CI run 32693391909 completed the regression-sensitive gates before the subsequent docs-only #733 push superseded the remaining matrix: all four threads-gil shards passed, including the original failing shards 2 and 3; all four ordinary unit shards, all four TSan unit shards, and focused-engine-tests also passed. The descendant #733 run contains the same #732 engine code and is now the current main matrix.

Sign in to comment on this issue.