ReviewOS

zig-utils/zig-js

VM: model direct eval across parameter and body environments

#776
Closed chris opened this 9 days ago ·
9 days ago

Parent: #771 Related: #465, #498

Problem

A function with parameter expressions has an outer ParameterEnvironment and a distinct body VariableEnvironment. The ordinary VM currently merges their static names into one frame namespace, so #771 correctly keeps every default-bearing direct-eval function fail-closed.

This is not only a wrapper-order problem. A body direct eval may dynamically introduce a var binding whose name matches an outer parameter. That new body binding must shadow the parameter for later compiled body reads, while closures created during parameter evaluation retain the outer parameter binding. Direct eval inside a parameter default runs before the body VariableEnvironment exists and has separate arguments and declaration restrictions.

Scope

  • Represent parameter-binding and body-var binding identity separately while retaining exact activation storage and direct slot access where semantics permit it.
  • Materialize the ParameterEnvironment outside the body VariableEnvironment in the exact order required by FunctionDeclarationInstantiation.
  • Make dynamic eval var/function declarations visible to later body execution without snapshots, effect replay, or global fallback.
  • Preserve earlier-parameter visibility, own-parameter TDZ, unmapped arguments, named function self, this, super, new.target, private names, destructuring/default order, and body var/function initialization.
  • Cover body-only eval, parameter-phase eval, fixed/spread/tail calls, nested arrows, escaping closures, recursion/reentrancy, moving GC, deterministic OOM, debugger/watchdog state, and shared no-GIL execution.

No-workaround rules

  • No flattened parameter/body record, speculative eager body binding for every parameter, local-value snapshot, source scan of eval strings, rewriting, globalizing declarations, eager heap allocation of every local, AST callback from required bytecode, or hidden tree-walker fallback.
  • No stacks, queues, jobs, Promise/microtask, Worker lifecycle, or Map/Set iterator implementation changes.

Acceptance

  • Tree-walker and required bytecode agree for body eval reads/writes/declarations that shadow or do not shadow parameters and body vars.
  • Parameter-default eval preserves TDZ, earlier/later parameter visibility, arguments restrictions, declaration targets, errors, and side-effect order.
  • Parameter-created and body/eval-created closures retain the correct distinct binding cells across mutation, recursion, and moving GC.
  • Activation views remain bounded, failure-atomic under injected OOM, race-free under TSan/no-GIL, and released at teardown.
  • Focused compiler/VM/frontend, full units, exact relevant Test262 accounting, debugger/watchdog, moving-GC, TSan, no-GIL, and threadfuzz gates pass with zero regressions.
  • Conventional commits land directly on main and cross-link #771/#465/#498 with exact measured evidence and no unsupported performance claim.
9 days ago

Implementation progress: the tree-walker now predeclares every bound name in a non-simple parameter list to its live TDZ before left-to-right IteratorBindingInitialization. The focused ordinary/arrow/direct-eval/destructuring witness passes in the normal build and under TSan (1/1 each, no leaks or sanitizer report). Exact conformance scope is 32 tracked *dflt-params-ref-later.js cases; the before/after run is queued until an unrelated Home Zig build releases the machine, so no flip count is claimed yet.

9 days ago

First slice landed in d814995a: non-simple parameter bindings are all created in TDZ before left-to-right initialization, so defaults, destructuring initializers, iterator hooks, and direct eval cannot fall through to an outer later-name binding. Focused adversarial unit: 1/1 normal and 1/1 TSan, no leaks/sanitizer reports. Exact Test262 *dflt-params-ref-later.js: 32/32 before -> 32/32 after, 0 flips. The issue stays open for the VM ParameterEnvironment/body VariableEnvironment boundary.

9 days ago

Body-phase ParameterEnvironment/VariableEnvironment support landed in commit 1f594e7b. The VM now keeps parameter and body binding identities distinct, performs spec-ordered parameter-to-body initialization, lazily publishes kind-tagged live activation records through separate atomic GC roots, and resolves later compiled parameter reads/writes through only the exact defining-frame body record.

Evidence: focused direct-eval 25/25; body-boundary differential 1/1 normal and 1/1 TSan; two-record failure-atomic OOM 1/1 normal and 1/1 TSan; moving compaction preserves the escaping body closure; full units 1,961 passed / 1 skipped / 0 failed / 0 leaked. Exact scored Test262 test/language scope-paramsbody-var-*: 10/10 before -> 10/10 after, 0 flips, 0 regressions.

This issue stays open. Parameter-phase direct eval plus dedicated shared no-GIL/threadfuzz coverage remain, and this commit makes no performance claim.

9 days ago

Parameter-phase direct eval support landed in commit f49add08.

The VM now keeps raw call inputs separate from live TDZ-checked formal cells, initializes formals left-to-right, and eagerly materializes one failure-atomic ParameterEnvironment before initializer bytecode only when a compiled parameter eval plan requires it. Parameter-created closures retain that exact record; later eval declarations update the same live activation identity. Nested arrows now retain the parent phase they were created in, so parameter-created closures cannot statically capture the not-yet-created body VariableEnvironment. Parameter/body eval use is tracked independently, and eval var/arguments early errors use exact formal BoundNames rather than every dynamic binding in the record.

Evidence on the landed tree:

  • exact scored Test262 scope-param*/eval-var set: 21/21 parent -> 21/21 candidate, 0 flips, 0 regressions;
  • exact required-bytecode Test262 witnesses: 15/15 pass beside their tree-walker controls across function declarations, function expressions, and arrows;
  • focused parameter differential: normal + TSan green, including TDZ, repeated eval vars, arguments/formal conflicts, recursion, shadowed eval, escaping closures, moving GC, and parameter/body shadowing;
  • injected entry OOM: normal + TSan green with caller state and activation-pool restoration;
  • shared no-GIL required-bytecode witness: normal + TSan green across four concurrent workers repeatedly invoking the same compiled function;
  • full units: 1,964 passed, 1 skipped, 0 failed, 0 leaked;
  • threadfuzz default: 400 programs from seed 1, 0 failures.

The issue stays open. The six scope-param-rest-elem Test262 variants still hit the existing explicit mixed/rest-pattern parameter-prologue barrier in required-bytecode mode; they pass in scored/tree-walker execution but are not counted among the 15 required-tier witnesses. This commit makes no performance claim.

9 days ago

The final ordered-rest slice landed in commit 04913596. Mixed/default/destructuring rest arrays are now created at their exact left-to-right parameter-entry point; pending call tails are precise moving-GC roots, completed arrays are published through synchronized frame slots, activation storage is bounded/reused, injected OOM restores the caller, and unsupported native lowering remains fail-closed.

Final evidence: exact scored scope-param-rest-elem-var set remains 10/10 with 0 semantic flips; all six ordinary function/function-expression/arrow required-bytecode witnesses now pass beside tree-walker controls; targeted TSan is 6/6 including the shared no-GIL witness; full units are 1,967 passed / 1 skipped / 0 failed / 0 leaked; threadfuzz default is 400 programs from seed 1 / 0 failures. No performance claim.

Together with the three preceding linked commits, every acceptance item is covered, so I am closing #776.

Sign in to comment on this issue.