Test: fuzz exact semantics across every execution tier and backend

Parent: #459 Depends on: #465–#490 as their surfaces land

Goal

Continuously prove that interpreter, VM, baseline JIT, optimizer, and Wasm native tiers are semantically identical across optimization plans, GC timing, no-GIL schedules, and declared architectures.

Scope

• Build owned seeded generators for JavaScript source/bytecode shapes, Wasm modules/functions, objects/arrays/typed data, closures/environments, exceptions/finally, async/generators, modules, GC roots, atomics, and host callbacks.
• Execute the same case with tree-walker, VM, baseline, optimizer, Wasm interpreter/baseline/optimizer, GC modes, serialized/no-GIL, and applicable backend feature sets forced on/off.
• Compare result bits/types, exception name/message/metadata, observable mutations/order, output, steps/watchdog, jobs/microtasks, tier counters, GC relocation, memory bytes, and teardown.
• Randomize tier thresholds, invalidation/deopt timing, collection/safepoints, allocation failure, thread schedule points, CPU features, and code-cache pressure deterministically.
• Shrink failures to one reproducible seed/source/module and retain corpus regressions.

No-workaround rules

• No permanent suppressions, flaky retry-until-pass, result-only comparison, unsupported path counted as a tier success, or seed blacklist without a terminal documented semantic disposition.
• The generator and shrinker remain owned code with no external fuzz runtime dependency.

Acceptance

• Every new native/optimizer opcode, IR operation, backend, recovery outcome, and runtime fast path adds generator coverage before release.
• Normal, ReleaseSafe, TSan, moving-GC, no-GIL, OOM/fault, debugger/termination, and lifecycle profiles have bounded deterministic CI witnesses.
• Longer scheduled runs publish seed/rate/coverage inventories and retain every actionable minimized failure.
• Cross-architecture traces can be compared without claiming emulated performance.
• Zero unexplained skips/suppressions and exact denominator sanity checks are enforced.

Foundation

Existing threadfuzz, seeded optimizer differentials, test262, PR-249, Wasm corpus inventories, and forced enable_jit controls.

Forced-tier differential infrastructure landed on main in cff13e9f.

Context.TestingOptions.bytecode_execution_mode now provides two explicit harness contracts: tree_walker suppresses plain synchronous bytecode, while required throws InternalError when a program or called plain function lacks compiled bytecode. Required mode therefore cannot count a silent fallback as VM coverage. Production contexts remain automatic.

Initial exact witnesses execute identical closure/property/array/loop source in both modes and compare result bits plus externally visible state. A second pair compares unhandled exception class/message and the mutation committed before the throw. Admission counters prove the intended tier in each context, and a parameter-prologue rejection fixture proves unsupported lowering fails closed.

Validation: focused Debug and TSan; full sharded unit suite 1,474/1,474 with 0 leaks; BunPress manifest/build and 89-file docs link gate. Flips 0 test262 cases.

This is the execution-control foundation, not completion of the issue: seeded source/module generation, shrinking, GC/schedule/fault randomization, and broader tier/backend matrices remain open.

The forced-tier matrix expanded on main in 28b00703.

Tree-walker vs required-bytecode execution now compares exact result bits and observable traces for catch/finally, class construction, private method branding, and object-method super, including a second super lookup after a nested method activation returns. Admission counters continue to prove both paths, and derived/field-initializing constructors fail closed with an exact reason rather than masquerading as VM coverage.

The exact-parent test262 differential was used as an active bug finder: it exposed 528 missing-private-brand failures and 2 unresolved computed-private-key failures during development. Both were fixed before landing; final class statements are 4,367/4,367 and object expressions 1,170/1,170 in baseline and candidate, with zero gains or regressions.

This adds deterministic scope/activation coverage to the foundation. Seeded generation, shrinking, fault/schedule randomization, and the broader native/Wasm backend matrix remain open.

CLI hardening child #620 is complete in 34f5c8c6. Unknown, incomplete, malformed, and trailing fixed-arity invocations now fail before corpus work; --help is side-effect-free; and the final clean valid-path smoke passed 38/38 with no surviving runner.

The scheduler-sensitive cooperative-GC witness tracked in #673 is now deterministic on main via fec063f8. It passed 20 focused Debug repetitions, three eight-shard placements, ReleaseSafe, TSan, and the one-shot full unit inventory with zero failures or leaks.

#708 / 1f925020 adds a forced-tree-walker versus required-bytecode differential for named/computed member updates, plus exact accessor/proxy/coercion/BigInt/Symbol/strict/abrupt/recursion/suspension witnesses. Relevant Test262 was 142/142 before → after; focused TSan passed 3/3.

#709 / ee0e0520 adds forced tree-walker↔required-bytecode tagged-template differentials plus exact base/key/get/substitution/call abrupt-order, recursion, template-cache/freeze, super receiver, generator/async suspension, and tail-opcode witnesses. Test262 tagged-template transcripts remained byte-identical at 27/27.

Added exact #710 tier witnesses in e818a068: forced tree-walker vs compiled-bytecode semantics, emitted tail_call_with_this identity, abrupt key/getter/argument/call order, generator suspension/spread, and 2,000-step frame reuse. Relevant Test262 stayed 102/102; focused TSan and full units are green.

Related #711 landed in d1c8251b — feat(compiler): lower spread calls and constructors (#711): ordinary program/plain/generator/async spread calls and constructors now use the existing variadic bytecode family with exact receiver and evaluation ordering. Exact call/new/super Test262 remained 245/245 with byte-identical diagnostics and 0 regressions.

Related optional/member call lowering landed via #712 in ef067c10, including one-shot lookup, receiver identity, spread, indirect eval?.(...), suspension, and strict tail positions.

#713 will add forced-tier differential and exact delete/optional-chaining corpus coverage for the currently uncompiled delete_expr surface.

Tier-differential coverage expanded in #713 / 7a1932cf: forced tree-walker vs required-bytecode delete semantics now cover primitives, arrays/strings, Proxy revocation/invariants, Symbols, coercion ordering, strict/sloppy behavior, native fallback, suspension, and confirmed moving-GC relocation. Focused TSan was 3/3 with zero warnings; exact Test262 regressions 0.

New opcode coverage from completed child #714: load_import_meta now has forced tree/VM/automatic fallback differentials, exact cross-module identity, generator+async suspension, confirmed moving relocation, eight-lane TSan first-use publication, and allocation-failure retry. The focused witnesses are deterministic; broader generated tier-fuzz incorporation remains owned by this parent.

Related child #715 is complete and landed on main in a5280fad. It closes the remaining captured super write lowering for direct/logical/compound/update expressions across plain and resumable bytecode; exact focused Test262 accounting was unchanged at 94/94 and 485/485.

Compiler/VM parity advanced in #716, landed as e051fc45: delete super.name and computed SuperReferences now lower across plain, generator, async, and async-generator tiers, with tree-walker/VM differential and suspension/GC witnesses. Full units: 1,833 pass, 1 skip, 0 fail/leak.

Tier parity advanced in #717, landed as c8e1b0c0: identifier deletion now has tree-walker/VM differentials across program/plain/resumable tiers, nested with depth, eval-created bindings, proxies/unscopables, moving GC, and cached native rejection. Full units: 1,837 pass / 1 skip / 0 fail/leak.

Completed child #754 in fix(iterator): preserve VM close ordering (#754).

This adds a fixed-seed 17-case tree-walker versus required-VM differential with exact result bits, protocol transcripts, exception messages, and bytecode-admission assertions. It also fixed two minimized VM discrepancies found by the matrix: IteratorValue failures no longer close, and a failed explicit break-close no longer calls return twice. Focused normal and TSan witnesses are clean; exact-parent for-of and for-await-of inventories have zero regressions.

Added a deterministic Proxy ownKeys semantic witness in 9906d22f while closing #758. It covers exact String/Symbol ordering, wide and encoded-NUL keys, duplicate rejection, and target invariants under the newly seeded native membership index. Focused normal/TSan paths pass and the exact-parent test/built-ins/Proxy/ownKeys inventory stays 27/27; this is available for future tier-differential expansion under #491.

Expanded the existing fixed forced-tree/required-bytecode for-in differential in 1adc7b7b with 4 KiB and NUL-bearing keys plus non-enumerable prototype shadowing. Both tiers produce the same exact state with bytecode admissions/no fallback intact; moving-GC and TSan paths pass. Exact-parent for-in Test262 remains 115 pass / 0 fail.