ReviewOS

zig-utils/zig-js

Compiler: classify captured loop bindings in one AST pass

#616
Closed chris opened this 9 days ago ·
9 days ago

Parent: #465 Related: #493, #496, #615

Problem

Plain-function admission currently determines whether a lexical for or for-in/of head needs per-iteration Environment Records by walking the loop head, condition, update, iterable, and body separately for every bound identifier. A wide declaration/destructuring head combined with a large closure-free body therefore performs binding-count × AST-size classifier work before bytecode emission.

This is distinct from #615: #615 scans pending TDZ names once. This child owns repeated loop-capture classification while preserving the exact CreatePerIterationEnvironment / ForIn/OfBodyEvaluation boundary.

Scope

  • Collect lexical loop-head binding names once, without treating binding positions as references.
  • Traverse each relevant loop region once and match nested function/arrow references against exact binding membership.
  • Preserve conservative shadowing and deferred-class behavior unless exact scope information proves a narrower answer.
  • Preserve ordinary frame-slot lowering for uncaptured heads and Environment Record lowering for genuinely captured heads.
  • Cover declaration groups, object/array destructuring, defaults/computed keys, initializers, condition/update expressions, iterable expressions, nested closures, classes, and nested loops.
  • Measure clear, unrelated, early/late capture, and destructuring growth using production parse + admission/compile work with exact structural checksums and allocation replay.

No-workaround rules

  • No skipped syntax, source rewriting, heuristic size cutoffs, guessed shadowing, test exclusions, or semantic fallback presented as an optimization.
  • A one-binding common case must not pay an attacker-sized temporary structure.
  • Any newly shared classifier state must remain compile-owned and allocation/lifetime explicit.

Acceptance

  • Exact semantic fixtures prove captured heads still use fresh per-iteration bindings and uncaptured heads remain slot-backed in tree-walker and required-bytecode execution.
  • Parent → candidate growth evidence demonstrates bounded classifier scaling for wide heads and large unrelated bodies, with all control shapes and allocation metrics.
  • Full unit, focused TSan where applicable, and relevant Test262 subtrees pass with exact before → after failure accounting.
  • Evidence and parent issues are cross-linked; published claims name the scored boundary and exact revisions.
9 days ago

Pushed e6200a82 (bench(frontend): add loop capture growth witness (#616)).

The production frontend runner now has eight parse + plain-function admission/compile contracts: clear 1,024/2,048/4,096-binding growth, a 4,096-binding plus 4,096-unrelated-reference body, first/last capture controls, and wide for-of destructuring clear/captured controls. Every job validates admission, environment-vs-slot policy, complete chunk structure, frozen checksum, and an untimed identical-work allocation replay.

All 8/8 contracts and allocation replays pass. This is benchmark infrastructure only, flips 0 Test262 cases, and makes no timing claim while the host is occupied. The exact-parent implementation remains next.

9 days ago

Pushed 0bfd79d0 (perf(compiler): scan loop capture names once (#616)).

The compiler now collects lexical loop-head names once and checks them through the existing exhaustive identifier walker. Classic for and for-in/of head/test/update/RHS/body regions are visited once rather than once per binding. The one-binding path remains the allocation-free exact-name specialization; only multi-binding heads allocate one compile-arena hash set. Conservative shadowing and deferred-class handling are unchanged.

Validation so far:

  • all 8 parent/candidate benchmark admission and complete chunk checksums are identical, with allocation replay passing;
  • new 14-case direct classifier matrix passes;
  • compiler-filtered units: 69/69;
  • forced tree-walker ↔ required-bytecode lexical loop identity and captured-destructuring differentials pass;
  • focused frontend: 12/12.

Full-suite, Test262 before→after accounting, and exact-parent performance evidence remain pending. No corpus-flip or publishable timing claim is made yet.

9 days ago

Current-main full-suite evidence is green after the #616 implementation: zig build test-parallel completed 1,732 passed, 1 platform skip, 0 failed, 0 leaked across 10 shards in 167.1 s wall time. The new classifier unit, compiler-filtered set, forced loop-identity/destructuring differentials, focused frontend, and 8/8 structural benchmark contracts also remain green.

Relevant Test262 before→after accounting and exact-parent performance evidence are still pending; corpus execution is being held behind a newly started user-owned Home unit compile.

9 days ago

Exact parent→candidate Test262 accounting is complete for every loop family sharing the changed classifier:

  • test/language/statements/for: 0 fail / 385 pass → 0 / 385
  • test/language/statements/for-of: 0 / 751 → 0 / 751
  • test/language/statements/for-in: 0 / 115 → 0 / 115
  • test/language/statements/for-await-of: 0 / 1,234 → 0 / 1,234

Total: 0 fail / 2,485 pass → 0 / 2,485. Every parent/candidate diagnostic transcript is byte-for-byte identical and every denominator is nonzero. The current-main full suite is also green at 1,732 passed / 1 platform skip / 0 failed / 0 leaked.

The classifier state is function-compile-local and introduces no shared mutable state, so a focused TSan gate is not applicable; the runtime identity checks are instead covered by the existing forced tree-walker↔required-bytecode tests. Only exact-parent growth/allocation evidence and final documentation/cross-links remain.

9 days ago

Completed and published the remaining exact-parent bounded-scaling evidence.

The seven-pair quiet-reference comparison scores only normalized retired instructions and exact allocation replay across the frozen 1,024/2,048/4,096 closure-free lexical loop-head rows. Parent and candidate binaries use byte-identical one-commit measurement overlays, and the candidate remains the exact first child of the benchmark parent.

Results:

  • parent 1K→4K instruction growth: 14.8145x, exponent 1.944
  • candidate 1K→4K instruction growth: 4.0486x, exponent 1.009
  • candidate/parent instructions: 0.1044x, 0.0556x, 0.0285x
  • instruction RSD: below 0.03% for every variant/width
  • allocations/job P/C: 2,211/2,220, 4,279/4,289, 8,394/8,405
  • bytes/job P/C: 2,064,466/2,134,178, 4,627,314/4,766,682, 8,646,922/8,925,578
  • checksums: exact at all widths

The bounded deterministic allocation increase is published explicitly. Wall, throughput, latency, cycle, energy, RSS, thermal, and full-efficiency observations remain diagnostic and are not promoted. The 1K/2K ordinary rows retain their blocked full-efficiency decisions rather than weakening that gate.

All three performance-attribution-v3 inputs, the algorithmic-growth-v3 aggregate, all 328 documentation links, the 50-page BunPress build, and git diff --check pass. Correctness remains the previously measured 2,485 -> 2,485 targeted Test262 passes with zero flips, plus the focused/full differential gates recorded above. This satisfies the remaining evidence and cross-link acceptance items and closes #616; parent #465 and related #493, #496, #615, and completed repeated-body boundary #618 remain linked.

Sign in to comment on this issue.