ReviewOS

zig-utils/zig-js

Bench: repin representative matrix after direct exact-parent support

#774
Closed chris opened this 9 days ago ·
9 days ago

Problem

Current main fails zig build benchmark-comparison-test:

V26 exact-parent integration changed without a matrix version bump:
tools/exact-parent-regression.ts

V26 froze tools/exact-parent-regression.ts at SHA-256 bb464660074fac79c82360306318efd9cd88c9828bf1face21847becdeec8ead. Commit 053ac8ad (#461) later added schema-v2 direct-binary validation, producing current SHA-256 27c35a0a23a74cc33efbe5284396af349e4bdb6419f9aaaf65d380952c1890bb, but did not publish a successor representative matrix.

The fail-closed gate is correct. The matrix inventory is stale.

Scope

Publish V27 as an evidence-integration-only child of V26. It must repin the exact-parent tool and document the schema-v2 direct-binary versus schema-v3 shared-overlay provenance distinction while inheriting all workloads, jobs, checksums, modes, scored operations, thresholds, and publication rulings unchanged.

This must not be fixed by:

  • editing the frozen V26 file;
  • accepting the current digest without a new matrix version;
  • weakening or bypassing validatePinnedFile;
  • changing benchmark workloads, checksums, jobs, lanes, or scoring;
  • silently changing historical artifacts or reports.

Acceptance criteria

  • representative-benchmark-matrix-v27.json hash-inherits V26 and repins only evidence-integration metadata required by #461.
  • The matrix loader defaults to V27, supports schema 27 explicitly, and preserves fail-closed parent/integration validation.
  • Matrix self-tests reject source drift and unsupported future versions while accepting the exact V27 tree.
  • docs/benchmarks.md describes V27 accurately and the owned bunpress output manifest is regenerated.
  • zig build benchmark-comparison-test passes in full on current main.
  • The change flips 0 Test262 cases.

Relationship

Found while validating the benchmark-first #772 row. This repair is intentionally separate so #772 can land only after the repository benchmark gate is healthy again.

9 days ago

Completed by e305bba0 — fix(bench): publish representative matrix v27 (#774).

The complete V24–V26 pin audit found exactly two current-tree deltas:

  • tools/exact-parent-regression.ts: bb4646… → 27c35a… from #461.
  • bench/frontend_parse.zig: e59966… → b463a6… from #770.

V27 hash-inherits V26, repins both files, models schema-v2 direct revision equality separately from schema-v3 identical one-commit overlays, refuses partial provenance, and leaves every representative workload, job, checksum, mode, threshold, replay, and publication ruling unchanged. The #770 classifier-growth rows remain outside representative coverage.

Validation:

  • representative matrix structural tests: 58/58;
  • consolidated benchmark harness: 18/18;
  • both engine adapters, quickening inventory, and independent-suite audit pass;
  • owned bunpress manifest generation and the 50-page check-manifest build pass;
  • flips 0 Test262 cases.

Sign in to comment on this issue.