ReviewOS

zig-utils/zig-js

bench: publish frozen representative family slices

#999
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Problem

The representative benchmark harness validates and publishes only the complete matrix. Focused performance work such as the JSON tranche under #473 therefore cannot produce a reproducible seven-sample direct-JSC report from the checked-in frozen contract without either running unrelated families or using an ad-hoc collector.

Scope

  • add an explicit family selector to tools/representative-benchmark.ts;
  • validate requested names against the loaded frozen manifest before filtering;
  • keep frozen jobs, checksums, timing floor, alternating engine order, clean-worktree checks, and tier-attribution evidence;
  • label reports as focused slices and state that unselected families are outside the report, not passes;
  • expose the selector through zig build representative-benchmark;
  • add structural self-tests and documentation.

Done when

A clean full-work json slice produces seven samples for every selected row plus the required attribution artifact, while an unknown, empty, or duplicate selection fails closed. This issue changes collection scope only; it must not alter workload source, jobs, checksums, or acceptance rules.

Parent: #473.

9 days ago

Landed on main in 0e9b51b8.

The new -Drepresentative-benchmark-families= option filters only after the complete frozen manifest validates. Requested names must be non-empty, unique, and implemented; selected rows retain frozen source, jobs, checksums, timing floor, deterministic engine ordering, clean-worktree checks, and tier attribution. Reports explicitly mark every unselected family outside their coverage.

Validation: representative harness self-test; zig fmt --check build.zig; docs link check (343 files); BunPress manifest regeneration and checked 51-page build. This is tooling/docs only and flips 0 test262 cases.

Sign in to comment on this issue.