ReviewOS

zig-utils/zig-js

Bench: attach native allocation replays to exact-parent batches

#769
Closed chris opened this 11 days ago ·
11 days ago

Parents: #767, #768 Related: #503, #670

Problem

The exact-parent collector records measured-boundary CPU work, but its allocation parser currently recognizes only frontend-runner allocation rows. Runtime comparison workloads such as the frozen #767 string-indexing matrix expose exact Context backing allocations through a separate attribution_no_jit invocation. Leaving those observations unavailable prevents the algorithmic-growth validator from scoring its required allocation dimensions; substituting zero or weakening the schema would be incorrect. Invoking the Home TypeScript checker separately for every frozen row also repeats minutes of tool startup while benchmark rows themselves must remain serialized.

Scope

  • Add an explicit untimed native attribution replay mode to exact-parent collection.
  • Parse exactly one invocation-phase attribution record and validate kind, mode, workload, lanes, jobs, checksum, no-JIT tier boundary, and allocation counters.
  • Attach runtime backing allocation count/bytes to the existing attribution metrics with exact provenance; keep replay outside elapsed_ns.
  • Add a serial batch-row interface so many frozen rows share one Home compilation without running benchmark processes concurrently.
  • Preserve per-row balanced ordering, competing-process checks, measured-boundary occupancy, independent artifacts, and fail-closed partial progress.
  • Version and pin the representative integration without rewriting historical evidence.

Acceptance

  • Parser self-tests reject missing/duplicate/wrong-phase, identity, checksum, tier, and allocation fields.
  • Two replay observations per variant remain exact for every #767 width.
  • Batch collection is byte-equivalent in schema and validation to individual collection and never overlaps benchmark children.
  • V26 matrix, dependency audit, docs links/build, and benchmark evidence self-tests pass.
  • #767 exact growth artifacts collect with measured instructions and allocations; flips 0 Test262 cases.
11 days ago

Landed cb90eb2f (fix(bench): attach native allocation replays (#769)). The collector now parses exactly one invocation-phase native attribution row, validates mode/workload/lanes/jobs/checksum/no-JIT tier/admissions, attaches positive Context backing allocation count and bytes outside elapsed_ns, requires exact repeats per variant, and refuses artifact overwrites. Repeated batch rows execute synchronously under one Home compilation and retain one independently validated artifact per row.\n\nValidation: exact-parent identity/pairing/native-replay/batch/regression/cleanliness/competing-job self-test passed; representative V26 validates 18 implemented families / 0 deferred and 55/55 mutations; dependency audit passed 16 edges / 0 migrations / 63 tools; 277 documentation files resolve; BunPress manifest check built 50 pages; flips 0 Test262 cases.\n\nThe first live ASCII-1024 probe was correctly rejected before any benchmark child launched because an unrelated Home/Bun Zig test began during Home compilation. No sample or artifact was written. Live replay validation and #767 batch collection will resume only after that competing job exits.

11 days ago

Live exact-parent integration is now proven for the first frozen #767 row after cb90eb2f. representative_string_utf16_ascii_1024 collected two balanced observations per variant with checksum 23,880,132, invariant native replay values of 176,365 allocations / 24,257,955 bytes, measured CPU occupancy 99.87–100%, and no competing processes. Retired-instruction RSD was 0.0024% parent / 0.0076% candidate.

The subsequent 17-row serial batch was correctly rejected before its first benchmark child when unrelated Home ts_checker/Zig jobs began during the one-time TypeScript compile. No partial remainder artifact was written. This proves the live fail-closed competitor boundary while preserving the accepted ASCII control; the remaining 17 rows stay pending for a genuinely quiet window.

11 days ago

Native allocation replay is now present in checked exact-parent evidence at 4e4531ec: every parent/candidate ASCII-1024 sample records exactly 176,365 allocation requests and 24,257,955 bytes, with matching workload identity/checksum and no missing values encoded as zero. The remaining multi-row #767 batch is still pending a host window; its latest attempt failed closed before row one on unrelated Home builds.

11 days ago

Completed by 33734e97 — bench(strings): publish guarded UTF-16 growth (#767).

Every one of the 18 widths/representations retains two native allocation observations per variant, and both allocation count and cumulative allocated bytes repeat exactly. ASCII is unchanged. Each non-ASCII candidate row adds exactly four allocations and 336/592/1104 bytes at 1K/2K/4K while changing retired-instruction growth from 1.93–1.97 exponents to 1.006–1.008.

The six accepted v3 growth artifacts embed all ordinary rows losslessly, retain measured instructions plus allocations, pass the official artifact validator, and keep non-scored wall metrics diagnostic. This evidence-only commit flips 0 Test262 cases.

Sign in to comment on this issue.