Build: measure focused-engine source-to-result latency

Parent: #494 Depends on: #462 Related: #617

Goal

Measure and gate the smaller production focused-engine artifact against the combined inline unit-test artifact with comparable source-to-result cache boundaries.

Scope

• Version the build-feedback artifact rather than rewriting the accepted V1 history.
• Add isolated cache groups for library, focused-engine, combined-unit, and TSan scenarios.
• Measure focused frontend cold link/run and cached runtime selection beside equivalent combined-unit cold/cached paths.
• Measure focused-engine and combined-unit TSan paths separately.
• Preserve the full cold/warm unit-suite phases and exact shard plan.
• Validate exact focused denominators and nonzero combined-unit denominators in every retained sample.
• Remove completed cache groups during collection so the evidence job stays within bounded disk usage.

No-workaround rules

• No warm-cache result labeled cold, shared cache pollution between compared artifacts, omitted full-suite phase, reduced sanitizer behavior, zero-match pass, outlier removal, dirty-worktree publication, or evidence collected under overlapping build/test work.

Acceptance

• V1 evidence remains accepted and reproducible; new collections use an explicit V2 schema.
• Comparable focused and combined cold/cached/TSan boundaries are machine-readable and fail closed on denominator or process-quality drift.
• Three sequential samples on a clean exact revision retain every raw command, output, resource measurement, and full-suite plan.
• Focused source-to-result latency improves materially while the full combined integration gate remains exact.
• The #494 documentation and checked evidence identify the precise scope; 0 Test262 behavior changes.

Implementation is on main:

• 7eb51f12 adds the V2 ten-scenario matrix, isolated cache groups, exact denominator/result validation, V1 migration coverage, and bounded completed-group cleanup.
• c3f186f2 corrects the singular one-case output contract and preserves incomplete raw artifacts when final validation fails.

Validation is green: direct/build-integrated collector self-tests, dependency audit (15 classified edges, 0 migrations), 267-file docs link audit, and checked 49-page docs build.

No performance evidence is promoted yet. The first one-sample diagnostic correctly failed final validation on the singular grammar mismatch. The corrected repeat established valid cold denominators but was deliberately aborted when an unrelated user-owned Bun/Home Zig test graph began compiling concurrently; that host load invalidates timing evidence. The reproducible temporary cache was removed, and no samples were discarded or retried into a claim.

#614 remains open for one clean diagnostic followed by the required three-sample publication run when the host is idle. Flips 0 Test262 cases.

Fail-closed overlap handling is now complete on main:

• fa89331c rejects another Zig compiler/build driver before each scenario and preserves all orchestration failures as incomplete raw artifacts.
• 483d0fa1 closes the start-after-preflight race by repeating the same process-boundary audit after every measured command.

The live integration fixture ran while the user-owned Home build was active. V2 rejected before clean_library and wrote exactly:

• schema: zig-js-build-feedback-v2
• complete: false
• requested samples: 1
• collected rows: 0

The fixture was removed after inspection. This is validation of evidence rejection, not timing evidence. The clean one-sample diagnostic and three-sample publication remain pending until the external suite stops and the host settles.

Evidence-safety hardening pushed in d8fc8c97 (fix(bench): reject contaminated build feedback (#614)).

The V2 collector now shares #617’s exact process classifier, rejects unrelated Zig/maker, Home test, unit/corpus, and frontend benchmark processes both before and after every scenario, records complete-process CPU occupancy, and fails below a 60% minimum. V1 validation remains unchanged for the checked historical artifact. The fail-closed policy lives in one shared module so exact-parent and focused-build evidence cannot drift independently.

Validation: build-feedback self-test passed; exact-parent self-test passed; dependency audit remains 15 classified edges / 0 migration targets / 60 tools. This changes tooling only and flips 0 Test262 cases. The clean three-sample V2 collection remains pending an exclusive host window.

Portability child #660 is complete in 8aabe412. The exact-parent structural self-test now passes under the supported JSC-backed Home runtime without a global polyfill or collector/schema change. The complete local 31/31 gate and fresh Linux benchmark-comparison harness pass; V17 remains unchanged and V18 repins only the portable self-test source. This removes a tooling blocker but does not substitute for the remaining quiet-host #614 evidence.

The exact-parent tooling extension in child #661 landed as b815b3e7. It adds fail-closed cold Context phase/RSS/finalizer telemetry while preserving the existing compiler evidence collector and V18 history through V19 hash inheritance. This does not replace the remaining clean quiet-host focused-engine measurements required by #614.

The first clean V2 diagnostic at be1478fd exposed a short-phase quality-gate defect before publication.

The collector retained six rows and then failed closed on focused_test_cached: the exact one-test command passed in 0.32 s with the required nonzero linked denominator and zero failures/leaks, and both process-boundary audits were clear, but macOS /usr/bin/time reported 0.08 s user + 0.11 s system. Its centisecond resolution therefore produced 59.375% process occupancy against the 60% gate.

The incomplete artifact remains local and is not publication evidence; no row will be dropped or retried into a claim. I am correcting the quality contract so subsecond cached latency uses a signal that is valid at that boundary, while retaining the 60% CPU-occupancy requirement for CPU-bound build/full-suite phases and the before/after competing-process rejection for every phase. The required diagnostic and three-sample run will restart on the resulting exact revision.

A fresh one-sample diagnostic on clean exact revision b76ae1e8 correctly failed closed after clean_library. An unrelated user-owned Home home-tsc ReleaseFast build started during the scenario; the post-command process audit identified both the maker build driver and its Zig build-exe child.

The preserved local artifact is V2, complete: false, requested sample count 1, with one retained clean_library row. That row is not timing evidence because final scenario validation failed. I will not discard it, retry around it, or begin the required three-sample publication while the external compiler remains active. The corrected short-cache boundary was not reached in this attempt.

A second clean one-sample diagnostic on b76ae1e8 again failed closed when a user-owned Home home-tsc ReleaseFast build started during focused_test_relink. The post-scenario audit identified the independent maker build driver and Zig build-exe child.

This separately preserved V2 artifact is complete: false with five successful retained rows (clean_library, incremental_library, focused_engine_relink, focused_engine_cached, and focused_test_relink). All five exit codes are zero, but none is publication evidence because final validation rejected the run. The earlier incomplete artifact remains separate. I will not start the three-sample collection under the recurring external load or select only the clean-looking prefix.

A third clean one-sample diagnostic on exact revision b76ae1e8 again failed closed when a user-owned Home ts_checker ReleaseFast test began during focused_engine_relink. The post-scenario audit identified the independent maker build ... test -Dfilter=ts_checker process and its Zig test child.

The separately preserved V2 artifact is complete: false, requested sample count 1, with three retained zero-exit rows: clean_library, incremental_library, and focused_engine_relink. The Markdown report was correctly not emitted. None of these rows is publication evidence because the run failed its final process boundary.

This is the third independently preserved diagnostic rejected by the same recurring external Home workload. I will not run the three-sample publication or select clean-looking prefixes until an actually exclusive host window exists; work continues on other open issues in the meantime.

A fourth clean one-sample diagnostic on exact revision b76ae1e8 again failed closed when a user-owned Home ts_driver ReleaseFast test began during focused_test_relink. The post-scenario audit identified the independent maker build ... test -Dfilter=ts_driver process and its Zig test child.

The separately preserved V2 artifact is complete: false, requested sample count 1, with five retained zero-exit rows: clean_library, incremental_library, focused_engine_relink, focused_engine_cached, and focused_test_relink. The focused production artifact had exact 12/1 denominators, but none of the prefix is publication evidence because final process validation rejected the run; no Markdown report was emitted.

This is the fourth independently preserved diagnostic rejected by recurring external Home work. I will not retry around that activity, select clean-looking rows, or begin the required three-sample publication until the host remains exclusive for the complete matrix.

A fifth clean one-sample diagnostic on exact revision b76ae1e8 again failed closed when an unrelated user-owned maker build ... debug -j1 process began during focused_engine_relink; its Zig build-exe child was using 98.8% CPU when inspected.

The separately preserved V2 artifact is /tmp/zig-js-614-diag5.json, complete: false, requested sample count 1, with three retained zero-exit rows: clean_library (73.41 s), incremental_library (0.18 s), and focused_engine_relink (74.79 s, exact 26-case focused frontend run). The Markdown report was correctly not emitted. None of these rows is publication evidence because post-scenario process validation rejected the run.

I will not retry around the active external build, select the clean prefix, or start the required three-sample publication or #531/#532 quiet-reference collection under the same contamination. Independent engine work remains available.

The V2 denominator repair landed in 2c659fc9: the collector, fixture, and checked docs now require the live 79-case production frontend gate. Direct/build-integrated self-tests pass, as does the 50-page docs build. Flips 0 test262 cases.\n\nA fresh one-sample diagnostic on exact revision 2c659fc9 then failed closed after full_unit_cold_history because an unrelated Home ts_checker build began during that scenario. The preserved local artifact is /tmp/zig-js-614-diag-2c659fc9.json, schema V2, complete: false, with seven retained zero-exit rows through the complete cold suite. The post-scenario audit identified the independent lang-bun-port maker and its Zig test child.\n\nThe exact 79-case and one-case frontend denominators passed, as did both nonzero combined-unit filters and the full 10-shard cold suite. None of the prefix is publication evidence because the process boundary rejected the run; no Markdown report was emitted. I will not retry around the active external test or begin the three-sample publication under recurring host load.