ReviewOS

home-lang/home

infra(bench): restore current-source TypeScript benchmark execution capacity

#732
Open glennmichael123 opened this 8 days ago ·
8 days ago

Parent: #416.

Current-source TypeScript benchmark execution is blocked by available build capacity:

  • On the 16 GB Apple-silicon development host, zig build home-tsc -Doptimize=ReleaseFast -j1 was run alone under the repository supervisor. A 3,584 MB ceiling stopped it at 3,588 MB; a final cached retry with a 3,840 MB ceiling stopped at 3,854 MB. The host guard reported 33% and 41% headroom respectively. The ceiling was not raised again.
  • The repository's existing manually dispatched TypeScript Frontend Benchmark workflow was then requested on main, but GitHub returned HTTP 422: “Actions has been disabled for this repository.” No runner started.
  • The pinned TS 6.0.3 and native TS 7.0.2 competitors and Hyperfine are prepared locally, but the existing zig-out/bin/home-tsc dates from August 26 and must not be used for current-source claims.

Restore an isolated benchmark execution path that can build the current ReleaseFast frontend and run the unchanged schema-3 admission plus the complete 30-round suite. Keep provenance, all successful finite samples, compiler-order rotation, exact negative controls, and artifact retention. Do not treat a ReleaseSmall/Debug build, stale executable, partial workload selection, raised timeout, or missing competitor as comparable evidence.

Once capacity is restored, publish Linux x64 results, compare every admitted row with the fastest pinned competitor, profile any loss, and update TS_PERFORMANCE.md and the README table from retained raw evidence.

8 days ago

Follow-up capacity check: a scoped stripped-ReleaseFast build option was evaluated locally as a possible way to reduce link memory. The first supervised run did not start because host headroom was 34% against the 35% admission floor; a second attempt at a 33% floor also did not start because headroom had fallen to 32%. No compiler process ran, no result was produced, and the experimental build-file edit was removed.

The working tree is back to current origin/main with no tracked changes. The only local item is the pre-existing untracked packages/runtime/test/test.txt, which remains untouched. #732 stays open; no benchmark or README timing claim changes.

8 days ago

Published build capacity and benchmark documentation to origin/main as glennmichael123 <gtorregosa@gmail.com>, without coauthors:

  • 11a38aa95 adds -Dhome-tsc-strip=true, uses it in the benchmark workflow and documented reproduction, and retains ReleaseFast semantics. The current compiler built with one job at 2,942 MB peak supervised tree footprint, down from the unstripped attempts that crossed 3,584 MB and 3,840 MB ceilings.
  • aa60cdbdf publishes focused result 20260911T033015Z in TS_PERFORMANCE.md and the README.

The focused run admitted unchanged checkjs_jsdoc and type_predicates_large, used 30 fresh processes after three warmups, rotated compiler order, retained all 180 samples, and verified artifact provenance unchanged before admission and after measurement. Mac14,9, 16 GiB, 10 cores, macOS 26.3.1 arm64. Benchmark peak was 452 MB.

WorkloadTypeScript 6.0.3Native TypeScript 7.0.2Home 0.1.0Home lead
checkjs_jsdoc270.0 ± 54.1 ms69.2 ± 3.7 ms39.0 ± 1.4 ms1.77×
type_predicates_large1257.2 ± 104.6 ms421.8 ± 20.1 ms288.0 ± 15.3 ms1.46×

This focused confirmation supplements the prior full 20-workload snapshot. It is not a cross-platform or full-suite claim. #733 tracks the narrower large-predicate scaling lead. #732 remains open because GitHub Actions is disabled and the Linux x64 full-suite runner still cannot be dispatched.

8 days ago

Execution-capacity checkpoint after #839 source/evidence publication: current-source parent build at 3318885 was attempted with Zig 2163, stripped ReleaseFast, unchanged 3840 MB footprint cap and unchanged 1024/512 MB disk floors. The guard exited 121 before compiler execution after a bounded 30-second wait for the machine lock; its live holder is the separate ts-program-file-syntax-admitted Program-suite run. The volume now has approximately 583 MiB free, below the unchanged admission floor. No build, test pass, speed claim, or cap relaxation is recorded for this attempt. The candidate worktree is clean, all frozen evidence is published, and unrelated jobs/files were left untouched. A new exact-parent A/B must wait for sufficient headroom and the shared runner before the next export-facts query-reuse candidate can be accepted.

8 days ago

Capacity revalidated after #839 publication: the previous Program-suite runner PID 11179 is absent, so this is no longer a verified wait for that process. Disk headroom fell to roughly 115 MiB and even creating a build log failed before guard execution. Reclaimed only about 150 MiB of task-owned rebuildable cache: the frozen-parent worktree .zig-cache and candidate incremental .zig-cache/z. Source, frozen binaries, raw evidence, retained profiling objects, and unrelated files/jobs are untouched. The subsequent real guard attempt exits 123 before compiler execution, reporting disk below the unchanged 1024 MiB admission floor (roughly 302 MiB free immediately after cleanup). No limit was relaxed. The next #839 source/test draft is preserved locally and uncommitted; complete validation needs external disk headroom recovery. No build or test pass is inferred from the guard refusal.

8 days ago

Capacity recovery allowed #839 to complete exact-parent/candidate builds, all complete semantic suites, pinned Zod stability checks, 356 paired rounds and the full 3000-sample five-compiler matrix. Source 69c3d28 and verified evidence/docs eaaddf2 are pushed to main. All guard ceilings/floors remain unchanged. Headroom then fell again to about 481 MiB after publication, below the 1024/512 MiB disk floors; no new heavy run has been started or limit relaxed. Successful completed runs remain valid and are not relabeled as failures. The candidate worktree is clean; further recursive-owner preparation work will require restored stable headroom.

8 days ago

Final authoritative check after the publication comments shows external headroom recovery to approximately 6.2 GiB. The accepted code/evidence worktree remains clean at eaaddf2 and origin/main matches. This changes the next action back to source-level investigation of remaining recursive export-facts preparation, rather than a blocked capacity audit. No other files/jobs were altered to obtain that recovery, and the guard remains mandatory and unchanged; this fluctuating snapshot is not a claim that execution capacity is permanently fixed.

8 days ago

Current #843/#844 candidate has complete semantic suites and exact 30-case TS oracle parity, with unchanged full-Zod identities. Publication remains held for broader graph coverage and performance/lint evidence. First screen/lint guard attempts exited 123 before execution at 861 MiB; following a measured external recovery above 1024 MiB, both queued retries later exited 123 before execution at 506 MiB. Neither is a timing or lint result, and no limit/fixture/diagnostic contract is relaxed. Candidate build caches in this worktree are now absent (authoritative directory check); the old task-owned parent cache is only 94 MiB, insufficient to restore admission headroom by itself. Frozen binaries, raw oracle evidence and source are preserved. Unrelated files/jobs remain untouched. Stable external disk headroom recovery is required for the remaining guarded runs.

8 days ago

Authoritative headroom check now shows approximately 16 GiB free. Earlier screen/lint guard refusals are terminal and remain retained; no process is being restarted on an observation timeout. New guarded attempts on unchanged frozen binaries/inputs are running/queued with exactly the same 3840 MiB cap and 1024/512 MiB disk floors. This external recovery changes the next action back to measurement, not a claim of permanent infrastructure resolution or completed goal.

Sign in to comment on this issue.