ReviewOS

zig-utils/zig-js

tools(evidence): migrate benchmark publication to Home TypeScript

#507
Closed chris opened this 9 days ago ·
9 days ago

Parent: #497 Runtime foundation: home-lang/home#170

Scope

Migrate all 12 benchmark_evidence tools in docs/.data/tool-migration-inventory-v1.json to TypeScript executed by Home.

This includes benchmark collection, exact-parent A/B orchestration, evidence validation, statistics, publication, and README/report generation.

Contract

  • Home emits TypeScript and zig-js is the default tool engine; system JSC remains a selectable differential backend for engine-neutral logic.
  • Workload engines remain explicit benchmark subprocesses. The tool runtime must not contaminate the engine-under-test identity.
  • No Bun, Node, or Python fallback.
  • Preserve sample ordering, warmups, A/B balancing, exact-parent cleanliness checks, statistics, raw TSV/JSON schemas, generated Markdown, and failure diagnostics.
  • Never publish from a dirty tracked worktree or promote quick/smoke timings.

Acceptance criteria

  • All 12 benchmark-evidence tools are TypeScript invoked through home-tool.
  • Synthetic structural fixtures match existing statistics and diagnostics.
  • Existing raw samples and generated reports reproduce byte-for-byte unless a schema change is separately reviewed.
  • Exact-parent and clean-worktree checks remain mandatory.
  • Build, performance CI, docs, skills, and release callers are migrated atomically.
  • Representative smoke collection succeeds without changing any public performance claim.
  • Dependency inventory contains no residual Python/Node/Bun edge for this family.
9 days ago

Closing as superseded by the selective disposition workflow in #497. The project is not migrating all 12 evidence tools merely to eliminate Python. Existing benchmark/evidence implementations remain authoritative until an individually selected durable tool has a real Home execution contract and byte/schema compatibility evidence.

9 days ago

Benchmark-evidence migration is complete on main.\n\nImplemented and pushed:\n- 481e0d1f — comparison/publication/representative benchmark workflow\n- 49802edb — explicit-compaction evidence\n- ec55dc73 — generational nursery policy evidence plus folded structural tests\n- 0150fdab — Wasm SIMD/scalar and shared-memory Threads evidence\n\nAll migrated drivers execute through Home TypeScript, retain raw/report validation, and were exercised with both the default zig-js Home engine and the configurable JavaScriptCore backend. Live quick benchmark runs passed checksum, oracle, metadata, movement/rendezvous, and JSC-boundary checks. The dependency audit reports 11 remaining Python tools, all outside the benchmark-evidence profile.\n\nFlips 0 test262 cases.

Sign in to comment on this issue.