ReviewOS

zig-utils/zig-js

Bench: compare zig-js single/no-GIL scaling with JavaScriptCore

#42
Closed chris opened this 11 days ago ·
11 days ago

Goal

Add a reproducible, honest performance comparison covering:

  • zig-js single-thread execution;
  • zig-js shared-realm no-GIL Thread scaling;
  • JavaScriptCore single-context execution;
  • JavaScriptCore independent-context parallel scaling.

Method

  • one shared pure-JavaScript workload source for both engines;
  • deterministic checksums so timing cannot hide wrong results;
  • ReleaseFast zig-js and the system JavaScriptCore framework;
  • warm-up outside timed regions;
  • seven full samples with median/min/max reporting;
  • equal work per lane and explicit hardware/software metadata;
  • clearly distinguish zig-js shared-realm threads from JSC independent contexts.

Deliverables

  • repeatable benchmark command and machine-readable raw output
  • documented methodology, caveats, and saved dated result
  • README summary linked to the full report
  • lightweight validation plus one batched full benchmark run
  • green full Actions gate

Evidence

11 days ago

Harness landed on main in dde04264.

Completed:

  • exact shared JS workload source for both engines;
  • separate ReleaseFast zig-js and system-JSC runners (avoids symbol interposition);
  • single GC context, shared-realm no-GIL Thread, JSC single-context, and independent-context modes;
  • deterministic per-lane checksums with cross-engine validation;
  • median/raw report driver and explicit macOS-only build failure;
  • compile-only runner gate and the complete reduced matrix are green.

The reduced run already shows why per-workload rows matter: arithmetic/property throughput scales, while arrays and recursive calls expose current shared-realm contention. Starting the single full seven-sample run now; documentation and README will use that saved result rather than the smoke timings.

11 days ago

Full result and documentation landed on main in d962d6bb.

Saved evidence:

  • docs/.data/benchmark-comparison-2026-07-13.tsv: 224 validated timing samples plus header;
  • docs/.data/benchmark-comparison-2026-07-13.md: environment, medians, all 1/2/4/8-lane rows, interpretation, and reproduction;
  • docs/benchmarks.md: stable workloads, execution-model distinctions, timing protocol, caveats, and commands;
  • README.md: concise single-thread and eight-lane comparison tables.

Full seven-sample findings on the 11-core M3 Pro:

  • JSC single-context throughput: 6.77x zig-js by geometric mean across this matrix;
  • zig-js 8-lane scaling: arithmetic 5.97x, properties 4.42x, arrays 1.32x, Fibonacci 0.87x;
  • JSC independent-context 8-lane scaling: 4.91x, 4.67x, 2.88x, 4.41x.

The docs explicitly avoid equating shared-realm zig-js Threads with isolated JSC contexts. Local gates: comparison runners compile, reduced full matrix/checksum validation passes, full 224-sample matrix passes, and bun run docs:build builds 22 pages. Waiting for the pushed Actions run before checking the final box and closing.

11 days ago

The benchmark commits exposed a pre-existing bb422edb CI regression unrelated to benchmark imports: stale tests/fuzzers still called the correctly removed FinalizationRegistry.prototype.cleanupSome. Tracked separately as #43 and fixed in f658843d; focused normal/TSan units plus broad/lifecycle/midgc fuzz profiles are green. #42 remains open until the replacement final Actions run proves the combined main.

11 days ago

Completed across dde0426, d962d6b, and f658843. The full seven-sample benchmark report and raw TSV are published, README and docs navigation are updated, checksum validation passed across the matrix, and the final Actions gate is green (31 passed, 0 failed): https://github.com/zig-utils/zig-js/actions/runs/29275718376

Sign in to comment on this issue.