ReviewOS

zig-utils/zig-js

Perf(shape): derive transition-map keys from one secure realm seed

#662
Closed chris opened this 10 days ago ·
10 days ago

Parent: #479 Related: #461, #493

Profile finding

A fresh current-main five-second sample of 30,000 production Context create/destroy iterations attributes the leading engine leaf to Shape transition construction: 257 samples in Shape.transitionFromState, 147 in transition publication, and 158 top-of-stack samples in the platform AES path reached by repeated secure-random requests. Every Shape currently requests a new 128-bit SipHash seed when its first transition map is created, even though all Shapes already share one realm lifetime and allocator.

This is diagnostic attribution, not a public timing claim.

Goal

Request secure entropy once for each root Shape, then derive a distinct keyed SipHash seed for every transition map from that realm-owned secret and a monotonic per-realm domain value. Preserve the current collision-resistant expected search bound without process-global mutable state, cross-realm sharing, predictable public keys, or entropy fallbacks.

Scope

  • Add a root-owned transition-key source whose lifetime exactly matches the Shape tree.
  • Fail Context creation closed when the single secure seed cannot be obtained, matching the current security contract.
  • Derive a separate key for each transition map with explicit domain separation and thread-safe unique allocation under parallel Shape publication.
  • Keep transition lookup, immutable names, release publication, per-Shape writer serialization, exact convergence, and arena teardown unchanged.
  • Add deterministic fixed-seed derivation tests plus concurrent uniqueness/publication coverage.
  • Measure the exact parent with the #661 no-evaluation lifecycle profile and a warm property/shape control; retain exact checksums, finalizer reconciliation, RSS plateau, and Test262 behavior.

No-workaround rules

  • No constant or public seed, weak PRNG, entropy-error fallback, process-global key/counter, shared mutable Shape tree, skipped transition caching, disabled locking, or benchmark-only fast path.
  • No changes to stacks, queues, jobs, Promise, Worker, or Map/Set iterator surfaces.

Acceptance

  • One secure entropy request initializes each root Shape; every transition-map key is uniquely derived within that realm.
  • Fixed-seed tests prove deterministic domain separation and distinct adjacent keys.
  • Concurrent transition publication preserves exact same-name convergence, logarithmic keyed lookup structure, and race-free unique key allocation.
  • Exact-parent cold lifecycle evidence shows the entropy/crypto hotspot removed with identical checksums, finalizers, and bounded retained RSS.
  • Warm shape/property controls do not regress materially.
  • Focused Shape, Context, no-GIL/TSan, full unit, and relevant Test262 gates pass with zero semantic flips.
10 days ago

Implementation landed on main in cb0200bc.

The root Shape now owns one fail-closed 128-bit secret and an atomic non-repeating domain allocator. Each transition head receives a separate two-domain SipHash-derived 128-bit key; Shapes carry the shared owner pointer, so the allocator field also shrinks from every immutable Shape. No process-global or cross-realm mutable state was added.

Current validation:

  • standalone Shape suite: 29/29
  • full sharded units: 1,750 passed, 1 skipped, 0 failed/leaked
  • focused TSan: concurrent domain allocation and same-name convergence pass
  • Test262 test/built-ins/Object: 3,411 -> 3,411 pass, 0 fail, flips 0
  • matched sampled 30,000-context diagnostic: create 24.516470460 s -> 18.647319643 s; total 28.483257526 s -> 22.611010675 s; platform AES leaves 158 -> 0; exact 75,870,000 finalizers and bounded retained RSS

The timing sample is diagnostic. The exact-parent lifecycle and warm controls remain open before the performance acceptance boxes are checked.

10 days ago

Fresh pushed CI for cb0200bc is green on the relevant repository-wide gates:

The guarded exact-parent cold collection twice passed an initially quiet preflight and then rejected when a separate Home/Bun Zig test graph appeared between pairs. Neither attempt wrote JSON or Markdown output, and no competing process was modified. Formal cold/warm evidence remains pending a sustained exclusive host window; diagnostic samples are not being promoted.

10 days ago

Cold exact-parent evidence landed in d7f0d2ff: raw JSON and generated report.

Seven order-balanced pairs measured the cold production Context create / exact destroy and finalization boundary. Candidate median is 702.634 ms versus 883.977 ms for the exact parent (0.795x; 1.30% / 2.39% RSD). Retired instructions are 0.887x, cycles 0.796x, and process energy 0.843x, with stable counters and nominal-to-nominal thermals throughout. Peak and retained RSS are both 0.998x. Every checksum and the exact 2,529,000-cell finalizer count match; no sample was discarded.

This completes only the cold-evidence box. The formal warmed transition-fanout control remains pending. The superseded implementation CI run also exposed the independently tracked Map/Set iterator TSan race, so the comprehensive no-GIL/TSan acceptance box remains open while the current-head matrix runs.

10 days ago

Current descendant evidence update: CI run 32485314750, which retains cb0200b, passed all ordinary unit shards, focused-engine-tests, Test262 parallel, docs/dependency gates, threadfuzz amplified/broad/verify, ReleaseSafe threadfuzz, and two TSan threadfuzz profiles.

It is not a comprehensive green matrix: the exact observed failures are the known #643 Map iterator TSan race plus two thread/lifecycle corpus cases (mc-lock-stop-vs-park and oom-one-thread), and later pushes superseded the run. The comprehensive box remains open. The warm shape/property control also remains pending; no timing claim or acceptance checkbox is being promoted from this CI evidence.

10 days ago

Two unblocks landed/identified: #643 is fixed (04d46c59), removing the Map iterator TSan race that kept descendant matrices red. The remaining corpus failures now have their own tracking issue: #722 (`mc-lock-stop-vs-park`, `oom-one-thread`). Once #722 lands, a single green matrix can close the comprehensive box here.

10 days ago

Warm-control preparation is complete without making a performance claim. Frozen parent b815b3e7 and candidate cb0200bc comparison runners were rebuilt with Zig 0.17.0-dev.1441; SHA-256 identities are db2fe8a356fbf73e60ac5ec78f7b99d642c26997787c07f4e057d2f9629695af and 24d5eaeb1e9df9165b44551f5d8950b900e679a3b04b5e829e7eb9983356f2fe. The embedded comparison and representative sources are byte-identical (7a1a5bef… / 94d9463c…), and a one-job representative_shape_transition_fanout contract probe returns checksum 2 on both revisions. The host remains on battery and intermittent Home tests are active, so elapsed diagnostics are discarded and no exact warm sample has started.

10 days ago

Comprehensive integration is now green in main CI run 32923883777: 53 required jobs passed, the one nightly-only job was skipped as configured, and 0 jobs failed or were cancelled. This closes the focused/full unit, Test262, no-GIL/TSan, threadfuzz, docs, ABI, Wasm, and benchmark integration box for the landed #662 implementation. The warm shape/property control remains open; the current host is AC/nominal but has an unrelated unit process and system indexing load, so no timing sample is being started or retained.

10 days ago

Completed and published the remaining warmed Shape/property control.

The seven-pair quiet-reference exact-parent warm control compares cb0200bc directly with b815b3e7 over 100,000 prepared Shape transitions. It passes the frozen 10% material-regression gate:

  • wall: 1.0275x
  • retired instructions: 1.0021x
  • cycles: 1.0190x
  • process energy: 1.0153x
  • checksum: exact (5,000,150,000)
  • thermal state: nominal throughout
  • all wall and efficiency series: stable

The existing cold lifecycle artifact remains the finalizer/retained-RSS evidence boundary; the warmed mixed-tier row does not claim allocation equivalence. Schema-v3 validation, all 314 documentation links, the 50-page BunPress build, and git diff --check pass. The publication commit is verified at origin/main and closes #662.

Sign in to comment on this issue.