ReviewOS

zig-utils/zig-js

Perf(shape): derive transition-map keys from one secure realm seed

#662
Closed chris opened this 9 days ago ·
9 days ago

Parent: #479 Related: #461, #493

Profile finding

A fresh current-main five-second sample of 30,000 production Context create/destroy iterations attributes the leading engine leaf to Shape transition construction: 257 samples in Shape.transitionFromState, 147 in transition publication, and 158 top-of-stack samples in the platform AES path reached by repeated secure-random requests. Every Shape currently requests a new 128-bit SipHash seed when its first transition map is created, even though all Shapes already share one realm lifetime and allocator.

This is diagnostic attribution, not a public timing claim.

Goal

Request secure entropy once for each root Shape, then derive a distinct keyed SipHash seed for every transition map from that realm-owned secret and a monotonic per-realm domain value. Preserve the current collision-resistant expected search bound without process-global mutable state, cross-realm sharing, predictable public keys, or entropy fallbacks.

Scope

  • Add a root-owned transition-key source whose lifetime exactly matches the Shape tree.
  • Fail Context creation closed when the single secure seed cannot be obtained, matching the current security contract.
  • Derive a separate key for each transition map with explicit domain separation and thread-safe unique allocation under parallel Shape publication.
  • Keep transition lookup, immutable names, release publication, per-Shape writer serialization, exact convergence, and arena teardown unchanged.
  • Add deterministic fixed-seed derivation tests plus concurrent uniqueness/publication coverage.
  • Measure the exact parent with the #661 no-evaluation lifecycle profile and a warm property/shape control; retain exact checksums, finalizer reconciliation, RSS plateau, and Test262 behavior.

No-workaround rules

  • No constant or public seed, weak PRNG, entropy-error fallback, process-global key/counter, shared mutable Shape tree, skipped transition caching, disabled locking, or benchmark-only fast path.
  • No changes to stacks, queues, jobs, Promise, Worker, or Map/Set iterator surfaces.

Acceptance

  • One secure entropy request initializes each root Shape; every transition-map key is uniquely derived within that realm.
  • Fixed-seed tests prove deterministic domain separation and distinct adjacent keys.
  • Concurrent transition publication preserves exact same-name convergence, logarithmic keyed lookup structure, and race-free unique key allocation.
  • Exact-parent cold lifecycle evidence shows the entropy/crypto hotspot removed with identical checksums, finalizers, and bounded retained RSS.
  • Warm shape/property controls do not regress materially.
  • Focused Shape, Context, no-GIL/TSan, full unit, and relevant Test262 gates pass with zero semantic flips.
9 days ago

Implementation landed on main in cb0200bc.

The root Shape now owns one fail-closed 128-bit secret and an atomic non-repeating domain allocator. Each transition head receives a separate two-domain SipHash-derived 128-bit key; Shapes carry the shared owner pointer, so the allocator field also shrinks from every immutable Shape. No process-global or cross-realm mutable state was added.

Current validation:

  • standalone Shape suite: 29/29
  • full sharded units: 1,750 passed, 1 skipped, 0 failed/leaked
  • focused TSan: concurrent domain allocation and same-name convergence pass
  • Test262 test/built-ins/Object: 3,411 -> 3,411 pass, 0 fail, flips 0
  • matched sampled 30,000-context diagnostic: create 24.516470460 s -> 18.647319643 s; total 28.483257526 s -> 22.611010675 s; platform AES leaves 158 -> 0; exact 75,870,000 finalizers and bounded retained RSS

The timing sample is diagnostic. The exact-parent lifecycle and warm controls remain open before the performance acceptance boxes are checked.

9 days ago

Fresh pushed CI for cb0200bc is green on the relevant repository-wide gates:

The guarded exact-parent cold collection twice passed an initially quiet preflight and then rejected when a separate Home/Bun Zig test graph appeared between pairs. Neither attempt wrote JSON or Markdown output, and no competing process was modified. Formal cold/warm evidence remains pending a sustained exclusive host window; diagnostic samples are not being promoted.

9 days ago

Cold exact-parent evidence landed in d7f0d2ff: raw JSON and generated report.

Seven order-balanced pairs measured the cold production Context create / exact destroy and finalization boundary. Candidate median is 702.634 ms versus 883.977 ms for the exact parent (0.795x; 1.30% / 2.39% RSD). Retired instructions are 0.887x, cycles 0.796x, and process energy 0.843x, with stable counters and nominal-to-nominal thermals throughout. Peak and retained RSS are both 0.998x. Every checksum and the exact 2,529,000-cell finalizer count match; no sample was discarded.

This completes only the cold-evidence box. The formal warmed transition-fanout control remains pending. The superseded implementation CI run also exposed the independently tracked Map/Set iterator TSan race, so the comprehensive no-GIL/TSan acceptance box remains open while the current-head matrix runs.

9 days ago

Current descendant evidence update: CI run 32485314750, which retains cb0200b, passed all ordinary unit shards, focused-engine-tests, Test262 parallel, docs/dependency gates, threadfuzz amplified/broad/verify, ReleaseSafe threadfuzz, and two TSan threadfuzz profiles.

It is not a comprehensive green matrix: the exact observed failures are the known #643 Map iterator TSan race plus two thread/lifecycle corpus cases (mc-lock-stop-vs-park and oom-one-thread), and later pushes superseded the run. The comprehensive box remains open. The warm shape/property control also remains pending; no timing claim or acceptance checkbox is being promoted from this CI evidence.

9 days ago

Two unblocks landed/identified: #643 is fixed (04d46c59), removing the Map iterator TSan race that kept descendant matrices red. The remaining corpus failures now have their own tracking issue: #722 (`mc-lock-stop-vs-park`, `oom-one-thread`). Once #722 lands, a single green matrix can close the comprehensive box here.

9 days ago

Warm-control preparation is complete without making a performance claim. Frozen parent b815b3e7 and candidate cb0200bc comparison runners were rebuilt with Zig 0.17.0-dev.1441; SHA-256 identities are db2fe8a356fbf73e60ac5ec78f7b99d642c26997787c07f4e057d2f9629695af and 24d5eaeb1e9df9165b44551f5d8950b900e679a3b04b5e829e7eb9983356f2fe. The embedded comparison and representative sources are byte-identical (7a1a5bef… / 94d9463c…), and a one-job representative_shape_transition_fanout contract probe returns checksum 2 on both revisions. The host remains on battery and intermittent Home tests are active, so elapsed diagnostics are discarded and no exact warm sample has started.

9 days ago

Comprehensive integration is now green in main CI run 32923883777: 53 required jobs passed, the one nightly-only job was skipped as configured, and 0 jobs failed or were cancelled. This closes the focused/full unit, Test262, no-GIL/TSan, threadfuzz, docs, ABI, Wasm, and benchmark integration box for the landed #662 implementation. The warm shape/property control remains open; the current host is AC/nominal but has an unrelated unit process and system indexing load, so no timing sample is being started or retained.

9 days ago

Completed and published the remaining warmed Shape/property control.

The seven-pair quiet-reference exact-parent warm control compares cb0200bc directly with b815b3e7 over 100,000 prepared Shape transitions. It passes the frozen 10% material-regression gate:

  • wall: 1.0275x
  • retired instructions: 1.0021x
  • cycles: 1.0190x
  • process energy: 1.0153x
  • checksum: exact (5,000,150,000)
  • thermal state: nominal throughout
  • all wall and efficiency series: stable

The existing cold lifecycle artifact remains the finalizer/retained-RSS evidence boundary; the warmed mixed-tier row does not claim allocation equivalence. Schema-v3 validation, all 314 documentation links, the 50-page BunPress build, and git diff --check pass. The publication commit is verified at origin/main and closes #662.

Sign in to comment on this issue.