ReviewOS

zig-utils/zig-js

Environments: elide private writes when concurrent tracing is impossible

#535
Closed chris opened this 9 days ago ·
9 days ago

Problem

Issue #533 safely removed read locking from uncaptured mutator-private activations, but every private binding write still acquires Environment.binding_lock. Exact schema-next attribution for representative_shape_transition_fanout at 8 lanes / 1,000 jobs accounts for all 72,531 invocation acquires:

  • 16,334 reads (16,238 root, 96 captured)
  • 56,197 writes
  • 0 concurrent-trace acquires
  • 56,191 of the writes are uncaptured private-activation writes; only 6 are root writes

Thus private writes are 99.99% of Environment writes and 77.47% of all Environment locking in this workload. The context uses parallel mutators with mid-script collection disabled, so no marker can trace activation tables concurrently, yet the process-global lock policy cannot express that fact.

Required design

Give each Environment an immutable/inherited synchronization capability derived from its owning Context. An uncaptured private activation may elide binding-write locking only when concurrent environment tracing is impossible for that Context. Root, module, captured/shared, standalone concurrent-marker, and parallel mid-script collector paths must remain locked. Closure publication must continue to revoke private access before another mutator can observe the environment.

This must be a general ownership/capability rule, not a workload check or global flag workaround.

Acceptance

  • Unit tests prove private write elision and capture revocation with exact lock attribution.
  • TSan covers shared closure mutation and concurrent marker/parallel collector configurations.
  • threadfuzz stays deterministic and race-free.
  • Full unit suite passes without leaks.
  • Exact affected test262 subtree comparisons show no regressions.
  • Re-measure #531 1/2/4/8 scaling and report raw checksums/medians honestly; keep #531 open unless its acceptance is met.

Related: #531, #532, #533.

9 days ago

Diagnostic prerequisite landed in 3577d307.

One safety refinement from the collector audit: a default no-GIL Context can still enter driveParallelCollection during allocation-failure recovery, even with routine parallel mid-script GC disabled. Therefore an immutable “no concurrent tracer” bit alone is insufficient. The implementation must use a race-free dynamic marker/mutator handshake: publish trace-active before tracing, drain already-entered private writes, and force later private writes through binding_lock until concurrent tracing finishes or aborts.

9 days ago

Implemented and landed in 2d895edb.

The implementation refines the initial static-capability design into a heap-scoped active-trace capability. Every private write enters a seq_cst per-interpreter guard before reading the shared heap's trace flag. A collector publishes that flag, then drains pre-existing writers across the owner, live siblings, bootstrapping realms, and retiring realms before it can trace an Environment. Writers arriving after publication use the existing binding mutex. This is necessary because a precise heap collector traces all registered realms, not only the Context that initiated collection.

Exact 8-lane attribution for representative_shape_transition_fanout, 1,000 jobs:

  • Environment lock acquisitions: 72,531 -> 16,340 (-77.47%)
  • private-write elisions: 56,191
  • remaining writes that acquire the lock: 6 root writes
  • reads that acquire the lock: 16,334
  • checksum: 18,026,000 before and after

Validation on the landed code:

  • full unit suite: 1,591 passed, 1 skipped, 0 failed, 0 leaked
  • TSan: shared-precise-realm writer drain, standalone concurrent marker, parallel collector, and shared closure mutation witnesses pass
  • threadfuzz default: 400/400, 0 failures
  • threadfuzz exact-value verifier: 300/300, 0 failures
  • ReleaseFast test/language: 23,648 -> 23,648, 0 flips and byte-identical diagnostics
  • attribution schema self-test passes

Capture revocation, root environments, shared/captured environments, and active concurrent tracing remain on the locked path. #531 has the raw scaling samples and remains open because throughput still declines after two lanes.

Sign in to comment on this issue.