ReviewOS

zig-utils/zig-js

Memory: add a coordinated host pressure checkpoint

#975
Closed chris opened this 9 days ago ·
9 days ago

Parent: #480 Depends on: #974

Goal

Add one Context-owned host pressure checkpoint that safely reclaims precise-GC/slab and executable-code memory under the existing movement, shared-realm, and execution-epoch protocols, with an exact machine-readable result.

Scope

  • Add a public Zig Context pressure API that enters the GC/JIT conductor once, retires live native artifacts, performs a precise compacting collection when movement is safe, and releases eligible cell-slab tails.
  • Return versioned before/after heap, native mapping, and slab/reclamation fields plus explicit per-domain status; unsupported movement or a non-owner shared realm must remain visible rather than reported as reclaimed.
  • Preserve shared ownership: one precise-heap/JIT owner performs reclamation; sibling realms never double-clear or claim shared bytes.
  • Keep the ordinary execution path unchanged. No polling, RSS heuristic, background thread, or counter on allocation/dispatch hot paths.
  • Feed the result into the memory inventory/benchmark diagnostic boundary without changing scored performance rows.

Acceptance

  • GC-off/JIT-off, private precise-GC, JIT, budgeted, shared-realm owner/sibling, active-reader, allocation-failure, and teardown fixtures have exact status and byte deltas.
  • A successful pressure checkpoint leaves no retired executable mapping once prior readers release, preserves checksums/identities, and trims every eligible empty tail.
  • Unsafe movement fails closed while code retirement remains epoch-safe; no redundant second full collection or repeated recompilation loop is introduced.
  • Disabled production use allocates no pressure state and changes no hot path.
  • Focused JIT/moving-GC, full unit, no-GIL/TSan, ABI, and benchmark/docs gates pass.

No-workaround rules

No RSS estimate substituted for owned bytes, forced process purge, global JIT disable, unbounded retired epoch, duplicated shared-owner reclamation, collection while roots are unsafe, or a success status that hides a skipped domain.

9 days ago

Landed the public checkpoint API in b635141 (feat(memory): add coordinated pressure checkpoint (#975)).

The new Context.relieveMemoryPressure():

  • acquires the existing GC/JIT conductor once;
  • performs at most one precise full compaction and one native-code epoch rotation;
  • returns schema-v1 exact before/after heap, slab, live-code, retired-code, and cumulative reclaimed-code gauges;
  • distinguishes unavailable, observed_shared, unsafe, unchanged, reclaimed, retired_pending, and out_of_memory per domain;
  • leaves shared siblings observation-only and lets the primary act on shared storage;
  • defers native unmapping behind active execution epochs; and
  • adds no persistent hot-path state.

Coverage now includes GC/JIT disabled, budgeted precise GC plus JIT, shared sibling/primary ownership, active evaluation fail-closed movement, active native reader retirement, and deterministic compaction scratch OOM without heap mutation.

Validation:

  • focused linked unit tests: 6/6, 0 leaked;
  • the same focused tests under TSan: 6/6, 0 leaked;
  • standalone context module: 7/7;
  • documentation: 50-page manifest-checked build.

Next I am adding machine-readable checkpoint diagnostics to the representative benchmark path and its validators, without putting the checkpoint on the scored timing boundary.

9 days ago

Completed in three small commits:

  • b635141 — feat(memory): add coordinated pressure checkpoint (#975)
  • 7a887d3 — bench(memory): validate pressure checkpoint results (#975)
  • ee2c184 — test(memory): prove pressure teardown reaches zero (#975)

The final contract provides one public Context.relieveMemoryPressure() call, one GC/JIT conductor acquisition, at most one compacting full collection, one native-code epoch rotation, explicit per-domain status, exact before/after gauges, owner/observer separation, and no new execution-path state.

Acceptance evidence:

  • GC/JIT disabled: exact unavailable results with stable gauges.
  • Private precise GC + JIT + budget: exact heap/slab reduction, exact native reclamation, checksum and protected identity preservation, and coherent heap-budget totals.
  • Shared realm: sibling is observation-only; primary owns the action; both realm globals survive.
  • Active evaluation: moving collection reports unsafe without mutating the active heap.
  • Active native reader: code becomes retired_pending, remains mapped until release, then reaches zero retired bytes.
  • Allocation failure: compaction scratch OOM reports out_of_memory without root, live-byte, or slab mutation.
  • Teardown: after pressure and native recompilation, Context backing, collector auxiliary bytes, recovery reserve, live/retired mappings, and reconciled owned bytes all reach zero.
  • Reclamation semantics: collection-only live-byte or empty-tail reduction reports reclaimed even when relocation counts remain zero.
  • Diagnostic boundary: representative matrix V41 / sidecar schema 15 attaches one pressure result after the invocation attribution and process snapshots; configuration/warmup remain null and scored/replayed fields are unchanged.

Validation:

  • focused pressure tests: 6/6 normally and under TSan, zero leaks;
  • teardown extension: 1/1 normally and under TSan, zero leaks;
  • full unit suite: 2,489 passed, 1 platform skip, 0 failed, 0 leaked;
  • no-GIL TSan parallel_js: 74/74 passed, 0 failed, 0 leaked;
  • focused JIT: 430 passed, 1 platform skip, 0 failed, 0 leaked;
  • relocation inventory: 9 cell kinds, 27 pointer surfaces, 21 boundary tags;
  • C API audit: 117/117 complete and present; C API tests passed;
  • representative matrix structural tests: 79/79;
  • tier-attribution validator self-test passed;
  • complete quick V41 sidecar: 510/510 snapshots, 170/170 pressure rows, zero unsafe/shared/pending/OOM results;
  • live JIT, no-JIT, shared, and module attribution probes passed;
  • documentation: manifest-checked 50-page build.

No JavaScript semantics changed, so test262 flips are not applicable.

Sign in to comment on this issue.