ReviewOS

zig-utils/zig-js

Threads: serialize no-GIL access to embedder allocators

#54
Closed chris opened this 9 days ago ·
9 days ago

Parent: #1 Related: #15, #49

Problem

A preserved pre-rewrite experiment identified a valid invariant gap that remains in the current allocator architecture. Under parallel_gc, arena chunk refills are serialized by LockedArena and GC slab refills are serialized by GcCellBacking, but those are independent locks and both ultimately call the embedder-provided std.mem.Allocator. Zig allocators do not universally promise concurrent safety. Two no-GIL mutators can therefore enter a non-thread-safe embedder allocator concurrently even though each engine subsystem is internally locked.

The stale experiment itself cannot be applied: it targets the removed GC layout and points zig_gc at a deleted marker worktree. This issue tracks a clean adaptation to current main.

Acceptance criteria

  • True-parallel contexts serialize every underlying embedder allocator alloc/resize/remap/free across arena, GC backing, JIT ownership, and context side structures.
  • Single-threaded and explicit-GIL contexts add no allocator lock.
  • Heap-budget accounting and allocation-recovery lock ordering remain correct and deadlock-free.
  • Context create/error/teardown paths free the wrapper after every allocator consumer.
  • Add a concurrent probe allocator test that fails on overlapping underlying calls and passes through the wrapper.
  • Pass focused lifecycle/no-GIL semantic tests and suppression-free TSan.
  • Measure allocator contention after the fix; optimize only if #49 evidence identifies it as material.
9 days ago

Implemented and pushed in 35dff22. True-parallel contexts now install one bottom-level serialized allocator beneath the budget allocator and every context consumer; single-threaded/GIL contexts retain the raw allocator path. The wrapper is destroyed last. Validation: zig build test -Doptimize=ReleaseFast -Dtest-filter=SerializedAllocator — 2/2 passed, 0 leaked. TSan and contention measurement remain deliberately open.

9 days ago

Closing as complete. The serialized bottom allocator from 35dff22 has since passed repeated full lifecycle, no-GIL, and suppression-free TSan gates. Current shared object-churn profiles resolve the remaining hot path to GC allocation metadata publication/hand-off; libc chunk growth and underlying allocator work are negligible, so the acceptance-mandated contention measurement does not justify another wrapper change.

Sign in to comment on this issue.