GC: release reusable empty slab tails under hard-cap pressure

Parent: #480 Related: #637, #645, #650

Evidence

The unchanged 4 MiB torn-object recovery witness passes normal Linux and local TSan, but fresh Linux TSan repeatedly reclaims no usable post-recovery allocation window:

• prior run: https://github.com/zig-utils/zig-js/actions/runs/32388315489/job/96488305083
• current run: https://github.com/zig-utils/zig-js/actions/runs/32394940123/job/96509796504

Both failures complete the pressure phase and full collection, publish no torn object, and leak nothing, but all 500 complete-object attempts catch OOM (made == 0).

Diagnosis

GcCellBacking.trimEmptyTailChunks deliberately retains up to eight empty tail chunks behind each live size-class prefix for task-to-task reuse. That is sensible with ordinary allocator headroom, but under Context.heap_limit_bytes those fully empty slabs still consume the same fail-closed quota needed by reclaimed object side storage. A full collection can therefore free the reserve graph yet strand the returned bytes behind the reusable-tail policy. #637 makes a new final slab adapt to available headroom; it does not return already-empty retained slabs when quota pressure remains high.

Required change

• Give full-collection cleanup a Context-level hard-cap headroom policy.
• Preserve bounded reusable tails when quota headroom is healthy.
• Under high quota utilization, release every empty tail slab so reclaimed bytes are available to complete object publication and side stores.
• Apply the same policy to explicit, quiescent, serialized allocation-failure, and parallel allocation-failure full collections.
• Add deterministic backing-policy coverage plus the unchanged 4 MiB end-to-end witness.

No-workaround rules

No cap increase, reserve inflation, fewer pressure/probe iterations, weaker made > 0 oracle, forced arena mode, skipped TSan, or unconditional cache deletion in healthy heaps. Live slabs and exact realm ownership remain untouched.

Acceptance

• Healthy heaps retain the existing bounded reuse tail.
• Hard-cap pressure releases only fully empty tails and restores usable headroom.
• The unchanged torn-object witness passes normal and TSan with at least one validated post-recovery object.
• Full units have zero regressions/leaks.
• Test262 flips 0 (allocator retention policy only).

Landed fdbdbffd (fix(gc): release empty slabs under cap pressure (#655)).

The bounded allocator now releases every fully empty GC-cell suffix slab after collection when remaining headroom falls below max(limit / 16, 2 * 64 KiB). Healthy heaps retain the normal eight-slab reuse tail; live prefixes and interior holes are unchanged.

Local evidence with Zig 0.17.0-dev.1818:

• focused normal: 3 passed, 0 failed, 0 leaked;
• focused TSan: 3 passed, 0 failed, 0 leaked, no sanitizer report;
• unchanged 4 MiB torn-object recovery witness: 1/1 normal (1.896 s), 1/1 TSan (12.994 s);
• full sharded unit suite: 1,746 passed, 1 skipped, 0 failed, 0 leaked;
• checked-in conformance suite: 33/33 before and 33/33 after (0 flips).

The local test262 checkout lacks its test/ corpus, so I am not claiming a test262 denominator. Keeping this open for the fresh Linux TSan shard that previously reproduced the all-OOM recovery failure.

Follow-up landed in 30d57cc1 (fix(gc): force slab reclaim on allocation retry (#655)).

Final review identified a distinct large-request boundary: an allocation can fail while total remaining headroom is still above the ordinary percentage threshold. Allocation-failure collections now always relinquish every eligible empty suffix before retrying, for both serialized and parallel recovery. Ordinary/explicit collections continue to use the bounded headroom policy and retain the warm tail when healthy.

The cause policy is directly covered for both bounded and unbounded allocators. Focused results on the final code:

• normal: 3 passed, 0 failed, 0 leaked;
• TSan: 3 passed, 0 failed, 0 leaked, no sanitizer report;
• unchanged 4 MiB torn-object witness: 1/1 normal (1.886 s), 1/1 TSan (12.828 s).

A user-owned Home compiler cycle was continuously restarting, so I did not overlap it with another local full suite. The preceding #655 candidate full suite was 1,746 passed / 1 skipped / 0 failed / 0 leaked; the fresh Linux matrix for the final follow-up is now authoritative and this issue remains open pending it.

Fresh Linux TSan on final commit 30d57cc1 still failed the witness: job 96521458116 made zero post-recovery objects after 500 OOMs. I am not treating the local pass as acceptance.

The remaining failure is now isolated to the witness contract and tracked as #657. It nulls the reserve and collects inside the same native evaluation frame that created the graph; conservative stack scanning may legally retain stale reserve pointers on Linux TSan, so the comment that this guarantees reclamation is false. #657 keeps the exact 4 MiB cap and live hoard but moves release and collection across a returned quiescent host boundary. #655 remains open until that corrected invariant and the slab policy both pass fresh Linux TSan.

The quiescent-boundary correction is now on main in 4ad7dd7b via #657. It preserves the exact cap and pressure workload while removing the false in-frame reclamation assumption. The corrected witness passed locally under the exact TSan shard ordering at position 98 after all 97 predecessors. #655 stays open for the new Linux run, which will now distinguish the final slab policy from conservative stale-root retention.

Closed by fdbdbffd, 30d57cc1, and the corrected quiescent witness in 4ad7dd7b. Healthy collections retain the bounded warm tail; hard-cap and actual allocation-failure recovery release only fully empty suffix slabs. Fresh Linux normal shard 3 and TSan shard 3 pass the exact 4 MiB publication oracle with zero leaks. Local full units: 1,748 passed / 1 skipped / 0 failed / 0 leaked.