ReviewOS

home-lang/home

Native task ownership: discard queued C++ payloads safely during VM shutdown

#465
Open chris opened this 8 days ago ·
8 days ago

Follow-up to #66, #461, and #462. The MessagePort close-lifecycle audit found broader native task cleanup debt: ScriptExecutionContext creates EventLoopTask allocations whose captured references are released only by performTask/deleteEventLoopTask. Worker ticks stop before final GC; VirtualMachine.deinit does not drain or cancel remaining event-loop payloads. EventLoop.deinit frees containers but not queued CppTask captures/concurrent nodes. enqueueTaskConcurrent and its batch variant also return on has_terminated without releasing already-allocated task ownership, despite a comment claiming no leak.

The targeted #462 fix prevents locally generated teardown close drains from entering a stopped loop; it does not solve remote work already queued during shutdown. Implement an explicit cancellation/ownership protocol, destroy thread-affine native captures on their owning thread, synchronize producer admission with shutdown, and never execute user JavaScript in a stopped VM. Avoid freeing arbitrary captures inside the global contexts-map lock (destructors may post/close recursively). Add bounded repeated-worker tests with remote queued native work, exact destructor/ownership counters where feasible, natural process exit, and no relaxed memory thresholds. Preserve live peer notifications and existing NAPI finalizer ordering. This issue is an audited gap, not a claim that all native queued task types are safely cancellable today.

8 days ago

A small deterministic native probe now confirms the queued-transfer leak: one worker installs a message handler that closes a received port, then reports a timer callback is blocked in a finite 500 ms Atomics.wait. The parent queues one transferred MessagePort and terminates the worker before delivery. Its surviving peer never receives close after worker exit and bounded GC settling; the normal drained counterpart does. No large allocation or RSS threshold is involved.

The implementation audit also found prerequisites beyond deleting queue storage: cleanup-only DeleteCallbackDataTask needs its cleanup action; thread-affine callback/Strong handles must be released on their owner before JSC teardown; Worker dispatchExit has create-time reference cleanup inside a callback that cancellation would discard; heap-snapshot cancellation has a parent-owned promise handle. A correct fix needs producer admission/quiescence plus both FIFO and concurrent C++ task disposal, preserving unrelated task tags and avoiding destructor calls under registry/queue locks. Local Bun Rust has useful partial code, but copying it blindly would miss these cases. #466 pending-close semantics is being implemented independently; #66 remains incomplete.

8 days ago

Ownership clarification from the linked-object audit: the active native Worker.cpp comes from external UnifiedSource-src_jsc_bindings_webcore-5.cpp, not the stale Home mirror. Its dispatchExit object and current home-debug contain the pre-enqueue hook and three-argument postTaskTo overload that already release the create-time C++ Worker reference. Therefore that specific reference bug is a mirrored-source ownership gap, not the demonstrated runtime cause or prerequisite here. Callback-only parent-poll release and other thread-affine cleanup still need cancellation handling. The one-worker queued-transfer probe continues to fail independently on the new #466 binary, with normal delivery passing; it is now preserved as tests/runtime/worker-queued-transfer-close.pending.mjs, not skipped or counted as a pass.

8 days ago

Audit update: splitting incoming Worker inbox disposal from general task cancellation. The new retained-Worker regression fails on the current Home binary (surviving peer close count 0), while its Node control passes; this does not require wrapper GC. An owned Worker.cpp fix and native ownership/build wiring are being verified now. General CppTask cancellation remains open: CleanupTask must execute owner-thread cleanup (DeleteCallbackDataTask), off-thread JSAbortAlgorithm currently reaches the local queue, parent-poll release is callback-only, and getHeapSnapshot captures a raw parent-owned Strong promise handle. Native work-pool jobs are another lifetime family and need quiescence before VM destruction. Do not treat an inbox regression pass as proof that all native queued allocations are released.

8 days ago

The observable queued-transfer leak is fixed locally by the inbox ownership checkpoint 6de9852 (#469), and its old pending probe is superseded by seven passing inbox regressions. This does not reclaim generic queued CppTask allocations. Keep this issue open: safe admission, owner-thread cleanup tags, parent-poll cleanup, and producer synchronization still need implementation. Related explicit gaps: #468 native work-pool quiescence, #470 nested termination, #471 snapshot settlement. Full verification and local-only publication status are recorded in #66.

8 days ago

Bounded ownership progress is complete in #565 and pushed to main as 71c8ff953aab47c918a4f79ebf87592c736c0508.

For workers, ScriptExecutionContext identifier-based admission now closes under its contexts-map lock before JSC teardown. Every admitted C++ task in the local FIFO or concurrent queue is then handled on the owner thread: cleanup-tagged tasks perform their cleanup action, ordinary callbacks are deleted without execution, and cleanup-generated C++ work is processed to a fixed point. The exact native regression covers both queues and distinguishes callback execution, deletion, cleanup dispatch, and cleanup action execution.

Keep #465 open. This tranche does not make enqueueTaskConcurrent() safe for every raw VM producer after has_terminated, cancel arbitrary non-C++ Zig task tags, or prove that producers holding raw VM/event-loop pointers have quiesced. In-flight EventLoopTaskNoContext jobs remain #468. The completed evidence and the one independent aggregate baseline failure are recorded in #565/#566 and #66.

8 days ago

Checkpoint landed in 72415f9f9 (fix(runtime): cancel concurrent promise tasks).

This extends the #468 per-VM native work-pool admission/barrier to the Zig ConcurrentPromiseTask family. A task is now admitted before scheduling, remains counted until its worker body has enqueued completion, and is cancelled on the JS owner thread after shutdown closes admission and joins producers. Cancellation explicitly releases the promise/keepalive wrapper and each context's native or JS-thread-affine ownership for the current four instantiations:

  • async glob walking
  • image pipeline work
  • Bun.Transpiler.transform()
  • file-to-file Bun.write() copy work

The event-loop shutdown pass now dispatches only task tags with an explicit cancellation contract and retains unknown families. The file-copy completion path also now destroys its context on normal success/error, fixing the adjacent retained-context ownership gap without double-releasing its stores.

Regression coverage adds four bounded real worker cycles. Each worker arms a 16 MiB async transpile, proves one admitted native job, terminates, and requires the exact concurrent-promise cancellation counter to advance by one. The same test exercises normal file-to-file completion and verifies its byte count and contents.

Exact-source validation:

  • focused concurrent-promise shutdown regression: pass
  • all real-runtime regressions: 40 passed, 0 failed
  • native home_rt: 1,815 passed, 19 skipped, 0 failed (22/22 suites)
  • targeted Zig formatting, Pickier, and git diff --check: pass

#465 remains open. Other event-loop tags and direct/WorkTask work-pool producers still need bounded producer admission plus owner-thread cancellation audits.

While constructing the transform regression I found a separate deterministic cross-VM correctness crash after a failed/caught main-VM transform; pinned Bun passes the same sequence. That is isolated in #567 because it crashes inside the worker transform before completion enqueue or shutdown cancellation.

8 days ago

Direct work-pool checkpoint landed in eb04c8f75 (fix(runtime): cancel queued pbkdf2 jobs).

AnyTask now supports an explicit, opt-in owner-thread shutdown callback. Unannotated AnyTask payloads remain retained for later audits; the queue does not guess that arbitrary opaque contexts are safe to destroy.

The async native PBKDF2 job now:

  • enters the per-VM native work registry before WorkPool.schedule
  • stays admitted until its worker body has enqueued the completion task
  • exposes an owner-thread cancellation callback that releases the protected password/salt, result buffer, strong promise, keepalive, and job allocation
  • captures the promise before handling closed-admission cancellation

The regression verifies the RFC 6070 normal-completion vector and three bounded worker shutdown cycles at 16 million SHA-256 iterations. Each cycle proves one admitted job and exactly one AnyTask shutdown cancellation.

Exact-source validation:

  • focused PBKDF2 shutdown regression: pass
  • all real-runtime regressions: 41 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • targeted Zig formatting, Pickier, and diff hygiene: pass

#465 remains open for the other direct work-pool AnyTask producers, the three WorkTask instantiations, and unaudited event-loop tags.

8 days ago

Generated crypto work-pool checkpoint landed in f4110e49d (fix(runtime): cancel queued crypto jobs).

Both native crypto job generators now use the per-VM admission/barrier and the opt-in AnyTask owner-thread shutdown callback:

  • Zig CryptoJob contexts (including scrypt and async random fill)
  • opaque C++ ExternCryptoJob contexts (prime/key/sign/DH/HKDF families)

Every job is admitted before WorkPool.schedule, remains counted until its completion task is queued, and releases its context, callback strong reference, keepalive, and allocation on the JS owner thread without invoking stopped JavaScript.

The focused regression covers normal async scrypt parity against scryptSync, three high-cost scrypt worker shutdowns, and two 2048-bit generatePrime worker shutdowns. It passed five consecutive focused runs; every shutdown proved one admitted native job and exactly one audited AnyTask cancellation.

Exact-source validation:

  • focused crypto-job shutdown regression, five consecutive runs: pass
  • all real-runtime regressions: 42 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Zig formatting, Pickier, and diff hygiene: pass

#465 remains open for JSSecrets, password hash/verify, Zstd, filesystem/direct work-pool producers, the three WorkTask instantiations, and remaining unaudited event-loop tags.

8 days ago

Zstd work-pool checkpoint landed in f505b6c2d (fix(runtime): cancel queued zstd jobs).

Bun.zstdCompress() and Bun.zstdDecompress() now enter the per-VM native work registry before scheduling, stay admitted until completion enqueue, and use the audited AnyTask owner-thread shutdown callback. Cancellation releases the protected input, output allocation, strong promise, keepalive, and job without entering stopped JavaScript. Closed-admission paths capture the returned promise before releasing the native wrapper.

The regression verifies a normal async compress/decompress round trip and three bounded worker shutdown cycles using 16 MiB of high-entropy input at compression level 22. Each cycle proves one admitted job and exactly one audited owner-thread cancellation; the focused test passed three consecutive complete runs.

Exact-source validation:

  • focused Zstd shutdown regression, three consecutive runs: pass
  • all real-runtime regressions: 43 passed, 0 failed
  • clean-cache native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Zig formatting, Pickier, and diff hygiene: pass

The first native-gate attempt was blocked before test execution by a full disposable .zig-cache; clearing only that isolated 6.0 GiB cache recovered 13 GiB, and the clean-cache rerun produced the passing result above.

#465 remains open for JSSecrets, password hash/verify, filesystem/direct work-pool producers, the WorkTask family, and other unaudited task tags.

8 days ago

Password work-pool checkpoint landed in 11f306e11 (fix(runtime): cancel queued password jobs).

Async Bun.password.hash() and verify() now participate in the per-VM native work registry. Their worker wrappers transfer the strong promise and keepalive into result AnyTasks; those result payloads now opt into owner-thread shutdown cancellation. Hash cancellation frees the generated hash bytes, while both paths release the transferred promise, keepalive, and result allocation without invoking stopped JavaScript. The native admission remains held until the result task has been enqueued.

A focused run exposed and corrected a thread-affinity error in the draft: the original worker wrapper must not call global.bunVM() when it destroys itself on the pool thread. Closed admission now happens before that wrapper is ref'd, and normal worker completion transfers/clears the ref before destruction; only the owner-thread result releases it.

The final regression covers normal async hash+verify, three cost-11 bcrypt hash shutdowns, and two upstream cost-10 bcrypt verify shutdowns. It passed three consecutive focused runs with exact one-job/one-cancellation assertions for every worker.

Exact-source validation:

  • focused password shutdown regression, three consecutive runs: pass
  • all real-runtime regressions: 44 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Zig formatting, Pickier, and diff hygiene: pass

#465 remains open for JSSecrets, filesystem/direct work-pool producers, the three WorkTask instantiations, and remaining unaudited task tags.

8 days ago

Archive task checkpoint landed in 7124c4d78 (fix(runtime): cancel queued archive tasks).

The generic native archive wrapper now gives all four task contexts one per-VM admission/barrier and owner-thread cancellation contract:

  • archive blob/bytes materialization
  • archive file-map reading
  • archive writing
  • archive extraction

The job remains admitted until its completion task is enqueued. Shutdown dispatch now explicitly recognizes the four archive task tags; cancellation releases the context-specific store/path/pattern/data/result ownership, strong promise, keepalive, and wrapper without entering stopped JavaScript. Unrecognized task tags remain retained.

The regression's normal path exercises bytes(), files(), Archive.write(), and extract() with content checks. Three bounded worker cycles compress 16 MiB of high-entropy archive input at gzip level 12, prove one admitted job, terminate, and require the exact archive cancellation counter to advance once. The focused test passed three consecutive runs.

Exact-source validation:

  • focused archive shutdown regression, three consecutive runs: pass
  • all real-runtime regressions: 45 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, Pickier, diff hygiene, and targeted formatter audit: pass

#465 remains open for JSSecrets, filesystem/direct work-pool producers, the three WorkTask instantiations, and remaining unaudited task tags.

8 days ago

Runtime module-transpiler checkpoint landed in e895ad216 (fix(runtime): cancel queued transpiler jobs).

RuntimeTranspilerStore.TranspilerJob now participates in the per-VM native work-pool admission/barrier. A job remains admitted until its worker has published the completed result into the VM-owned transpiler queue and enqueued the store task. Shutdown then recognizes the RuntimeTranspilerStore task tag and drains that queue on the owner thread without fulfilling module promises or entering stopped JavaScript.

Cancellation releases the keepalive, strong promise, input specifier/referrer, path, fetcher, log, generated source strings, optional module metadata, optional bytecode buffer, and Hive slot. Duplicate store notifications are safe because the first notification drains the queue and later notifications see it empty.

The regression imports a normal TypeScript module and checks its value, then runs three worker cycles against a 16 MiB TypeScript module. Each cycle proves exactly one admitted native job before termination and requires the runtime-transpiler cancellation counter to advance exactly once.

Exact-source validation:

  • focused regression: three consecutive stress runs plus a post-rebase run passed
  • all real-runtime regressions: 46 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Pickier/Zig formatting, and diff hygiene: pass

#465 remains open for JSSecrets, filesystem/direct work-pool producers, the three WorkTask instantiations, and remaining unaudited task tags.

8 days ago

Libc DNS WorkTask checkpoint landed in e84f117dc (fix(runtime): cancel queued dns work tasks).

Generic WorkTask now has an opt-in per-VM admission/barrier and owner-thread cancellation path. The libc GetAddrInfoRequest is the first opted-in context: it remains admitted through completion publication, balances debugger scheduling on cancellation, and shutdown recognizes the queued task tag without entering stopped JavaScript.

DNS cancellation removes the pending host-cache slot, retires the resolver timer/reference, releases the libc query or result list, deinitializes the embedded and chained strong promises/keepalives/resolver references, and destroys the request plus wrapper. Blob read/write stay on the original untracked schedule() path because they can park on secondary I/O poll callbacks and require a separate shutdown signal.

The regression verifies a normal forced-libc Bun.dns.lookup, then uses 1,024 held native work-pool probes (the maximum configured pool width) ahead of the DNS job. Two worker cycles prove exactly 1,025 admitted jobs, prove termination blocks before release, and require exactly 1,024 probe completions plus one WorkTask cancellation.

Exact-source validation:

  • focused regression: repeated stress runs passed
  • all real-runtime regressions: 47 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Pickier/Zig formatting, and diff hygiene: pass

#465 remains open for JSSecrets, Blob read/write poll-aware WorkTask lifecycle, filesystem/direct work-pool producers, and remaining unaudited tags.

8 days ago

Bun.secrets activation and shutdown checkpoint landed in d1823dd97 (fix(runtime): activate secrets jobs).

Home's native Bun.secrets host functions were present, but Bun__Secrets__scheduleJob still resolved to a no-op stub. Calls returned promises that admitted no native work and never settled. The executable root now force-links the real Zig scheduler and the stub is removed.

SecretsJob now participates in the per-VM native work-pool barrier. Scheduling closes cleanly when admission is shut down; native completion is published before the barrier decrements; and queued completion uses an AnyTask shutdown callback that destroys the opaque C++ options (including its secure buffer zeroing), strong promise, keepalive, and Zig job without entering stopped JavaScript.

The new regression waits until every configured work-pool thread is occupied by a held probe before scheduling a read-only missing-secret lookup. Two worker cycles prove exact thread-count + 1 admission, prove teardown blocks until native jobs are released, and require every probe to finish plus exactly one AnyTask cancellation per cycle. The DNS WorkTask regression now uses the same fully-started handshake, replacing the expensive 1,024-probe ceiling without weakening saturation.

Rebased exact-source evidence:

  • direct missing-secret smoke: one admitted native job, resolves null, returns to zero jobs
  • repeated focused DNS and secrets shutdown stress: pass
  • real-runtime aggregate: 48 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Pickier/Zig formatting, and diff hygiene: pass

#465 remains open for poll-aware Blob read/write WorkTask lifecycle, filesystem/direct work-pool producers, and other unaudited native task tags.

8 days ago

Allocation-free Node FS work-pool checkpoint landed in 550e8a05b (fix(runtime): cancel queued fs tasks).

The shared POSIX NewAsyncFSTask wrapper now opts all allocation-free result shapes (void, bool, and the Null sentinel) into per-VM work-pool admission and owner-thread cancellation. This covers 25 async APIs: access/exists, file writes/copy/close, and the path/fd metadata mutation family.

Each admitted job publishes an AnyTask completion before releasing the VM barrier. Shutdown balances debugger scheduling and releases cloned errors, thread-safe/unprotected arguments, keepalive, strong promise, and the task without entering stopped JavaScript. Rejected admission is safe before a result exists. Complex string/buffer/directory/path/stat results remain on the original path until their native result destructors are explicit.

The regression validates normal writeFile/access/exists, then fully occupies every configured work-pool thread before queuing one task of each admitted result shape. Two worker cycles prove exact thread-count + 3 admission, blocked teardown before release, complete probe retirement, and exactly three AnyTask cancellations.

Evidence:

  • repeated focused stress: pass
  • real-runtime aggregate: 49 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, targeted Pickier, and diff hygiene: pass

#465 remains open for complex-result FS tasks, recursive cp/readdir task graphs, poll-aware Blob read/write tasks, and other direct producers.

8 days ago

Value-only stat ownership and native StatFS checkpoint landed in a439ceb92 (fix(runtime): own stat filesystem tasks).

The shared POSIX NewAsyncFSTask wrapper now admits the remaining value-only stat payloads: Stats, StatOrNotFound, and StatFS. This adds owner-thread shutdown cancellation for fstat, stat, lstat, and statfs while continuing to exclude strings, buffers, directory arrays, returned descriptors, and other results that need explicit cleanup semantics.

This checkpoint also replaces Home's placeholder StatFS payload and zero-returning libc shim with the real macOS/Linux/FreeBSD layouts plus the libuv Windows layout. Per-target size/alignment/field-offset assertions are compiled for all four ABI families. fs.promises.statfs() now reports real block and file counts instead of an all-zero object.

The existing saturated-pool regression now queues access, exists, writeFile, stat, lstat, fstat, and statfs. Two worker cycles require exact thread-count + 7 admission, blocked teardown until every probe retires, and exactly seven owner-thread cancellations.

Evidence:

  • repeated focused worker-shutdown stress: pass
  • real-runtime aggregate: 49 passed, 0 failed
  • native home_rt: 1,816 passed, 19 skipped, 0 failed
  • debug build, Pickier, Zig formatting, diff hygiene: pass
  • standalone ABI compilation: macOS, Linux, FreeBSD, Windows pass

Validation also reproduced a pre-existing, state-dependent JSC AtomStringImpl::remove GC assertion seen in an August 30 crash report. The exact named test passed 20/20 and a clean full retry passed; it is tracked separately in #568.

#465 remains open for owned string/buffer/directory results, returned-FD cleanup, read/write buffer lifetimes, recursive cp/readdir graphs, Blob poll lifecycles, and other direct producers.

8 days ago

Directory-result shutdown ownership checkpoint landed on main in 7c818831b, audited against pinned Bun 4982b91e3702094330f3be3883354c52b8c01323.

What changed:

  • admitted async recursive mkdir (StringOrUndefined) and mkdtemp (jsc.ZigString) into the VM native-work-pool barrier;
  • added cancellation-only result destruction, dereferencing the returned bun.String or releasing the globally allocated ZigString after worker completion and before task teardown;
  • corrected ZigString.deinitGlobal() to free UTF-16 allocations as []u16 rather than routing them through the Latin-1 slice() path;
  • extended the deterministic saturated-pool worker regression from 7 to 9 filesystem jobs, with exact admission/cancellation counts across two teardown cycles;
  • covered normal recursive mkdir return semantics and normal/cancelled mkdtemp with a non-ASCII prefix so the UTF-16 ownership branch is exercised.

Validation on the final minimal diff:

  • zig build debug -j1: pass
  • focused worker teardown regression: pass (7 consecutive standalone runs, plus aggregate coverage)
  • HOME_BIN=./zig-out/bin/home-debug scripts/runtime-regression.sh: 49 passed / 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed / 19 skipped / 0 failed (two clean full runs)
  • targeted Pickier: pass
  • git diff --check: pass

The next ownership-sensitive FS results remain StringOrBuffer (readFile/readlink/realpath), Readdir arrays, and returned Open file descriptors; each needs its own cancellation destructor before admission.

8 days ago

StringOrBuffer filesystem result ownership is now landed on main in 3124040cf, audited against pinned Bun 4982b91e3702094330f3be3883354c52b8c01323.

What changed:

  • added a result-only StringOrBuffer.deinitForShutdownResult() path, kept separate from argument cleanup because argument buffers may borrow protected JS typed arrays;
  • cancellation now deinitializes SliceWithUnderlyingString variants, frees encoded slices, and calls MarkedArrayBuffer.destroy() for native owned byte buffers;
  • admitted the shared StringOrBuffer return type into the VM native-work-pool barrier, covering async readFile, readlink, realpath, and realpath.native results;
  • added normal-result content checks for buffer and UTF-8 string variants of readFile, readlink, and realpath;
  • expanded the saturated-pool worker regression from 9 to 15 exact FS admissions/cancellations per cycle, covering all six new buffer/string result paths across two worker teardown cycles.

Validation:

  • zig build debug -j1: pass
  • focused worker teardown regression: pass (6 consecutive standalone runs)
  • HOME_BIN=./zig-out/bin/home-debug scripts/runtime-regression.sh: 49 passed / 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed / 19 skipped / 0 failed
  • targeted Pickier, Zig AST checks, and git diff --check: pass

The next complex FS result owners are Return.Readdir (per-tag arrays and element teardown) and Return.Open (close the returned descriptor when JS conversion is skipped).

8 days ago

Async open result ownership is now landed on main in d3af6452b, audited against pinned Bun 4982b91e3702094330f3be3883354c52b8c01323.

What changed:

  • admitted the POSIX native-work-pool Return.Open (bun.FD) result into the VM shutdown barrier;
  • successful results now call FD.close() when worker teardown cancels the owner-thread task before JS conversion, preventing descriptors from leaking when no FileHandle is created;
  • kept normal JS ownership unchanged: the normal regression opens a promise FileHandle, stats it, and closes it explicitly;
  • expanded the saturated-pool worker regression from 15 to 16 exact FS admissions/cancellations per cycle by adding async open, across two teardown cycles.

Validation:

  • zig build debug -j1: pass
  • focused worker teardown regression: pass (6 consecutive standalone runs)
  • HOME_BIN=./zig-out/bin/home-debug scripts/runtime-regression.sh: 49 passed / 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed / 19 skipped / 0 failed
  • targeted Pickier, Zig AST check, and git diff --check: pass

One initial native compile attempt could not write its outputs while a concurrent ReleaseFast build exhausted disk space; after that build released its temporary space, the clean native retry completed with the result above.

The next complex generic FS result owner is Return.Readdir, which needs per-tag array and element destruction before it can join the same shutdown barrier.

8 days ago

Pushed 480f3a2cfac416f984e5bf5c6ddfc02aa2dbfec8 (fix(runtime): own readdir filesystem tasks) to main.

This advances the native filesystem worker shutdown port against pinned Bun 4982b91e3702094330f3be3883354c52b8c01323:

  • admits nonrecursive async readdir tasks to the VM native work pool shutdown barrier;
  • adds result-only cancellation cleanup for all three owned Return.Readdir graphs;
  • dereferences each Dirent's owned name/path strings, frees each native buffer payload, dereferences each string result, and frees each outer result array;
  • preserves the normal JS transfer paths while preventing abandoned worker results from leaking after the worker realm is gone;
  • extends the regression with normal string, byte-buffer, and Dirent result checks plus two saturated teardown cycles, increasing the exact cancellable filesystem job count from 16 to 19.

Validation:

  • zig build debug -j1
  • focused filesystem shutdown regression: 6 consecutive passes
  • runtime aggregate: 49 passed, 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed, 19 skipped, 0 failed
  • Pickier, Zig AST check, and git diff --check

The commit is authored and committed by Chris Breuer with no coauthors, and the remote main ref was verified at the exact SHA.

8 days ago

Pushed e41e19ae99f7342aed2b8773299440d9903efe63 (fix(runtime): join filesystem io tasks) to main.

This completes the POSIX scalar I/O result family in the native filesystem worker shutdown barrier:

  • admits callback read and readv through their shared value-only Return.Read result;
  • admits callback write and writev through their shared value-only Return.Write result;
  • keeps buffer JS values protected while native work is pending and releases them through the existing argument deinitAndUnprotect() paths after cancellation;
  • keeps cloned readv/writev I/O vector arrays alive through native execution and frees them during task destruction;
  • extends normal-path coverage for positional scalar and vectored reads/writes;
  • increases the saturated teardown regression from 19 to 23 exact filesystem jobs in each of two worker cycles.

Validation:

  • zig build debug -j1
  • focused filesystem shutdown regression: 6 consecutive passes
  • runtime aggregate: 49 passed, 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed, 19 skipped, 0 failed
  • Pickier, Zig AST check, and git diff --check

The commit is authored and committed by Chris Breuer with no coauthors, and remote main was verified at the exact SHA.

8 days ago

Pushed e365672a28e361af78f6aecdbde1adb87acc7b1e (fix(runtime): join recursive readdir tasks) to main.

This brings the custom async recursive readdir graph into the native work pool shutdown protocol:

  • admits one VM barrier entry before scheduling the root scan;
  • holds that entry until the root scan and every dynamically spawned directory subtask have retired, using the graph's existing atomic subtask count as the quiescence boundary;
  • joins the per-thread result queues, publishes a shutdown-aware AnyTask, and only then releases the barrier;
  • cancellation runs on the JS owner thread, balances debugger tracking, destroys the joined/queued string, buffer, or Dirent results, releases the promise/keepalive, and does not execute stopped JavaScript;
  • fixes recursive argument teardown to use deinitAndUnprotect() after toThreadSafe();
  • adds normal recursive string, byte-buffer, and Dirent checks and three saturated recursive jobs, increasing each two-cycle teardown run from 23 to 26 exact filesystem jobs.

Validation:

  • zig build debug -j1
  • focused filesystem shutdown regression: 6 consecutive passes
  • clean runtime aggregate retry: 49 passed, 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed, 19 skipped, 0 failed
  • Pickier, Zig AST check, and git diff --check

During validation, unrelated concurrent sanitizer and ReleaseFast Zig builds reduced free disk below 1 GB. An aggregate attempt produced 10 empty-output failures while this filesystem test still passed; after temporary pressure eased, an individual failed case and the complete 49-test aggregate passed cleanly. No shared cache or other task's artifacts were deleted.

The commit is authored and committed by Chris Breuer with no coauthors, and remote main was verified at the exact SHA.

8 days ago

Recursive fs.cp shutdown checkpoint — f41171f8e

Pushed fix(runtime): join recursive copy tasks to main.

  • The JavaScript async recursive-copy owner now enters native_work_pool_jobs before scheduling its root scan and holds that admission until the root scan and every spawned per-file SingleTask have quiesced.
  • Final dispatch uses a shutdown-aware AnyTask; worker teardown cancels and destroys the promise, debugger tracker, keepalive, arena, cloned error, and thread-safe source/destination paths on the owning thread.
  • Rejected admissions no longer inspect an undefined result, and JavaScript copy arguments now pair toThreadSafe() with deinitAndUnprotect().
  • Shell and mini-event-loop copy ownership remain on their existing paths.
  • The worker regression now performs a successful nested recursive copy, then queues recursive copy behind a saturated native pool. Its exact accounting moves from 26 to 28 public filesystem owners, proves termination remains blocked until pool release, and proves both copy-path owners are cancelled. The focused case passed once and then six consecutive runs.

Validation:

  • zig build debug -j1
  • ./zig-out/bin/home-debug run tests/runtime/native-worker-fs-task-shutdown.test.mjs (1 + 6 consecutive passes)
  • HOME_BIN=./zig-out/bin/home-debug scripts/runtime-regression.sh → 49 passed, 0 failed
  • zig build test -Dfilter=home_rt -j1 → 1816 passed, 19 skipped, 0 failed
  • zig ast-check packages/runtime/src/runtime/node/node_fs.zig
  • bunx --bun pickier tests/runtime/native-worker-fs-task-shutdown.test.mjs
  • git diff --check

Parity reference remains Bun 4982b91e3702094330f3be3883354c52b8c01323.

8 days ago

Landed and pushed the next one-shot native-task checkpoint on main:

Native node:zlib Deflate, Brotli, and Zstd stream writes now enter the per-VM work-pool admission barrier before scheduling. A worker publishes its owner-thread completion before leaving the barrier, so teardown cannot destroy the VM between native work and completion ownership transfer. Queued completions have explicit type-specific shutdown cancellation; the shared cleanup path clears the strong handle, releases both detachment pins and generated GC roots, drops the poll reference, and balances the stream reference without entering stopped JavaScript. Rejected late admission uses the same cleanup path.

The new native-worker-node-zlib-shutdown.test.mjs first completes all three codec families normally, then saturates every native work-pool thread, queues one write per family, proves teardown remains blocked for 50 ms, releases the probes, and verifies exact probeCount + 3 admission, probe completion, and +3 cancellation accounting. It passed once plus six consecutive stress repetitions. The existing compression lifetime regression also passes, preserving input/output pins, independent GC roots, shared-backing pin balance, reentry, and completion cleanup.

Verification at this commit:

  • zig build debug -j1: pass
  • focused shutdown race: 1 pass + 6/6 consecutive repetitions
  • native-compression-lifetime.test.mjs: pass
  • scripts/runtime-regression.sh: 50/50, 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed / 19 skipped / 0 failed
  • focused Pickier, six-file Zig AST checks, and git diff --check: pass
  • origin/main verified at the exact commit above

The remaining WorkTask users were audited. Blob read/write tasks are a different lifecycle: they can park on the process-wide I/O poller and later re-enter the work pool. Holding the one-shot barrier across blocked readiness can deadlock teardown; releasing it permits a late poll callback into a destroyed VM. That boundary is now tracked explicitly in #569 with cancellation/ownership acceptance criteria. Generic queued native-task disposal remains open here. This checkpoint does not claim whole-suite parity or make the umbrella zig build test green.

8 days ago

Landed and pushed the next native task-ownership checkpoint on main:

fs.watchFile initial-stat jobs and periodic scheduler hops now enter the per-VM native work-pool barrier. Each native hop publishes an explicit owner-thread holder before leaving admission. Initial-success, initial-error, file-change, and cross-thread timer-update holders all have shutdown-only cleanup that releases their watcher/scheduler references without entering stopped JavaScript. After admitted jobs quiesce, Worker and opt-in main-VM teardown retire the scheduler while its timer heap is live, close/drain queued watchers, then let the generic queue cancellation pass dispose published completions.

The deterministic regression covers a normal observed file change, BUN_DESTRUCT_VM_ON_EXIT=1 with an outstanding watcher in a child process, and explicit Worker termination while every native pool thread is blocked ahead of an initial stat. It proves teardown stays blocked for 50 ms, then verifies exact probeCount + 1 admission, probe completion, and +1 stat-watcher cancellation. The complete scenario passed 6/6 consecutive repetitions.

The local mirrored watchFile corpus was stale. This commit updates it to the exact pinned Bun SHA 4982b91e3702094330f3be3883354c52b8c01323 bytes (SHA-256 a37d38e083f6087e105753bc1fbbbfa764263194bd1afaf9bf23e1885dbe8b7f). The pinned file passes 9 ordinary tests / 2 platform skips / 0 failures / 30 assertions across four complete runs, including shared watcher identity/stop semantics, the 1,000-watcher stress test, queued-initial-stat Worker termination, and thrown-listener rescheduling. The standalone pinned Node worker fixture is also byte-identical (SHA-256 e73d23c94e861a10d3f87c2b9219768b75096783721c402b0b7d7c3f5d8cda4e) and exits successfully. The two skips are Windows-gated cases on this Darwin host; they are not counted as passes.

Other verification:

  • zig build debug -j1: pass
  • native test-CLI/isolation regression: pass
  • scripts/runtime-regression.sh: 51/51, 0 failed
  • zig build test -Dfilter=home_rt -j1: 1,816 passed / 19 skipped / 0 failed
  • focused Pickier, Zig AST checks, source hashes, and git diff --check: pass
  • origin/main verified at the exact commit above

One earlier unit attempt ended with NoSpaceLeft while writing the executable; after removing stale updater cache, the unchanged command completed with the totals above. This does not make the umbrella zig build test green or establish other-platform coverage. Poll-parked Blob I/O remains #569, and the audited addon-managed napi_async_work ownership boundary is recorded in #459. Generic task-family auditing remains open here; whole-suite parity is not claimed.

8 days ago

Checkpoint landed on main: 5848c3a58fe074612706daf7cdbbf04c68784140 (fix(runtime): quiesce shell work on shutdown).

This closes the safe one-shot shell WorkPool portion of the VM-lifetime audit:

  • ShellMkdirTask, ShellTouchTask, ShellGlobTask, ShellCondExprStatTask, ShellMvCheckTargetTask, and ShellMvBatchedTask now enter the creating JS VM's native-work shutdown barrier before scheduling.
  • Workers publish their owner-thread completion before releasing the barrier.
  • Queued completions have explicit no-JS shutdown cancellation paths that release keep-alive refs, task allocations, errors, owned paths, and result file descriptors as applicable.
  • Mini-event-loop behavior remains outside the per-JS-VM barrier.
  • bun:internal-for-testing exposes an exact shell cancellation counter.

The deterministic regression saturates every native worker thread, queues five distinct shell task types, verifies Worker.terminate() cannot pass the blocked tasks, releases the pool, and observes exactly five cancellations. The shutdown path passed six consecutive repetitions; the final strengthened normal-path form also passed.

Validation:

  • zig build debug -j1
  • exact pinned Bun mv and conditional-expression suites: 11 passed / 0 failed / 31 assertions
  • scripts/runtime-regression.sh: 52 passed / 0 failed
  • zig build test -Dfilter=home_rt -j1: 1816 passed / 19 skipped / 0 failed
  • focused zig ast-check, Pickier, and git diff --check

The recursive/parked cp, ls, and rm graphs require a graph-level protocol and are split into #570. Process-global DNS-cache and installer WorkPool users were also classified in this audit: they do not retain a creating VM/event loop and are not per-VM barrier candidates.

8 days ago

Runtime parity checkpoint: 71a25f41dce29d77cd223614b5b9aef8cacb8d9d joins/cancels the shell cp WorkPool graph during VM teardown and fixes non-pollable regular-file shell capture exposed by the exact upstream suite.

Evidence: pinned Bun cp.test.ts is byte-identical and passes 30/30 with 111 assertions; the deterministic saturated-pool teardown regression passed six consecutive runs; runtime aggregate is 53/53; native home_rt is 1,816 passed, 19 skipped, 0 failed.

Recursive/parked shell follow-up remains tracked in #570 (ls and rm still pending).

8 days ago

Runtime parity checkpoint e0612cfd850718d27e37375af690c1cefc7f541b quiesces every root/recursive shell ls WorkPool task across worker VM teardown.

The deterministic saturated-pool producer-chain test passed six fresh processes; the real-runtime aggregate is 54/54; native home_rt is 1,816 passed, 19 skipped, 0 failed. The byte-identical pinned Bun ls corpus passes every ls assertion (26/27 overall); its sole remaining test times out in nested home install before comparison and is split into #571.

#570 is now narrowed to the remaining rm WorkPool graph.

8 days ago

Runtime parity checkpoint 1611693c74f997e437a2512dd05f45bd2a0c1300 completes the shell rm recursive WorkPool ownership boundary and closes the remaining scope in #570.

Evidence: deterministic worker teardown passed six fresh processes; runtime aggregate is 55/55; current native home_rt is 1,816 passed, 19 skipped, 0 failed. The pinned Bun rm.test.ts mirror is now byte-identical and its four non-install tests pass with 45 assertions, including Bun's new ten-iteration directory-to-symlink TOCTOU race.

The sole filtered test enters the shared nested install fixture tracked in #571.

8 days ago

The exact shell checkpoint is now complete after resolving the external install/test-timeout boundary in #571.

  • Pantry commit: 4d1b539e7
  • Home timeout commit: 6999b4dec
  • exact rm.test.ts: 5 passed, 0 failed, 49 assertions
  • exact ls.test.ts: 27 passed, 0 failed, 81 assertions
  • runtime shutdown/regression suite: 56 passed, 0 failed
  • native home_rt: 1816 passed, 19 skipped, 0 failed

The recursive cp, ls, and rm ownership work remains green with their full pinned Bun command files, including the dependency-heavy fixtures that were previously filtered or timing out.

8 days ago

Pushed 4368f9e (fix(runtime): own bundle tasks through shutdown) to main. The detached Bun.build bundle thread now enters the creating VM native-job barrier before dispatch, publishes its owner-loop completion before releasing that barrier, and uses explicit AnyTask shutdown cleanup for promises, plugins, configuration, output graphs, and keepalives without running stopped JavaScript. Lazy HTML routes now install the completion pointer and route retain before scheduling, removing the prior setup race; cancellation clears the borrowed pointer and balances that retain on the owner thread. The deterministic regression saturates every native pool thread, admits two build jobs in one Worker, proves termination stays blocked for 50 ms, releases the probes, and observes exactly two AnyTask cancellations. It passed once plus six consecutive Debug processes and in ReleaseFast. The installation-shaped ReleaseFast runtime aggregate is 66/66, native home_rt is 1,827 passed / 19 skipped / 0 failed, and the pinned HTML manifest file is 4/4 with 16 assertions and one snapshot. The broader HTML serving corpus is 14/16 in Home versus 16/16 in pinned Bun due two pre-existing over-relative asset URLs, now #675; no snapshot or assertion was weakened. One ReleaseFast attempt was killed when the volume had 169 MiB free; after deleting only recoverable artifacts from this dedicated worktree and waiting for a concurrent Zig job, the unchanged retry passed. Keep #465 open for remaining producer-specific task families and #569 for poll-parked Blob I/O.

Sign in to comment on this issue.