ReviewOS

home-lang/home

Bun parity: implement real VM per-context memory accounting

#870
Open chris opened this 9 days ago ·
9 days ago

Native VM memory attribution checkpoint (2026-10-12): memory source, native distinct-root repair and verified evidence. Native full-collection analysis records JSC marked graph nodes/references and realm reachability, with engine size estimates plus deduplicated backing allocations. Indexed array capacity corrects the first failed unequal-allocation control. Exclusive/shared/unattributed weights produce estimates and ownership ranges instead of duplicating the whole heap for every context.

Default requests wait for full collection; eager requests trigger it. Results deliver asynchronously on the owning context, overlapping requests retain independent promises, and native requests retire on process/worker/test-realm shutdown. Unequal-allocation, collection timing, overlap and shutdown controls pass. Mixed-root native dispatch now preserves caller context and distinct roots; the original regression arguments/assertions are restored and pass.

54/54 Debug steps; 27 native groups; 55 unchanged pinned originals: 179 passed cases plus thirty-eight process checks, zero failures/skips/filters and two TODOs. Fourteen ABI controls (452 assertions) and 32 report-validator controls pass. All 1676 artifacts and 251 replayed report outcomes validate, retaining 26 negatives. Ten source hashes match tested bytes; explicit Pickier scans retain one existing error and eleven warnings, zero new findings. Formatting/docs paths pass. Allocation, compile, disk, stale-cache and both mixed-root repair failures remain visible.

Separate lifetime failure remains: evaluated contexts stay reachable after references are dropped. An unused context collects; a control that never calls measureMemory and installed Bun both retain evaluated contexts. The graph keeps that live row and the unchanged retirement assertion remains failed under #885. #870 stays open. Copy-on-write/shared backing proof, every memory type, profiler interactions and platform/Release/ASAN/full-suite proof remain incomplete. General Promise/native producers/transfer #877, directory #878, corpus #876, context #871/#856, performance #869/#873/#874/#872 and HTTPS #875/#532 remain open. Two TODOs receive no pass credit; known crash controls were not rerun.


Source 5808f41 brings the pinned VM module and its connected native context/Script/module implementations into Home ownership. The selected ordinary conformance scope passes, but vm.measureMemory() is not complete.

The pinned JavaScript implementation reads bun:jsc.memoryUsage() for the whole heap, then attributes the same current/peak range to each tracked context. That matches the result shape but does not establish real per-context accounting. Summary/detailed totals, context attribution, collection timing and context lifetime must derive from actual engine accounting rather than duplicated heap estimates.

Acceptance:

  • Implement actual current-context and other-context memory ownership measurements.
  • Preserve options validation, experimental warning behavior, eager/default execution semantics and asynchronous results.
  • Verify unchanged memory-specific original tests alongside real multi-context allocation/collection behavior.
  • Do not fabricate context totals or treat result-shape checks as measurement correctness.

Parent tracker: #66. This remains separate from ordinary VM/module ownership verification and inspector lifecycle #428.

9 days ago

VM/inspector ownership checkpoint (2026-10-10): source 5808f4149 and verified evidence. Home owns 125 registry modules plus the internal test adapter, adding seven modules and six native VM/profiler sources. All 20 selected implementation/header files match the Bun pin exactly. 77 unchanged original files: 62 registered passes plus 70 process checks, zero failures, skips or TODOs cover ordinary context/module state, cached scripts, async graphs, source URLs, profiler structure, serialization and console behavior. Both host and final Home ABI gates pass 101 tests / 1,512 assertions; seven native groups and Debug 34/34 pass. Build peak 5,658 MB under 8,192 MB; Home ABI 412 MB, original runs 219/185 MB under 3,840 MB. 447 committed archives verify and 77 reports restore. Disk refusals receive no credit; obsolete cache cleanup retains newest objects and invalidates discarded-output manifests. Ownership does not complete pinned stubs: inspector lifecycle remains #428, real per-context VM memory accounting is #870, and tracing/REPL, remaining module/native ownership, complete suite and Release/ASAN/all-platform verification remain open.

9 days ago

Native VM memory attribution checkpoint (2026-10-12): memory source, native distinct-root repair and verified evidence. Native full-collection analysis records JSC marked graph nodes/references and realm reachability, with engine size estimates plus deduplicated backing allocations. Indexed array capacity corrects the first failed unequal-allocation control. Exclusive/shared/unattributed weights produce estimates and ownership ranges instead of duplicating the whole heap for every context.

Default requests wait for full collection; eager requests trigger it. Results deliver asynchronously on the owning context, overlapping requests retain independent promises, and native requests retire on process/worker/test-realm shutdown. Unequal-allocation, collection timing, overlap and shutdown controls pass. Mixed-root native dispatch now preserves caller context and distinct roots; the original regression arguments/assertions are restored and pass.

54/54 Debug steps; 27 native groups; 55 unchanged pinned originals: 179 passed cases plus thirty-eight process checks, zero failures/skips/filters and two TODOs. Fourteen ABI controls (452 assertions) and 32 report-validator controls pass. All 1676 artifacts and 251 replayed report outcomes validate, retaining 26 negatives. Ten source hashes match tested bytes; explicit Pickier scans retain one existing error and eleven warnings, zero new findings. Formatting/docs paths pass. Allocation, compile, disk, stale-cache and both mixed-root repair failures remain visible.

Separate lifetime failure remains: evaluated contexts stay reachable after references are dropped. An unused context collects; a control that never calls measureMemory and installed Bun both retain evaluated contexts. The graph keeps that live row and the unchanged retirement assertion remains failed under #885. #870 stays open. Copy-on-write/shared backing proof, every memory type, profiler interactions and platform/Release/ASAN/full-suite proof remain incomplete. General Promise/native producers/transfer #877, directory #878, corpus #876, context #871/#856, performance #869/#873/#874/#872 and HTTPS #875/#532 remain open. Two TODOs receive no pass credit; known crash controls were not rerun.

9 days ago

Ordinary lifetime investigation narrows #885 to the last evaluated context. In an array/fill/map/Math.max matrix, earlier contexts collect and the final Math.max context stays live. Reversing the order retires Math.max and retains the final array-only context. A host runInThisContext call does not release the last sandbox. The original payload retirement assertion remains failed; an additional ordinary timer control still retains it after 22 seconds/full collections.

Retained exported closures and compiled functions stay callable. A retained Script alone does not retain its prior sandbox in the comparison. These controls preserve valid lifetimes and receive no full-suite parity credit.

A temporary private native heap observer reports RootMarkReason::Output (ordinal 13) for the remaining NodeVMGlobalObject, and JITWorkList (ordinal 14) for code roots named call and an unnamed function. The pinned engine's ScriptExecutable traversal marks code-block globals. This is a concrete compiled-code/worklist lead; it does not yet prove which plan or cache owns the realm. The earlier fill-specific hypothesis is superseded by the ordering controls.

Production sources are restored. No context filtering, cache flushing, worklist cancellation, assertion weakening or production diagnostic change is introduced. Diagnostic compilation errors and the unavailable snapshot-metadata attempt remain in the investigation evidence. #66, #870 and #885 stay open.

Evidence: 58 verified artifacts on main.

9 days ago

Refined native-root probe on main confirms the observed JITWorkList code roots named call and the unnamed function have host globals (VM_REALM=0), rather than NodeVMGlobalObject realms. Those worklist roots therefore do not establish the retained sandbox's owner. Direct Script.runInContext ordering controls also retain only the last evaluated sandbox, matching the convenience wrapper.

The actual owner remains unproved and the original dead-context assertion remains failed. Temporary private probes are restored. Probe and controls retain this correction alongside the completed CLI profiling batch; profiling passes receive no VM lifetime credit.

9 days ago

Native VM sandbox reuse is fixed on main in 41b8a2400, with 347 verified evidence artifacts.

Before the change, Script.runInNewContext created a fresh native realm for every call and did not contextify fresh sandboxes. The ordinary control failed isContext on Home while Node passed. Fresh sandboxes now register in the native map; contextified sandboxes reuse their realm. Script/module calls preserve lexical declarations, built-in identity and afterEvaluate microtask checkpoints. EventLoopHandle JS entry/exit forwards to the actual nesting-aware loop, and the platform loop alias is restored.

54/54 Debug steps; twenty-eight native groups; seven byte-identical original VM process checks; fourteen ABI controls with 452 assertions; 32 report-validator controls. No original failures/skips/filters/retries. Pickier, formatting and documentation paths pass. Three source hashes match tested bytes, and all 51 report outcomes replay, retaining eleven deliberate negatives.

The distinct last-context retirement assertion remains failed under #885. No live-context filtering, cache flushing or weakened lifetime assertion is used. All EventLoopHandle callers/Windows paths, Release/ASAN/all-platform and full Bun-suite proof remain incomplete.

9 days ago

Native VM context option semantics are fixed on main in 6bca26e9c, with verified evidence.

The baseline read ignored creation-option getters three times on Home while Node read zero. Existing contexts now return before reading creation options. Module new-context calls map contextName/contextOrigin/contextCodeGeneration into creation options; native Script uses the matching context option names, and native createContext validates import callbacks while preserving the default-loader constant. Script execution options keep the Script-owned callback rather than reading/installing another context callback.

54/54 Debug steps; twenty-eight native groups; twelve byte-identical original VM process checks; fourteen ABI controls with 452 assertions; 32 report-validator controls. Original outcomes have no failures/skips/TODOs/filters/retries. All 380 artifacts, three source hashes and 56 report outcomes replay, including eleven deliberate negatives. Explicit Pickier scans retain zero errors and one existing quotes warning, with zero new findings; docs paths pass.

Last-context retirement #885, inspector context name/origin metadata, every import lifetime, all-platform/Release/ASAN and full-suite proof remain incomplete. No live-context filtering, cache flushing or weakened lifetime assertion is introduced.

9 days ago

Native VM compiled-function scope and realm ownership are fixed on main in 8169912f2, with 412 verified evidence artifacts.

The baseline exposed a function-specific context extension to later realm evaluation. Extensions now remain in the function scope chain, and the generated executable carries the with-scope taint. Retained functions still see live extension changes while realm scripts and plain compiled functions cannot see those bindings. Function construction uses the parsing realm, preserving Function.prototype identity. Special parsing contexts use the shared native lookup; options/import callback validation and extension index labels are corrected.

54/54 final Debug steps; twenty-eight native groups; sixteen byte-identical original VM process checks; fourteen ABI controls with 452 assertions; 32 report-validator controls. Original outcomes have no failures/skips/TODOs/filters/retries. All 412 artifacts, two source hashes and sixty report outcomes replay, including eleven deliberate negatives. Explicit Pickier scans report zero errors/warnings; documentation paths pass.

The initial Node v26.10.0 reference aborted in native ContextifyFunction::CompileFunction on a nullable-extension case. Its failure logs remain preserved; that fixture was not rerun or published as executable source and receives no pass credit. The first compiled variant also failed prototype identity; constructing in the parsing realm corrects it without weakening the assertion. Nullable/proxy extension completeness, callback ownership cycles, last-context retirement #885, every memory type, all-platform/Release/ASAN and full Bun-suite proof remain incomplete.

9 days ago

Native VM cache requests and measurement delivery are fixed on main in 5867cb50f, with 804 verified evidence artifacts.

Cache consumption and production are independent requests. Supplied cache input no longer suppresses requested production of actual engine cache bytes. Serialization uses the executable's external with-scope flag; source strict directives remain source-derived. Complete own-runtime function/module caches roundtrip, and consuming function code with another extension object binds that fresh environment without leaking bindings into realm evaluation.

The first broader native run aborted during worker measurement delivery. The diagnostic and disassembly identify array construction returning null on an exception path before indexed access at offset 4. Range/result construction now checks fallible allocations/writes before use. Terminated owners retire native requests without clearing engine termination. The unchanged shutdown regression passes the focused check and guarded broader batch; the original abort remains failed in its own summary/logs.

54/54 final Debug steps; twenty-eight native groups; seventeen byte-identical original VM process checks; fourteen ABI controls with 452 assertions; 32 report-validator controls. Original outcomes have no failures/skips/TODOs/filters/retries. All 804 artifacts, three source hashes and 122 report outcomes replay, retaining 22 deliberate negatives. Explicit Pickier scans report zero errors/warnings; docs paths pass. Compile failures, the missed first include edit, the worker abort and a disk stop are retained.

Every measurement/termination schedule, invalid/cross-version cache input, callback ownership cycles, last-context retirement #885, every memory type, all-platform/Release/ASAN and full Bun-suite proof remain incomplete.

Sign in to comment on this issue.