ReviewOS

home-lang/home

VM parity: retire unreachable evaluated contexts

#885
Open chris opened this 8 days ago ·
8 days ago

Additional inspector-held lifetime acceptance is repaired on main: canonical console buffers retain the real timer object and realm before attachment, replay it through actual remote handles, then allow collection after console entries and their group are cleared. Dormant injected-script roots retire only after checking actual native bound handles, pending calls and debugger state. A retained unrelated handle and pending async call survive clearing. Node and Home lifecycle assertions pass unchanged after the earlier Node oracle mistakes were corrected from observed reference behavior; the original failures remain archived. Verified merged evidence de6aea239. Complete module/export/callback and all owner/test-realm shutdown acceptance remain open. Evaluated inspector realms outside explicit console clearing remain #888/#428.

The native cached-Structure owner is identified and repaired in source 7cfff1c01, with verified merged-main evidence 4852a2330. JSC sets the supplied Structure's realm during global initialization; reusing the parent-cached Structure retained the last VM global. Each instance now receives its own Structure. The pinned old executable retains the last of two contexts; Node and repaired Home collect both. The unchanged original lifetime assertion passes, and the real measured graph has zero leftover contexts. Five unchanged earlier controls collect all array/fill/map/Math contexts in either ordering, preserve callable closures/compiled functions, and preserve reusable Scripts. Announced unused realms emit owner-loop destruction. Normal and merged builds, 33 native groups and required ABI/report gates pass.

This issue remains open for complete retained module/export/callback, native owner-thread shutdown and test-realm lifetime acceptance. Inspector-held remote objects/controller roots remain #888. Earlier failed assertions and root investigations remain immutable below.

Earlier investigation

Latest investigation on main: ordering controls retain the last evaluated context, while earlier contexts collect. Reversing an array/fill/map/Math.max sequence changes which context remains live. Temporary native probes report Output realm roots. Refined probes show the observed JIT-worklist code roots belong to host globals, so they do not establish the retained sandbox owner. Direct Script.runInContext has the same last-context ordering behavior; the actual retaining owner is still unproved. The earlier blanket evaluated-context and fill-specific interpretations are superseded. Original failed lifetime assertions remain unchanged; production sources are restored.

Investigation and verification, 58 immutable artifacts.

Original reproduction:

Evaluated VM contexts remain reachable after their application references are dropped. In an ordinary control, an unused context's WeakRef clears while a context that evaluated globalThis.payload = new Array(8000).fill(9); undefined remains live across four event-loop turns/full collections. The same comparison runs without calling measureMemory, so measurement result construction is not needed to reproduce the retention.

The installed Bun 1.4.2 reference also retains the evaluated context in the control that never measures memory.

The new native memory graph correctly reports the remaining live context; suppressing that row would hide the lifetime gap. Node's equivalent memory/lifetime control retires the context. Retain the failing lifetime assertion separately from passing allocation/shape tests.

Acceptance:

  • Identify and retire the actual strong owner of an evaluated context when no reachable user object requires it.
  • Preserve retained Script, compiled function, module, callback and exported-value lifetimes.
  • Verify ordinary dropped-context collection and memory graph membership without filtering live contexts or weakening assertions.
  • Cover native owner-thread shutdown and test-realm retirement alongside ordinary VM conformance.

Parents: #66 and #870. Related inspector lifecycle: #428. This remains incomplete VM lifetime work, distinct from native memory attribution.

8 days ago

The independent control now confirms retention in both Home and installed Bun 1.4.2 without invoking measureMemory: an unused context clears, while an evaluated context remains reachable across four event-loop turns/full collections. The Node memory/lifetime reference retires its dropped context.

Native graph attribution must continue reporting the marked evaluated context. The lifetime assertion remains failed and is preserved alongside passing unequal-allocation, collection timing, overlapping-request and shutdown controls. #870 remains open for complete memory/lifetime proof.

8 days ago

Native VM memory attribution checkpoint (2026-10-12): memory source, native distinct-root repair and verified evidence. Native full-collection analysis records JSC marked graph nodes/references and realm reachability, with engine size estimates plus deduplicated backing allocations. Indexed array capacity corrects the first failed unequal-allocation control. Exclusive/shared/unattributed weights produce estimates and ownership ranges instead of duplicating the whole heap for every context.

Default requests wait for full collection; eager requests trigger it. Results deliver asynchronously on the owning context, overlapping requests retain independent promises, and native requests retire on process/worker/test-realm shutdown. Unequal-allocation, collection timing, overlap and shutdown controls pass. Mixed-root native dispatch now preserves caller context and distinct roots; the original regression arguments/assertions are restored and pass.

54/54 Debug steps; 27 native groups; 55 unchanged pinned originals: 179 passed cases plus thirty-eight process checks, zero failures/skips/filters and two TODOs. Fourteen ABI controls (452 assertions) and 32 report-validator controls pass. All 1676 artifacts and 251 replayed report outcomes validate, retaining 26 negatives. Ten source hashes match tested bytes; explicit Pickier scans retain one existing error and eleven warnings, zero new findings. Formatting/docs paths pass. Allocation, compile, disk, stale-cache and both mixed-root repair failures remain visible.

Separate lifetime failure remains: evaluated contexts stay reachable after references are dropped. An unused context collects; a control that never calls measureMemory and installed Bun both retain evaluated contexts. The graph keeps that live row and the unchanged retirement assertion remains failed under #885. #870 stays open. Copy-on-write/shared backing proof, every memory type, profiler interactions and platform/Release/ASAN/full-suite proof remain incomplete. General Promise/native producers/transfer #877, directory #878, corpus #876, context #871/#856, performance #869/#873/#874/#872 and HTTPS #875/#532 remain open. Two TODOs receive no pass credit; known crash controls were not rerun.

8 days ago

Ordinary lifetime investigation narrows #885 to the last evaluated context. In an array/fill/map/Math.max matrix, earlier contexts collect and the final Math.max context stays live. Reversing the order retires Math.max and retains the final array-only context. A host runInThisContext call does not release the last sandbox. The original payload retirement assertion remains failed; an additional ordinary timer control still retains it after 22 seconds/full collections.

Retained exported closures and compiled functions stay callable. A retained Script alone does not retain its prior sandbox in the comparison. These controls preserve valid lifetimes and receive no full-suite parity credit.

A temporary private native heap observer reports RootMarkReason::Output (ordinal 13) for the remaining NodeVMGlobalObject, and JITWorkList (ordinal 14) for code roots named call and an unnamed function. The pinned engine's ScriptExecutable traversal marks code-block globals. This is a concrete compiled-code/worklist lead; it does not yet prove which plan or cache owns the realm. The earlier fill-specific hypothesis is superseded by the ordering controls.

Production sources are restored. No context filtering, cache flushing, worklist cancellation, assertion weakening or production diagnostic change is introduced. Diagnostic compilation errors and the unavailable snapshot-metadata attempt remain in the investigation evidence. #66, #870 and #885 stay open.

Evidence: 58 verified artifacts on main.

8 days ago

Refined native-root probe on main confirms the observed JITWorkList code roots named call and the unnamed function have host globals (VM_REALM=0), rather than NodeVMGlobalObject realms. Those worklist roots therefore do not establish the retained sandbox's owner. Direct Script.runInContext ordering controls also retain only the last evaluated sandbox, matching the convenience wrapper.

The actual owner remains unproved and the original dead-context assertion remains failed. Temporary private probes are restored. Probe and controls retain this correction alongside the completed CLI profiling batch; profiling passes receive no VM lifetime credit.

8 days ago

Native VM sandbox reuse is fixed on main in 41b8a2400, with 347 verified evidence artifacts.

Before the change, Script.runInNewContext created a fresh native realm for every call and did not contextify fresh sandboxes. The ordinary control failed isContext on Home while Node passed. Fresh sandboxes now register in the native map; contextified sandboxes reuse their realm. Script/module calls preserve lexical declarations, built-in identity and afterEvaluate microtask checkpoints. EventLoopHandle JS entry/exit forwards to the actual nesting-aware loop, and the platform loop alias is restored.

54/54 Debug steps; twenty-eight native groups; seven byte-identical original VM process checks; fourteen ABI controls with 452 assertions; 32 report-validator controls. No original failures/skips/filters/retries. Pickier, formatting and documentation paths pass. Three source hashes match tested bytes, and all 51 report outcomes replay, retaining eleven deliberate negatives.

The distinct last-context retirement assertion remains failed under #885. No live-context filtering, cache flushing or weakened lifetime assertion is used. All EventLoopHandle callers/Windows paths, Release/ASAN/all-platform and full Bun-suite proof remain incomplete.

8 days ago

The collection investigation now narrows to native VM-global structure ownership. Ordinary WeakRef targets collect correctly in both Node and Home, while the dropped VM sandbox remains alive. The heap snapshot contains one main GlobalObject -> Structure -> NodeVMGlobalObject path.

Pinned JSC JSGlobalObject::finishCreation writes structure()->setRealm(vm, this) before initialization. Home passes its parent-cached NodeVMGlobalObjectStructure to every VM-global allocation. This is consistent with the cache retaining the latest realm. A multiple-context behavioral control is prepared to test that specific prediction before changing allocation. No lifetime fix or collection pass credit yet.

The console draft remains preserved in recovery 658f2cd; its fresh-cache normal build is actively compiling. Earlier missing-cache failures and disk admission exit 123 remain retained. Inspector parent #888/#428; full parity #66.

8 days ago

The first fresh-cache console build compiled but failed linking with duplicate InspectorConsoleAgent symbols from the SDK unity object. That failure is retained; no Home pass credit. Revised recovery b67038130 removes the SDK-agent replacement and uses a Home console client that forwards real arguments through canonical native wrapping while delegating SDK behavior. Public ABI headers remain unchanged.

The revision also gives each NodeVMGlobalObject its own Structure, preserving the template prototype. JSC mutates that structure's realm during global initialization; the prior cached parent structure has a heap path retaining the VM global. Node controls pass dropped-context collection, announced unused-realm destruction and live-context usability. Home validation remains pending. The old executable is separately pinned for the multiple-context baseline, so queued rebuilding cannot overwrite its evidence.

Normal revised build is queued behind a confirmed live shared-machine job. Full console methods/schema, remote-object/controller lifetime and complete inspector/all-platform/Bun-suite acceptance remain open (#885/#888/#428/#66).

8 days ago

Revised recovery b670381 built successfully: normal Debug 56/56 steps, peak guarded tree 5653 MB. The SDK symbol duplication is resolved through a Home console client that delegates existing SDK behavior; public ABI headers remain unchanged.

The separately pinned old Home executable confirms the cached-structure prediction: with two dropped contexts, Node reports [false,false] for WeakRef liveness while old Home reports [false,true], retaining only the last realm. This matches the main-global -> Structure -> VM-global heap path and JSC's per-global setRealm initialization. The fresh per-global structure implementation is built, but repaired Home collection/console acceptance is still queued behind a confirmed live machine validation job. No new Home pass credit yet.

Earlier missing-cache failures, disk admission refusal, copied-baseline permission failure and SDK-link duplication remain retained. Full remote-object/controller lifetime, console methods/schema, teardown and full inspector/all-platform/Bun-suite acceptance remain open (#885/#888/#428/#66).

8 days ago

Investigation evidence 8544a0330 is published on main, including the completed 56/56 revised build, source/executable identity, pinned last-context retention baseline, native heap path, Node controls, and all infrastructure/link failures.

Implementation remains preserved in recovery b670381 pending Home acceptance and broader gates. The exact built-source/executable snapshot is verified; its acceptance job is queued behind the confirmed live shared-machine validation job. No repaired Home collection/console pass credit or full parity claim yet.

8 days ago

Repaired Home source b670381 passes dropped-context collection and owner-loop destruction of an announced unused realm. The separately pinned earlier executable retains only the last dropped context; the repair reports both collected. Five byte-identical earlier lifetime controls now pass: all array/fill/map/Math contexts collect in both orderings, payload measurement reports zero leftover contexts, and retained closures/compiled functions remain callable; Script reuse still works.

The console control passes owner/realm events and native object properties but fails synchronous delivery to the second connected session. Recovery a1f71644d drains each enabled local frontend during native console delivery. Its normal rebuild is active. The unchanged original lifetime assertion was recovered for the next batch; broader gates and full lifetime/console acceptance remain pending.

No console pass credit for the failed multi-session control. Full Bun/all-platform parity remains open; source is preserved remotely while verification continues.

8 days ago

Native console delivery and VM structure lifetime (2026-10-13) landed in source 7cfff1c01, integrated with current main in e37bbf2, with verified evidence 4852a2330. Each VM global has its own Structure; the parent-cached structure retained the last context because JSC writes its realm during initialization. Old Home retains [false,true]; Node and repaired Home collect both. The unchanged original lifetime assertion passes, including zero leftover measured contexts; five earlier controls preserve callable closures/compiled functions and Script reuse. Announced unused realms retire on the owner loop. Native console events preserve actual arguments/object handles, timestamps, frames and owning context IDs; Runtime.disable and synchronous delivery to independent local sessions pass. Callback GC/reconnection passes. Normal and merged builds: 56/56 each. The changed merged executable received a fresh validation batch: eleven focused commands, 33 native groups, seven Node/seven Home realm controls, 14 ABI tests/453 assertions and 32 validator tests pass; lint/format/docs paths pass. Twenty unchanged originals retain 53 passed cases, one failed expectation and 17 process checks, with zero skips/TODOs/filters/retries. All 64 report outcomes match, including twelve negatives; no #886 credit. Earlier failures remain archived. #885 retains full module/export/callback/shutdown lifetime acceptance; #888/#428 retain complete console/backend schema, remote-object/controller roots and inspector retirement/teardown. Full Bun suite and Release/ASAN/Linux/Windows remain incomplete.

Source and 771 raw artifacts are published on origin/main. The goal remains incomplete; full lifetime, inspector and all-platform/full-suite acceptance retain their open criteria.

8 days ago

Native console methods, buffered replay and cleared-realm lifetime (2026-10-13) landed in source 56df99d7f, merged integration be4364972, and verified evidence de6aea239. Canonical SDK records provide count/reset, typed timeLog/timeEnd and per-session buffered replay with actual stacks/timestamps. Owner attachment differs from VM console state; Runtime.disable suppresses notifications while preserving connected state. groupEnd/clear arguments match the native contract. Clearing the buffer and remote group now retires idle realm injected-script roots; actual still-bound handles, pending calls and paused debugger state prevent retirement. Unrelated object handles and an in-flight async call remain usable. Last frontend detach retires injected-script state, and the non-owning console registry retires across finalization threads. Both source and merged builds pass 56/56 steps; fresh merged validation passes all nine focused commands, six broad drivers, 33 native groups, seven Node/seven Home realm controls, 14 ABI tests/453 assertions and 32 report-validator tests. All 64 report outcomes match expected counts/errors/exits, including twelve negatives. Twenty unchanged originals retain 53 passed cases, one unchanged unsupported-method expectation failure and 17 passed process checks, with zero skips/TODO/filter/retries; #886 receives no pass credit. Failed Node oracle assumptions, the Home post-clear lifetime failure and resource stops remain immutable; temporary tracing is removed. Full schema, independent remote-object/session ownership, evaluated-realm retirement outside explicit console clearing, module/export/callback/owner-shutdown acceptance and full suite/platform validation remain open under #888/#428/#885/#66.

8 days ago

Additional native VM evidence is published in ec4c45382: the unchanged vm-script-fetcher-leak.test.ts passes all four registered cases, and the full selected 24-file module/options batch passes on merged source f08bd50d7. See #889 for the separately completed option-default/getter-exception fix.

The callback-capture diagnostic retained dropped scripts/modules in both Node v26.10.0 and Home, while escaped functions successfully invoked their dynamic-import callbacks in both runtimes. This comparison does not establish a collection defect or justify weakening callback ownership. The broader evaluated-context, module/export and callback lifetime acceptance remains open. Raw comparisons, original identities and native reports.

8 days ago

Additional lifetime evidence is published in 4041585dd: unchanged happy-dom VM reproduction, script-leak, SourceTextModule-leak and concurrent-GC module-linking files all pass on source 57f6260bf. The concurrent-GC batch uses the original 500-specifier linking fixture and unchanged runtime limits; no source filtering or iteration reduction was applied. All 104 discovered pinned VM files were executed. The main VM file retains 66 TODOs and two skips, and broader module/export/callback/inspector realm acceptance remains open. Raw reports and exact original identities.

Sign in to comment on this issue.