Observability: publish generated-code symbols and exact native stacks

Parent: #459 Depends on: #461, #483 Related: #466, #467, #487, #495

Goal

Make generated JavaScript and WebAssembly code fully visible to host profilers, crash reporters, debuggers, and the zig-js inspector with exact native↔source↔bytecode attribution.

Current boundary

The saved macOS baseline profile sees hot anonymous MAP_JIT addresses as <unknown binary> because generated mappings have no native symbol table. #461 requires symbolized attribution and #483 includes debug/unwind metadata, but neither owns end-to-end profiler registration, native stack reconstruction, or lifecycle validation.

Scope

• Define artifact-owned symbols for functions, tiers, OSR regions, runtime stubs, Wasm functions, side exits, and retired generations.
• Publish native PC ranges to macOS sampling/crash tooling and Linux perf/jitdump/perf-map interfaces through owned writers and explicit platform APIs.
• Emit/register exact unwind metadata for every native frame, call stub, deopt/side exit, trap, and host transition on each ABI.
• Map native PC → artifact generation → optimizer/baseline/Wasm origin → bytecode/Wasm offset → source URL/line/column, including inlined frames.
• Reconstruct logical stacks across JS↔host↔Wasm, async continuations, generators, exceptions, proper-tail calls, deopt, and optimized-away frames where required.
• Deregister/retire symbols and unwind records only after the final executing reader leaves the artifact epoch.
• Measure registration, sampling, symbolization, memory, and disabled-path overhead.

No-workaround rules

• No permanent executable mapping retention for symbolization, stale/reused PC attribution, guessed stack scanning, profiler-only different codegen, unsafe unwind through Zig frames, or dependency on a third-party profiler library.

Acceptance

• Owned fixtures validate PC ranges, names, source offsets, inline/deopt frames, unwind, crash traces, and async/host/Wasm boundaries on each backend.
• System profiler samples attribute generated leaves to stable symbols instead of anonymous mappings.
• Inspector CPU profiles and host-native samples agree on artifact/source identity for the same execution.
• Invalidation/retirement/teardown races never expose stale metadata and pass TSan/artifact lifetime tests.
• Disabled observability has measured zero or bounded negligible overhead; enabled overhead and memory are published.

Foundation

Baseline dispatch profile; inspector/CPU-profile foundations under #139/#404.

Landed the native-PC ownership foundation in 1b14ab9 (feat(observability): own native PC metadata).

This adds opt-in, context-owned baseline/optimizer PC ranges, stable symbol/function/source identity, exact hoisted-function definition coordinates, owned lookup snapshots, and execution-epoch-aware retirement. The row is removed before executable unmapping; focused TSan covers lookup racing reclamation, and disabled contexts allocate/retain no native metadata.

Validation: 1,511 unit tests passed with 0 skipped / 0 failed / 0 leaked across 10 shards; focused native-PC TSan passed; docs link check covered 100 files; the 49-page docs manifest/build passed.

This does not claim external profiler visibility yet. Next is the Darwin system-profiler publication/unwind layer, followed by native-PC↔bytecode/inline/deopt maps and inspector/host-stack agreement.

Pushed 93cd3ade (feat(observability): publish native symbols to LLDB).

This lands the debugger-symbol publication slice on top of the owned native-PC registry:

• adds an embedder-owned publication interface; supplying it implies native observability and propagates through shared realms;
• adds an opt-in standard GDB JIT protocol adapter for macOS LLDB;
• emits an in-memory Mach-O object whose one __text section names the exact existing executable PC range—no copied, linked, or remapped executable image;
• owns registration with the artifact and unregisters before metadata destruction and executable unmapping;
• keeps __jit_debug_descriptor and __jit_debug_register_code out of the default static archive unless the embedder explicitly selects this process-global adapter;
• serializes concurrent register/unregister list mutation.

Owned evidence:

• zig build native-observability-lldb-test: real LLDB resolves a pending zig_js_baseline_1_observedNative breakpoint at __text + 0, then reports zero matching functions after Owner teardown;
• focused publication lifecycle under TSan: 3 passed, 0 failed/leaked;
• full zig build test-parallel: 1,515 passed, 0 skipped/failed/leaked across 10 shards;
• docs build/manifest/link gates pass;
• C API audit 117/117 and Home public ABI audit 50/50 pass;
• the rebuilt default libzig-js.a contains no __jit_debug globals.

The issue remains open. This does not yet claim Instruments/sample symbol visibility, unwind publication, external source/inline maps, deopt/crash/async/host/Wasm coverage, inspector agreement, or measured disabled/enabled overhead. Those remain the next acceptance slices; the next investigation stays on public Darwin system-profiler and unwind mechanisms rather than treating LLDB registration as profiler evidence.

Progress landed in 7e232c21:

• baseline numeric and optimizer artifacts now establish canonical Darwin AArch64 x29/x30 frame records; every optimizer exit restores the record
• the LLDB gate now uses a production Context-compiled function, resolves the exact generated symbol at __text + 0, stops after the two-instruction prologue, and walks to a valid host frame
• the same gate still proves symbol removal after Context teardown
• constant-return leaf stubs intentionally remain frameless because they neither change SP nor call another function

Validation: focused JIT 192/192; sharded unit suite 1516/1516, 0 skipped/failed/leaked; real LLDB gate; docs manifest/build; offline link check 100/100. This changes no test262 behavior (0 flips).

This completes the canonical frame-chain prerequisite, not the whole issue. #501 stays open: Instruments/sample visibility still needs system-profiler image publication, while exact unwinding in the prologue and crash-path symbolication still need per-artifact CFI/compact-unwind and source mapping.

Dynamic unwind publication landed in 19d3439a:

• AArch64 codegen now publishes an explicit leaf vs frame-pointer unwind plan; publishers do not recognize or guess instruction bytes
• baseline and optimizer exits converge on one final epilogue, giving each artifact one exact terminal CFI row while reducing duplicated restore sequences
• the baseline plan describes every saved x19–x28 and d8–d15 register, plus exact entry/prologue/body/final-restore state; optimizer and constant-leaf plans carry their narrower contracts
• the opt-in Darwin adapter emits a PC-relative DWARF CIE/FDE for the exact generated PC range, calls __register_frame before publication completes, and calls __deregister_frame before FDE storage or executable mappings are released
• the production Context fixture requires _Unwind_Find_FDE to resolve the live artifact to its exact function start and to stop resolving it after teardown, alongside the existing real-LLDB symbol and generated-to-host frame checks

Validation: focused JIT 193/193 normally and under TSan (including concurrent register/deregister churn); full sharded suite 1517/1517, 0 skipped/failed/leaked; real LLDB gate; docs manifest/build and offline links 100/100. The rebuilt default archive contains no __register_frame, __deregister_frame, or __jit_debug references, so the process-global adapter remains opt-in. This flips 0 test262 cases.

#501 remains open. Darwin system-profiler image publication is still required to replace anonymous MAP_JIT leaves in Instruments/sample; source/inline/deopt maps, inspector agreement, async/host/Wasm logical stacks, crash-path integration, other backends, and enabled/disabled overhead evidence also remain.

Baseline native-PC/source attribution landed in 5248dadf:

• observability-enabled constant and numeric baseline compilation now emits sorted, artifact-relative PC change rows with exact bytecode offsets; prologue/epilogue plumbing is explicitly unmapped
• fused comparison/branch code and slow checkpoint accounting carry distinct, exact bytecode identities
• adoption resolves source only for bytecodes with an explicit inspector statement node, requires the same script id/URL, and copies the inspector's one-based source position; no nearest-line inference is used
• the owned publisher artifact receives those rows, and native-PC lookup returns the active bytecode/source identity while preserving epoch-retirement lifetime guarantees
• disabled contexts allocate and retain no native PC map

Validation: full sharded suite 1,519/1,519, 0 skipped/failed/leaked; focused publisher, compiler, inspector-resolution, and epoch-race tests pass normally and under TSan; the real LLDB symbol/frame/unregister gate passes; docs build 49 pages and offline links cover 100 files. Test262 was not run because this slice changes observability metadata only, not interpreter/VM semantics.

I also verified the platform boundary before this slice: WebKit treats useGdbJITInfo and useJITDump as separate outputs, and its jitdump writer only creates the executable detection mapping under Linux. That is not evidence of Instruments/sample registration, so no Darwin system-profiler claim is made here.

#501 remains open. Optimizer/inline/deopt maps, external-object line records, Darwin system-profiler publication, crash and async/host/Wasm stacks, other backends, and overhead evidence remain.

Optimizer native PC/source maps landed on main in 25c3263e.

Implemented in this slice:

• carries exact bytecode origins from optimizer graph nodes through real SSA operations and hoisted/runtime staging operations;
• attributes branch tests, return arms, side exits, deopt/safepoint polls, loop guards, latches, and region controls from their explicit graph or recovery records;
• deliberately resets synthetic parallel edge copies, path-dependent joins, and the shared epilogue to unmapped instead of guessing a source line;
• allocates and retains optimizer PC rows only when the owning Context enabled native observability;
• resolves optimizer rows through the same Context-owned inspector statement registry and exact script identity boundary as baseline code.

Regression evidence:

• zig build test-parallel: 1,520 passed, 0 skipped, 0 failed, 0 leaked across 10 shards;
• zig build test -Dtsan=true -Dtest-filter=jit.optimizer_compiler: 49/49 passed, including observed ordinary branches/returns, asymmetric side exits, and multi-latch loop deopt mapping;
• zig build native-observability-lldb-test: passed real LLDB symbol discovery, exact __text address, caller unwind, and post-unregister disappearance;
• zig build docs-link-check: 100 files, all links resolve;
• bun run docs:manifest and bun run docs:build: 49 pages built and the checked manifest matched.

Test262 was not run because this changes native observability metadata, not JavaScript execution semantics.

Still open under #501: system-profiler image publication (including Instruments/sample), external debugger/profiler source rows and inline-frame maps, crash-path integration, logical async/deopt/Wasm stack reconstruction, other native backends, and measured disabled/enabled overhead evidence.

Exact external debugger source rows landed on main in 416252ea.

Implemented in this slice:

• GDB JIT Mach-O objects now own minimal DWARF v4 debug-abbrev, debug-info, and debug-line sections when an artifact has exact source rows;
• baseline and optimizer PCs publish their exact JavaScript file, one-based line, and column;
• explicitly unmapped rows terminate the active DWARF line sequence, so prologues, synthetic moves/joins, and epilogues cannot inherit a nearby source line;
• debug sections use a separate relative-address DWARF segment, matching LLDB JIT-loader remapping while the text section continues to name the existing executable mapping exactly;
• the production fixture drives the tiers allowed by production policy: one managed-baseline function and a separate property-access function compiled by the optimizer;
• teardown now proves both symbols disappear, both dynamic FDEs are gone, and every publication receives its unregister callback.

Regression evidence:

• full sharded suite: 1,521/1,521 passed, 0 skipped/failed/leaked across 10 shards;
• focused GDB JIT publication: 5/5 normally and 5/5 under TSan;
• real LLDB: pending baseline and optimizer symbols resolved at text offset zero; each module exposed one compilation unit; exact source breakpoints stopped at native-observability-fixture.js:23:3 and :27:3; baseline generated-to-host unwind passed; both modules retired at Context teardown;
• C API audit 117/117 and Home public ABI audit 50/50;
• docs build/manifest passed for 49 pages and offline links passed 100/100.

Test262 was not run because this is observability metadata and fixture work only; it flips 0 test262 cases.

#501 remains open. This completes the external debugger source-row slice, not Instruments/sample image publication, inline-frame maps, crash-path integration, logical async/deopt/Wasm stacks, other native backends, inspector/host-profile agreement, or overhead evidence.

Publisher-cost accounting and the exact-binary measurement boundary landed on main in a0d84a68.

This adds:

• lock-coherent process-wide counters for live GDB JIT registrations, owned Mach-O symbol/debug bytes, owned unwind bytes, and monotonic register/unregister totals;
• lifetime ordering that increments storage before the debugger registration callback and decrements it only after the unregister callback returns;
• concurrent normal/TSan tests proving live counts and bytes return exactly to their prior values and 256 concurrent publications produce 256 matching retirements;
• a same-binary ReleaseFast single_observed runner mode that registers real benchmark sources, selects the actual GDB JIT publisher, and reports generated-code state, publisher-owned storage, peak/current resident memory, and post-Context teardown state;
• a disabled single mode using that exact binary and telemetry boundary, so the forthcoming alternating collector measures the real runtime toggle instead of comparing different executables.

The real representative_json smoke preserved frozen checksum 5,864,992 in both states. Disabled reported zero external registrations and zero publisher bytes. Observed reported one live optimizer artifact, one debugger registration, 1,335 owned symbol/debug bytes, 72 owned unwind bytes, then zero live registrations/bytes and matching 1 registration / 1 unregistration after teardown. These single probes validate accounting only; they are not promoted as an overhead claim.

Validation: focused publication 5/5 normally and under TSan; real two-tier LLDB source/unwind/retirement; full benchmark harness family; default archive contains no GDB JIT or dynamic-frame symbols; docs manifest/build and 100/100 offline links. Test262 flips 0.

#501 remains open. The next measurement step is the balanced multi-pair collector and diagnostic artifact; a negligible-overhead claim still requires the existing quiet-reference, AC-power, counter-stability, and thermal gates.

Advanced by c13d7f6f (feat(profiling): measure debugger publication overhead).

This adds schema-v5 native_observability support to the balanced instrumentation-overhead collector. Disabled single and enabled single_observed samples use the exact same ReleaseFast binary; validation now fails on workload/checksum drift, tier/publication drift, generated-code drift, incoherent register/unregister totals, missing enabled symbol or unwind storage, nonzero disabled publisher state, or storage surviving Context teardown. It records invocation and process wall time, CPU, peak/current RSS, counters/thermal state, live code, symbol-object bytes, unwind bytes, and pre/post-teardown publisher totals.

A real two-pair diagnostic run kept the representative JSON checksum at 5,864,992 and the optimizer artifact at 16,384 live code bytes in both states. Disabled publisher state was zero. Enabled state had 1 live registration, 1,335 symbol-object bytes, and 72 unwind bytes; post-teardown live values were all zero with 1 registration / 1 unregistration. The run was quick diagnostic evidence and counter calibration reported noise, so it is deliberately not published as a negligible-overhead claim.

Validation: both real quick profiles, full zig build benchmark-comparison-test, instrumentation self-test failure fixtures, docs manifest/build, and docs-link-check across 100 files. Flips 0 test262 cases.

This advances the overhead acceptance item but does not close the issue; system-profiler attribution, stack reconstruction boundaries, and quiet-reference enabled/disabled publication evidence remain.

Crash-path native-PC resolution landed on main in 70190334 (feat(observability): resolve native PCs in crash paths).

Implemented in this slice:

• Context.lookupNativeCodeSignalSafe resolves an exact generated PC through its owning Context and returns artifact id, tier, live/retired state, native and bytecode offsets, function/script identity, and exact source coordinates;
• lookup uses only lock-free word atomics and caller-owned buffers—no allocation, locks, I/O, or borrowed result storage—and undersized buffers fail closed with NativeIdentityBufferTooSmall rather than truncating persisted crash evidence;
• the registry is Owner-local and lazily release-published/acquire-read, so disabled contexts allocate no registry or artifact metadata;
• retirement unlinks first, flips reader generations, waits only pre-existing readers, then permits metadata destruction and executable address reuse; new lookups cannot starve teardown;
• local visibility is established before an external debugger publisher is called and rolls back if external publication fails;
• focused tests cover exact source rows, explicit unmapped plumbing, live/retired/disappeared identity, buffer failure, first-registry publication, and continuous readers racing epoch reclamation.

Regression evidence:

• full sharded suite: 1,523/1,523 passed, 0 skipped/failed/leaked across 10 shards;
• first-publication and epoch-reclamation races passed under TSan;
• the real LLDB JIT-loader gate passed exact baseline/optimizer symbols, source rows, generated-to-host unwind, and post-unregister disappearance;
• C API audit/runtime hosts passed 117/117; pinned Home public ABI passed 50/50 with runtime smoke;
• docs built 49 pages and the offline link gate passed all 100 files.

Test262 was not run because this adds host observability infrastructure without changing JavaScript semantics.

#501 remains open. This completes the owned signal-safe native-PC lookup primitive, not an end-to-end crash-report fixture, Instruments/sample image publication, logical async/deopt/host/Wasm stack reconstruction, other native backends, or quiet-reference overhead publication.

An owned asynchronous signal fixture landed on main in f34032fc (test(observability): sample JIT PCs from signals).

The new native-observability-signal-test:

• installs a real macOS SA_SIGINFO handler and targets the JavaScript execution thread with SIGUSR1 from a helper thread;
• extracts the interrupted PC from the host ucontext and calls Context.lookupNativeCodeSignalSafe with fixed caller-owned buffers;
• accepts only an exact mapped source row from the real production-compiled signalTarget function, then verifies PC range/offset, artifact id, tier-specific symbol, bytecode offset, source byte/line/column, function name, and source URL outside the handler;
• performs no allocation, locking, or I/O in the handler;
• serializes delivery so every successful pthread_kill waits until its handler has returned, including fixture failure paths, preventing a pending signal from outliving the Context;
• verifies every publisher registration receives teardown before exit.

Evidence: 100 fresh Debug processes passed, 10 TSan processes passed, ReleaseSafe passed, the existing real LLDB symbol/source/unwind/unregister gate still passed, and the 49-page docs manifest/build plus 100-file link gate passed. Test262 was not run because this is fixture/documentation work only.

This validates a real asynchronous native-PC capture and exact signal-handler lookup. #501 remains open for fatal crash-report transport/child-process traces, full native stack reconstruction across host/async/deopt/Wasm boundaries, Instruments/sample image publication, other native backends, and quiet-reference overhead publication.

Implemented the fatal crash-report transport slice in 4d014a9d.

• Added zig build native-observability-crash-test. A child runs the production JIT, receives real SIGABRT deliveries on its JavaScript thread, and only commits a record after resolving an exact signalTarget generated PC.
• The handler copies the dynamic PC range, native offset, artifact/tier, symbol/function, script, bytecode, and exact source identity into a versioned 504-byte record, performs one async-signal-safe write to a pipe opened before exec, and calls _exit. A compile-time assertion keeps the record within the POSIX minimum 512-byte PIPE_BUF; short identity buffers fail closed.
• The parent requires the distinct handler exit code, exactly one complete record, no stderr, and validates every range and identity relationship from the interrupted PC.
• Validation: 100 fresh Debug parent/child processes; Debug and ReleaseSafe build steps; the existing asynchronous signal fixture; docs manifest; 100-file offline link/sidebar audit; checked 49-page docs build.

No sanitizer suppression or handler-specific instrumentation escape was added. A fatal-fixture TSan probe showed the sanitizer runtime can deadlock when the signal interrupts inside its own __tsan_atomic64_* interceptor and the handler then performs its first atomic load; the sampled stack demonstrated re-entry into TSan slot locking. Existing direct retirement-race and nonfatal handoff TSan coverage remains the race gate.

This does not close #501. External system-profiler image publication, inline-frame maps, and logical async/deopt/Wasm stack reconstruction remain open.

Linux perf jitdump publication landed on main in fec74a36 (feat(observability): publish Linux perf jitdump records).

This slice adds:

• an opt-in, embedder-owned js.jit.PerfJitDumpWriter; the default static library still claims no profiler-global ABI;
• secure exclusive jit-<pid>.dump creation, the perf executable discovery mapping, monotonic timestamps, Linux pid/tid identity, x86-64/AArch64 ELF machine IDs, serialized publication, a close record, flush/descriptor lifetime, and exact byte/record accounting;
• one JIT_CODE_LOAD per exact PC-map span. Source-bearing spans emit JIT_CODE_DEBUG_INFO first, as required by perf; prologue, epilogue, and synthetic spans remain named but deliberately source-unmapped instead of inheriting a nearby JavaScript line;
• fail-closed validation for malformed/overlapping maps, address/size overflow, code-index exhaustion, embedded NULs, invalid source lines, write failure, unsupported targets, and noexec discovery mappings;
• no retained artifact token storage. jitdump has no unload record; later address reuse receives a newer timestamped load and unique code index.

Validation:

• full sharded unit suite: 1,527 passed, 1 macOS platform skip, 0 failed, 0 leaked across 10 shards;
• focused JIT: 203 passed, 1 macOS platform skip;
• focused TSan: 4 passed, 1 macOS platform skip;
• focused VM/concurrency and GC relocation inventory passed;
• the complete Linux lifecycle cross-compiled for x86-64 and AArch64;
• docs manifest/build passed for 49 pages; offline link audit passed 100/100;
• the default libzig-js.a contains neither GDB JIT nor perf-jitdump publisher symbols.

This advances the Linux profiler-writer part of #501 but does not claim Linux native-tier execution before #485, real perf record attribution on a supported Linux backend, Linux unwind records, Darwin Instruments/sample publication, inline/logical async/deopt/Wasm stacks, other backends, or quiet-reference overhead evidence. The issue remains open. Test262 flips 0.

Follow-up from the native-memory validation: opened #508 for the reproducible macOS LLDB GDB-JIT descriptor ambiguity.

The same fec74a36 baseline binary can pass and fail across consecutive launches. LLDB sometimes selects a zero-address undefined debug-map __jit_debug_descriptor row instead of the valid defined external row, then logs failed to find JIT descriptor address. Stopping explicitly in the production registration callback shows a valid live register action and allows the real loader to consume the generated Mach-O, so symbol/source publication itself remains correct.

#508 owns a deterministic linker/symbol-boundary fix and repeated-launch gate. It explicitly forbids sleeps, blind retries, or replacing the real LLDB loader with a test-side parser.

#508 is fixed on main by aef1cf88. The opt-in LLDB fixture now exposes one unambiguous defined GDB-JIT descriptor, and 20 consecutive real-loader launches passed exact baseline/optimizer symbol, source, unwind, and deregistration assertions. The default archive remains free of the opt-in globals.

Opened related child #607 for debugger-location lookup overhead in ordinary tree-walker execution. It is limited to skipping AST tags that cannot exist in the parser-published statement registry; actual statements, late attachment, stepping, stack locations, host hooks, and VM/native checkpoint semantics remain in scope for exact validation.

Related observability child #607 is complete in 506d8950. The change preserves late debugger attachment, stepping, module graphs, exception/stack locations, host hooks, and VM checkpoints while removing only AST tags impossible in the statement registry. All 13 debugger gates and the focused TSan witness pass.

Debugger/JIT source observability now routes through the keyed exact registry from #860: fix(debugger): seed statement location registry (#860). Script ids, source coordinates, late attach/history, warmed-code fallback, baseline/optimizer PC mapping, and signal-safe published snapshots remain exact across the debugger matrix.

Completed native CPU-profile tier preservation in #1043 via f5c638e1. The observation-only profiler no longer disables baseline/optimizer execution; managed baseline samples resolve the exact checkpoint bytecode row, optimizer polls resume through the ordinary exact VM boundary, and debugger/host hooks retain their bytecode requirement. Focused normal/TSan witnesses, all three private fixture modes, full units, ABI audits, and docs gates pass.