ReviewOS

zig-utils/zig-js

API: plumb source locations for evaluation diagnostics

#23
Closed chris opened this 10 days ago ·
10 days ago

Parent: #22

Goal

Implement real source-location plumbing for evaluated scripts/modules before treating JavaScriptCore-shaped source metadata as stable API surface.

Background

JSEvaluateScript still accepts sourceURL and startingLineNumber, but the parser/runtime diagnostic path currently reports coarse error names rather than source-aware locations. Earlier #22 cleanup intentionally avoided a decorative implementation because attaching the arguments only at the C boundary would not improve parse/runtime diagnostics.

Scope

  • Add a source-name / starting-line representation to the parser/evaluator pipeline.
  • Preserve enough token/node or diagnostic span information to report meaningful parse errors.
  • Thread source names through runtime stack/error reporting where applicable.
  • Make JSEvaluateScript(..., sourceURL, startingLineNumber, exception) either honor those values with focused tests or replace/remove the inert arguments before stabilization.
  • Update docs so embedders know exactly what source metadata is supported.

Acceptance criteria

  • A focused C-API test proves sourceURL and startingLineNumber affect observable diagnostics/stack/error metadata.
  • Parser/runtime tests cover at least one parse error and one thrown runtime error with source metadata.
  • docs/api.md no longer lists sourceURL / startingLineNumber as inert compatibility arguments.
10 days ago

Implemented the first source-diagnostic slice in 784fb60.

What changed:

  • Added parser source-location infrastructure: SourceLocation, sourceLocationAt, and Parser.errorLocation, using lexer token byte offsets.
  • Context.evaluateWithThis now records the location of evaluation-time parser failures without widening the public RunError set.
  • JSEvaluateScript now uses sourceURL and startingLineNumber when formatting syntax/parse exception text, for example: UnexpectedToken: app.js:41:12.
  • Updated C API docs so sourceURL and startingLineNumber are no longer documented as inert.

Validation:

  • zig build test -Dtest-filter=parser-source-locations-equivalent --summary none via the parser source locations filter
  • zig build test -Dtest-filter=parser-records-current-token-equivalent --summary none via the parser records current token filter
  • zig build test -Dtest-filter=C-API: --summary none
  • bun run docs:build

Remaining issue #23 work: propagate richer source metadata into runtime stack/source notes and internal eval/dynamic-function parse sites, rather than only top-level C-API evaluation diagnostics.

10 days ago

Follow-up implemented in 0b05f2c.

What changed:

  • Added a reusable Interpreter parser-diagnostic helper that formats SyntaxError messages with operation context plus parser line/column.
  • Wired direct eval, $262.evalScript, ShadowRealm.prototype.evaluate, ordinary Function, and generator/async dynamic Function constructors away from generic parse-error text.
  • Runtime examples now report messages shaped like: eval: UnexpectedToken at 1:..., Function body: ... at ..., ShadowRealm.evaluate: ... at 2:...

Validation:

  • zig build test -Dtest-filter="eval: direct eval runs in the caller" --summary none
  • zig build test -Dtest-filter="Function constructor builds callable" --summary none
  • zig build test -Dtest-filter="ShadowRealm evaluate wraps" --summary none
  • zig build test -Dtest-filter="GeneratorFunction" --summary none

Remaining #23 work: lexer-originated errors still only get a start-of-source fallback because Lexer itself does not expose the failing byte offset yet; runtime stack/source-note metadata is also still separate work.

10 days ago

Follow-up implemented in 1313dd6.

What changed:

  • Lexer now records the byte offset where tokenization fails.
  • Parser exposes initWithDiagnostic so callers can get lexer-originated source locations from the same tokenization pass.
  • Top-level Context.evaluateWithThis, direct eval, $262.evalScript, ShadowRealm.evaluate, and Function/dynamic Function constructors now use those lexer diagnostics instead of falling back to source start.
  • C API diagnostics now report sourceURL/startingLineNumber for both parser failures and lexer failures such as unterminated strings.

Validation:

  • zig build test -Dtest-filter="parser init reports lexer failure source location" --summary none
  • zig build test -Dtest-filter="parser source locations" --summary none
  • zig build test -Dtest-filter="C-API: evaluation syntax exceptions include source URL and line" --summary none
  • zig build test -Dtest-filter="eval: direct eval runs in the caller" --summary none

Remaining #23 work: richer stack/source-note metadata is still separate, and diagnostics are still line/column text rather than structured JS Error fields.

10 days ago

Implemented another #23 source-diagnostic slice in d8bc664.

What changed:

  • JSEvaluateScript parse/lex failures now surface a real SyntaxError object instead of only a string exception.
  • The SyntaxError message still includes sourceURL and adjusted line/column, preserving the previous observable text.
  • Added structured, non-enumerable diagnostic fields: sourceURL, line, column, and byteOffset.
  • The C API copies sourceURL metadata into the context arena before attaching it, so releasing the JSStringRef cannot leave dangling diagnostic metadata.
  • docs/api.md now documents the structured diagnostic fields.
  • Extended focused C API coverage to read the structured fields through JSObjectGetProperty.

Validation:

  • zig build test -Dtest-filter='C-API: evaluation syntax exceptions include source URL and line' --summary none
  • bun run docs:build
  • git diff --check
10 days ago

Implemented the runtime source-metadata slice for #23 in 49b6f83.

What changed:

  • JSEvaluateScript now attaches sourceURL and startingLineNumber as non-enumerable fields to runtime-thrown Error objects when source metadata is provided.
  • Error.prototype.stack now includes the attached eval source frame when those fields are present, while preserving existing behavior for errors without source metadata.
  • The stack getter reads own fields directly rather than invoking getters/proxies, preserving the existing non-reentrant/security-conscious stack behavior.
  • Added focused C API coverage proving sourceURL and startingLineNumber are observable on runtime exceptions and through stack.
  • docs/api.md documents runtime Error metadata and stack behavior.

Validation:

  • zig build test -Dtest-filter='C-API: evaluation runtime errors include source metadata' --summary none
  • zig build test -Dtest-filter='Error.prototype.stack accessor' --summary none
  • bun run docs:build
  • git diff --check
10 days ago

Implemented another #23 diagnostic slice in 0ca7ac1.

What changed:

  • The shared interpreter parser-SyntaxError helper now attaches structured, non-enumerable line, column, and byteOffset fields to the SyntaxError object it throws.
  • This covers internal parser-error surfaces that already use that helper, including direct eval, $262.evalScript, ShadowRealm.prototype.evaluate, and dynamic Function/generator/async-function constructors.
  • The direct eval witness now verifies the fields exist and remain non-enumerable, so embedders/tests do not need to parse message text for source positions.
  • docs/api.md no longer says broader stack/source-note plumbing is still outstanding; it documents the current metadata behavior directly.

Validation:

  • zig fmt src/interpreter.zig src/context.zig
  • zig build test -Dtest-filter='eval: direct eval runs in the caller' --summary none
  • zig build test -Dtest-filter='Function constructor builds callable' --summary none
  • zig build test -Dtest-filter='ShadowRealm evaluate wraps' --summary none
  • zig build test -Dtest-filter='C-API: evaluation syntax exceptions include source URL and line' --summary none
  • bun run docs:build
  • git diff --check
10 days ago

Closing as complete against the current #23 acceptance criteria.\n\nCurrent evidence in-tree:\n- C-API syntax diagnostics: C-API: evaluation syntax exceptions include source URL and line covers sourceURL, adjusted line/column, and structured sourceURL / line / column / byteOffset fields.\n- C-API runtime throws: C-API: evaluation runtime errors include source metadata covers sourceURL / startingLineNumber on thrown Error objects and stack output.\n- Internal parser/runtime surfaces: direct eval, Function constructors, and ShadowRealm parser failures use the shared parser SyntaxError helper; direct eval now verifies non-enumerable structured line / column / byteOffset fields.\n- docs/api.md documents the current source metadata behavior and no longer describes sourceURL / startingLineNumber as inert compatibility arguments.\n\nFuture source-map, module-filename, or richer stack-frame work can get a new focused issue if/when we need it; the original JSEvaluateScript source metadata gap is implemented.

Sign in to comment on this issue.