ReviewOS

zig-utils/zig-regex

ECMAScript multiline ^ and $ ignore \r, U+2028 and U+2029

#28
Closed glennmichael123 opened this 9 days ago ·
9 days ago

Executed through zig-js (Debug ad1ca106, zig-regex 73cc22d) against node v24.4.1. JavaScriptCore agrees with node. Multiline ^/$ only recognise \n, but ECMA-262 22.2.2.4 (with LineTerminator, 12.3) uses LF, CR, U+2028 and U+2029:

programnodezig-js
/^b/m.exec("a\rb")["b"]null
/a$/m.exec("a\r\nb")["a"]null
/^b/m.exec("a
b"), /a$/m.exec("a
b")matchnull
"a\r\nb\rc
d".match(/^\w$/gm)["a","b","c","d"]null
"l1\r\nl2\r\nl3".replace(/^/gm, "> ")"> l1\r> \n> l2\r> \n> l3""> l1\r\n> l2\r\n> l3"
"x\r\ny".split(/$/m)["x","\r","\ny"]["x\r","\ny"]
[..."\r\n".matchAll(/^$/mg)].map(m => m.index)[0,1,2][2]
/(?=b)^b/m.exec("a\rb") (backtracker)["b"]null

Any /^…$/m over CRLF text silently misbehaves.

Cause: every engine hard-codes '\n' when it evaluates start_line/end_line:

  • backtrack.zig:1182-1189;
  • vm.zig:411-418;
  • the lazy DFA's FLAG_PREV_NL and emptyMask (dfa.zig:57, 201-202, 350, 376);
  • anchorHolds and nextLineStart (regex.zig:429-430, 640-646).

. already excludes all four terminators (compiler.zig:278-300). zig-regex sees WTF-8, so U+2028/U+2029 are the triples E2 80 A8/E2 80 A9, and the byte DFA needs to track them.

Fix: ecmascript-gated lineTerminatorBefore/lineTerminatorAt helpers in common.zig, used by the backtracker, VM and fast paths. The lazy DFA gets a virtual look-ahead column for the E2 that starts LS/PS, plus two transient look-behind bits. Non-ECMAScript behaviour is unchanged.

9 days ago

Taking this now, together with the other one filed alongside it; the fix is small and I'm gating it immediately.

9 days ago

Fixed in 6e805ad; pinned through zig-js in zig-utils/zig-js@15f93fe.

Shared, ecmascript-gated lineTerminatorBefore/lineTerminatorAt helpers now drive the backtracker, the VM, anchorHolds and nextLineStart. The lazy DFA gets a virtual LT_LEAD column for the E2 that starts U+2028/U+2029, two transient look-behind bits that carry the triple through to ^, and matching start flags. Only DFAs with ECMAScript multiline line anchors use any of this.

Evidence:

  • 23 node-verified rows run through a reused Matcher. They include the DFA consuming each terminator mid-scan, plus the é, ¨ and € look-alikes as negatives. The table fails on its first row with the old dfa.zig.
  • LF-only controls confirm non-ECMAScript behaviour is unchanged.
  • Through zig-js, 17 of 17 probes match node (2 before).
  • test-core: 365/365.
  • test262 before/after is identical, with 0 flips.

Sign in to comment on this issue.