[WAF-18] Implement phrase, dataset, Aho-Corasick, and IP operators

Objective

Implement phrase matching and files, datasets, Aho-Corasick compilation, IPv4 and IPv6 CIDR matching, prefix structures, caching, and reload behavior.

Deliverables

• Implement the complete scoped behavior described above.
• Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
• Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
• Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
• Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

• The scoped implementation and tests are merged on main in small conventional commits.
• Zig formatting, focused tests, and the relevant CI suite pass.
• Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
• Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
• Documentation and the machine-readable compatibility matrix are updated.

Relationships

• Parent: WAF-00

Blocked by

• WAF-11

Blocks

• WAF-20
• WAF-34

Baselines

• ModSecurity 3.0.16
• Coraza 3.7.0
• OWASP CRS 4.28.0
• libinjection 4.0.0
• Zig 0.17-dev resolved and locked through Pantry

Dependency update: WAF-11 #12 is complete at 19b64bc with exact hosted CI green. This issue can now consume the immutable typed IR, stable diagnostics, prefilters/macro programs, fingerprints, shared payloads, and explicit WAF plan ownership contract.

WAF-18 phrase (Aho-Corasick) and IP CIDR operators landed on main and are green in hosted CI:

• b0a9479 — an ASCII case-insensitive Aho-Corasick automaton: an immutable, shareable matcher with O(1)-per-byte membership through a precomputed failure-chain terminal flag, bounded pattern/state counts, empty-pattern always-match, and a non-overlapping capture iterator.
• 6dcbee1 — an IPv4/IPv6 CIDR matcher: dotted-quad and zero-compressed IPv6 parsing, the pinned bare-address /32 and /128 defaults, unparseable-token skipping, and cross-family safety.
• c3e3bbe — @pm and @ipMatch wired to the automaton and matcher, validated against the retained Coraza pm (15) and ipMatch (3623) corpora, including the Go net.To4 IPv4-mapped IPv6 normalization edge cases.
• f5f159a — @pmFromFile/@ipMatchFromFile line parsing (trim, skip blank and # lines), a compilePatterns dataset path, and a typed matcher-operator inventory resolving pm/pmFromFile/pmf/pmFromDataset/ipMatch/ipMatchFromFile/ipMatchF/ipMatchFromDataset.
• e12626f — ReleaseFast benchmarks (@pm ~170ns over a keyword denylist, @ipMatch ~18ns) and fuzz coverage asserting the matchers never crash, stay deterministic, and produce in-bounds monotonic iterators.
• cb8ee66 — matcher documentation and the operator.phrase-ip-operators compatibility-matrix entry.

Coverage: focused unit tests, the retained Coraza pm + ipMatch corpora (3638 cases), deterministic fuzzing, and ReleaseFast benchmarks pass on the closing SHA. Dataset-registry and reload wiring land with the engine dataset subsystem.

File-backed operators now load their data files (9011d04). @pmFromFile / @ipMatchFromFile previously treated their argument as an inline phrase/CIDR list, so the filename itself became the only "phrase" and the file was never read.

Implementation resolves the data file once at compile time (runtime never touches the filesystem): a new seclang.include.readDataFileAlloc resolves the filename relative to the referencing rule file and confines the canonical path to the configuration root (reusing withinRoot, so ../symlink escapes are rejected and absolute paths are opt-in); the compiler gained an optional DataProvider (compileWithProvider / compileTreeWithProvider) that interns the file's bytes in place of the filename; runtime_operator compiles the from-file variants via the newline-delimited path; and the CLI supplies a provider rooted at the config directory. Unit tests cover readDataFileAlloc (within-root read + traversal/symlink/absolute/missing/too-large rejections) and end-to-end @pmFromFile/@ipMatchFromFile matching through a provider.

Still open on this issue: @pmFromDataset / @ipMatchFromDataset (SecDataset registry) — parsed but not yet materialized.

Complete. The dataset half was genuinely missing and is now implemented rather than declared out of scope.

Phrase matching and Aho-Corasick compilation — src/phrase.zig builds an immutable automaton (nodes + goto/fail edges) from a space-separated pattern list, shareable across request workers because it is never mutated after compilation. ASCII case-insensitive, matching upstream. An empty pattern matches at every position and is represented explicitly (always_match) rather than as a degenerate automaton.

Files — @pmFromFile / @pmf resolve at plan-compile time through a DataProvider, which reads relative to the rule file that named it; the runtime then compiles from newline-delimited bytes with comments and blank lines skipped. Without a provider the filename is interned unchanged rather than silently loading from ambient state.

Datasets — @pmFromDataset and @ipMatchFromDataset were recognized by name but had no source of patterns: Coraza's SecDataset name backtick block did not even lex, and once lexed the operators fell through to the compile-free path and matched nothing. Both are now implemented (d11a1bd, f958fc6):

• The lexer learned Coraza's multi-line block: a value that spans physical lines without continuation markers, keeping the newlines that separate its entries. Inside a block nothing else is syntax — a lone quote, a #, a trailing backslash are all data, asserted directly — and an unterminated block reports UnterminatedBlock instead of swallowing the rest of the file.
• Declarations are collected before any rule is compiled, so a dataset may be declared after the rules that read it, which is how a bundled ruleset ships its data in a separately included file.
• A rule naming an undeclared dataset fails to compile. Compiling it to an empty set would be the worst outcome for a security control: configured-looking and detecting nothing.
• Verified end to end: a request carrying a dataset phrase and a client inside a dataset subnet are denied; non-matching ones pass.

ModSecurity has no block form and no dataset operators, so nothing here changes how a ModSecurity configuration lexes — a backtick could not appear unquoted in one.

IPv4 and IPv6 CIDR matching — src/ip_match.zig parses both families, including :: zero-compression and trailing IPv4 in an IPv6 address. Bare addresses take /32 and /128 as upstream does, and an IPv4-mapped IPv6 address compares as IPv4 so it can never match a genuine IPv6 subnet — the aliasing bug that would otherwise let a blocklist be bypassed.

Prefix structures and caching — subnets are stored as (network, prefix) pairs per family for masked comparison; compiled phrase automata and subnet sets are ruleset-owned and reused across transactions, with per-worker mutable state kept separate from the shared immutable program.

Reload behavior — pattern sets belong to a compiled plan, and plans are swapped atomically (Runtime.reload), with in-flight transactions pinned to the generation they started on. A reload therefore never shows a transaction a half-replaced dataset.

Follow-up: the lexer change that landed with this issue (d11a1bd) made a backtick a delimiter on every line, and OWASP CRS 4.28 rule 932370 carries one inside a regex character class. Its rule continues across several lines, so the block swallowed the continuation and REQUEST-932-APPLICATION-ATTACK-RCE.conf stopped parsing — 69 rules silently lost from a file that had compiled before.

Fixed in ce90bf6 by scoping blocks to SecDataset lines, which is the only directive that takes one; a backtick anywhere else is an ordinary byte again. That is narrower than trying to make quote tracking survive an adversarial regex, and it cannot affect a rule at all.

Regression-tested with the exact shape that broke — a backtick inside an @rx argument on a continued line — and every CRS 4.28 rule file now validates, with dataset blocks still parsing. The gap was found by validating the whole ruleset rather than the dataset fixtures alone, which is the check that should have run when the lexer changed.