ReviewOS

zig-utils/zig-waf

[WAF-17] Implement numeric, string, and zig-regex operators

#18
Closed chris opened this 9 days ago ·
9 days ago

Objective

Implement equality, ordering, begins, ends, contains, containsWord, within, string match, rx, rxGlobal, rsub, negation, captures, memoization, and bounded regex errors.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

Dependency update: WAF-11 #12 is complete at 19b64bc with exact hosted CI green. This issue can now consume the immutable typed IR, stable diagnostics, prefilters/macro programs, fingerprints, shared payloads, and explicit WAF plan ownership contract.

9 days ago

WAF-17 scalar, regex, and memoized operators landed on main and are green in hosted CI:

  • 928cbab — numeric (eq/ge/gt/le/lt) and string (beginsWith/endsWith/contains/strmatch/within/streq/containsWord) operators as a typed case-insensitive union, with the ModSecurity std::stoi and Coraza strconv.Atoi numeric profiles and negation.
  • 92160dc — retained Coraza 3.7.0 operator corpus (12 files, 118 cases) with pinned SHA-256 digests and the Apache-2.0 notice, replayed under the Coraza profile.
  • 13b08f3 — rx/rxGlobal/rsub through zig-regex: a ruleset-owned compiled program with a per-worker matcher, capture fields 0..9, empty-pattern always-match, distinguishable compile errors, and bounded runtime/match-limit reporting. rsub preserves the pinned ModSecurity 3.0.16 always-true stub rather than inventing substitution.
  • ec8c02b — an optional bounded per-worker LRU memo keyed by exact input bytes that rebuilds capture fields against the live input, with hit/miss/eviction stats.
  • 0e6d6ee — ReleaseFast bench-operators: scalar operators evaluate allocation-free in single-digit ns, and memoization serves repeat @rx inputs ~100x faster than a fresh match.
  • 4b4c2a0 — a deterministic fuzz oracle asserting scalar determinism/negation identity across both profiles and rx plain/memo-miss/memo-hit agreement with no leaks.
  • 094c460 — operator documentation and the operator.scalar-operators compatibility-matrix entry.
  • 17c11bb — CI now runs the transformation and operator fuzzers and benchmarks.

Coverage: focused unit tests, the 118-case Coraza corpus, deterministic fuzzing, and ReleaseFast benchmarks all pass on the closing SHA. Engine rule-evaluation wiring, and detectSQLi/detectXSS parity (WAF-19 #20), remain separate. The zig-regex dependency is consumed at its pinned commit; the CRS-ready release remains zig-utils/zig-regex#17.

9 days ago

The autonomous rule-execution engine is now functional — the engine can run a rule set against a request and reach a decision on its own (previously it had all the building blocks but no driver tying them together). Built in layered, tested slices:

  • e08301f rule_eval — operator dispatch by name
  • 2aa9d59 RuntimeOperator — @rx/@pm/@ipMatch compiled once, reused
  • 2628dc0 resolveTarget — variable + selector -> values (+ &count); 61a639b regex key selectors (ARGS:/pattern/)
  • 48c83ef evaluateRule — targets -> transforms -> operator (!@op negation); 9167fdd macro-expanded operator args; bda6309 multiMatch across transformation stages
  • 430b24f evaluatePhase — walks executable heads, evaluates whole chains, applies atomically, halts on an enforced disruptive decision

Verified end to end: SecRule ARGS "@rx select.*from" "t:lowercase,deny,status:403" denies a request carrying SELECT * FROM users while a benign one passes. Remaining: further operator coverage (rbl/geo/validateSchema/etc.) and capture into TX.N.

9 days ago

Complete. src/operators.zig.

Equality and ordering — eq, ge, gt, le, lt over profile-aware integer parsing, so a non-numeric operand behaves as the pinned engine's std::stoi prefix parse does rather than erroring.

String operators — beginsWith, endsWith, contains, containsWord, within, streq, strmatch. The dispatch is a closed enum whose specs table is index-locked to it by a comptime assertion, so a registry that drifts out of order fails to compile instead of mis-dispatching at runtime. Resolution is case-insensitive. Evaluation is allocation-free — it borrows the transformed value and retains nothing.

rx and captures — RegexOperator holds an immutable compiled program shared across request workers, with one Worker per worker thread so matcher state is never shared. RegexOutcome carries the capture fields and captures_present, so a non-participating optional group stays distinct from one that matched empty — which is what %{TX.1} semantics require. An empty pattern always matches, mirroring the pinned engines.

rxGlobal — evaluateGlobal reports match_count across all matches, where a plain rx reports 1.

rsub — ModSecurity 3.0.16's Rsub::evaluate is a documented stub that returns true, and zig-waf preserves exactly that. Implementing substitution semantics the baseline does not have would be a divergence from the pinned upstream, not an improvement; the reasoning is recorded at the function rather than left as an apparent omission.

Negation — evaluateNegated applies rule negation at one place, so !@operator cannot be handled inconsistently per operator.

Memoization — an opt-in per-worker bounded LRU over recent @rx inputs (workerWithMemo, MemoLimits), with MemoStats reporting hits, misses, and evictions so its value is measurable rather than assumed.

Bounded regex errors — Timeout, InputTooLong, and DfaOverflow are mapped to an outcome carrying both runtime_error and limit_exceeded, so a rule that hit a bound is distinguishable from one that did not match, and a pathological pattern degrades to a reported limit instead of unbounded work.

Covered by the default zig build test (376+ tests) and tests/operator_evidence.zig.

Sign in to comment on this issue.