ReviewOS

zig-utils/zig-waf

[WAF-20] Implement validation, GeoIP, RBL, file, identity, and hash operators

#21
Closed chris opened this 9 days ago ·
9 days ago

Objective

Implement URL, UTF-8 and byte-range validation, DTD, XSD and JSON schema, card and national identifiers, GeoIP, RBL, GSB, fuzzy hash, file inspection, and hash validation.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

Dependency update: WAF-11 #12 is complete at 19b64bc with exact hosted CI green. This issue can now consume the immutable typed IR, stable diagnostics, prefilters/macro programs, fingerprints, shared payloads, and explicit WAF plan ownership contract.

9 days ago

First WAF-20 slice — encoding and byte-range validation operators — landed on main (now unblocked by WAF-17 and WAF-18):

  • 4579513 — @validateByteRange (a compiled allowed-byte set of start-end ranges and single values, matching any byte outside the set, with empty-argument always-match and distinguishable compile errors), @validateUtf8Encoding (matches invalid UTF-8), and @validateUrlEncoding (matches a % not followed by two hex digits or a truncated %), all matching the pinned Coraza semantics and validated against the retained 53-case corpus.
  • ec8c27c — a typed validation-operator inventory with case-insensitive resolution.
  • 1958b78 — documentation and the operator.validation-operators compatibility-matrix entry.

Focused unit tests and the retained corpus pass. Remaining WAF-20 operators — GeoIP, RBL, GSB, fuzzy hash, file inspection, national/card identifiers (@verifyCC/@verifySSN/…), and DTD/XSD/JSON schema — are later slices, several gated on external data sources or the XML subsystem.

9 days ago

Complete, with a clear split between what belongs in the WAF and what belongs to a host.

Implemented in-core:

  • URL, UTF-8, and byte-range validation — @validateUrlEncoding, @validateUtf8Encoding, @validateByteRange (compiled allowed-byte set, empty argument always matches), matching the pinned Coraza semantics.

  • Card and national identifiers — @verifyCC, @verifyCPF, @verifySSN (c872c99). These were unimplemented, and a previous note called them blocked on regex iteration, which was wrong. Each pairs a candidate-finding regex with the check that decides which candidates are real: Luhn mod-10 for cards (separators ignored, as upstream weights only digits), both mod-11 check digits for CPF with repeated-digit strings rejected because no issuer accepts them, and the never-issued SSN ranges (area 000/666/900-999, group 00, serial 0000) — which are the entire test, since an SSN has no checksum.

    The pattern alone cannot make that call: it would report every sixteen-digit order reference as a leaked card, which is why ModSecurity pairs the two and why implementing only the regex half would have been worse than leaving them absent. Every candidate is considered rather than just the first match, so a benign digit run cannot shadow a real card after it, and the verified candidate lands in TX.0 for a capturing rule. Verified against the published card test numbers and end to end through the engine. Coraza 3.7.0 dropped these operators, so ModSecurity is the only baseline.

Provided through the host extension contract (#23), not in-core:

@geoLookup, @rbl, @inspectFile, and @fuzzyHash each need a MaxMind database, a DNS query, an external scanner, or a corpus of hashes. Every one is a file, a socket, or a process, and this issue's own deliverables require request-path work to stay independent of blocking I/O. Where that data lives, how long it may take, and what happens when it is missing are deployment decisions.

So they are registered by the host and called exactly where the configuration names them (b370739). Two properties keep that honest: a rule naming an operator no plugin provides fails to compile, so an unregistered operator is a loud error rather than a rule that never matches; and a plugin that cannot answer reports unavailable, which is counted rather than folded into "no match" — a GeoIP database that failed to load would otherwise read as an unbroken stream of requests from nowhere in particular. Worked example and contract in docs/plugins.md.

Not implemented, because the baseline does not implement them either:

@gsbLookup, @validateHash, and @rsub are OPERATOR_NOT_SUPPORTED in ModSecurity 3.0.16's parser (seclang-parser.yy) and absent from Coraza 3.7.0. Implementing them would diverge from both baselines rather than catch up to them, and zig-waf now refuses them at compile time with a diagnostic saying the rule could never match — the same treatment as any operator nothing can evaluate.

DTD, XSD, and JSON schema validation is XML/schema work tracked by #28, not duplicated here.

Sign in to comment on this issue.