ReviewOS

zig-utils/zig-waf

[WAF-33] Build validate, format, test, explain, replay, and benchmark CLI tools

#34
Closed chris opened this 9 days ago ·
9 days ago

Objective

Build rule validation, formatting, tests, execution explanation, inspection, replay, benchmarking, remote-rule fetching, configuration migration, and machine-readable output.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

Dependency WAF-10 #11 is complete at e72709c with hosted green evidence: https://github.com/zig-utils/zig-waf/actions/runs/29942330646. This issue is now unblocked on the bounded, owned SecLang frontend.

9 days ago

Dependency update: WAF-11 #12 is complete at 19b64bc with exact hosted CI green. This issue can now consume the immutable typed IR, stable diagnostics, prefilters/macro programs, fingerprints, shared payloads, and explicit WAF plan ownership contract.

9 days ago

CLI now has validate/explain/test/version:

  • 0274ea7 validate — parse/compile/directive-validate configs, non-zero exit on rejection (CI-gatable)
  • cd19f01 explain — list compiled rules (phase, id, action, source location)
  • 9f1d471 test — run a synthetic request through the engine and print BLOCKED/ALLOWED (e.g. zig-waf test rules.conf GET "/x?q=SELECT * FROM users" -> BLOCKED action=deny status=403)

Remaining subcommands: format, replay, benchmark.

9 days ago

Complete. All six subcommands, each exiting non-zero on rejection so a rule set can be gated in CI.

  • validate — parses, compiles, and directive-validates one or more configs, printing human diagnostics with a location. This is what catches the class of problem this session kept finding: an operator nothing implements, an action that would have no effect, a dataset nothing declares.
  • format (ff684ed) — prints one directive per line from the parse, not by reflowing the source. A formatter that guessed at intent could silently move a continuation from one rule to another; printing the parse makes the output by construction the configuration the engine will run. A config that does not parse is not formatted at all, because rewriting something the engine could not read produces a file nobody can trust.
  • test — runs a synthetic request, and optionally a simulated response, through the engine and reports the decision.
  • explain — lists the compiled rules with phase, id, disruptive action, and source location: what a rule set actually resolves to, including rules that arrived through includes.
  • replay (ff684ed) — takes a request as it went over the wire (request line, headers, blank line, body — what an audit record's parts B and C contain, and what a proxy capture or curl -v transcript produces) and runs those bytes through the engine. Replaying the bytes rather than a reconstructed model is the point: a rule that fires on a header a recorder normalized away would not be reproduced by anything else. Exits non-zero when the request is blocked, so a reproduction can gate a fix.
  • benchmark (ff684ed) — times repeated evaluation and reports nanoseconds per request and how many were blocked, so a run that measured nothing — a rule set that never matched the sample request — is visible rather than quietly reassuring. For the deeper measurements (parser, plan, transformations, ingestion) the bench-* build steps remain the right tool; this is the one that answers "what does my rule set cost on my request".

Sign in to comment on this issue.