ReviewOS

zig-utils/zig-waf

[WAF-05] Build the ModSecurity, Coraza, and zig-waf benchmark laboratory

#6
Open chris opened this 9 days ago ·
9 days ago

Objective

Pin and build all three engines and benchmark no-rules, CRS detection, malicious blocking, bodies, concurrency, HTTP versions, throughput, p99 latency, allocation, and RSS.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

Status: zig-waf's half is built and reproducible; the comparative half needs engines this environment does not have.

Done — zig-waf measured, with the methodology pinned

  • zig build gate-request-path measures the per-request path against a rule set using @rx, @detectSQLi, @pm, @contains, and @ipMatch over mixed benign and attacking traffic, reporting p50/p99/p999, throughput, and peak RSS. Developer machine: p99 9.6 µs, 126,719 rps, 3.9 MB. CI runner: p99 21.6 µs, 62,701 rps, 3.8 MB (#40).
  • Component benchmarks with pinned inputs, all run in CI: bench-parser, bench-plan, bench-directives (against real CRS 4.28 rule files), bench-transformations, bench-ownership, bench-lifecycle, bench-scalars, bench-collections, bench-persistence, and bench-ingestion (which produced the COPY-vs-statement crossover used in #55).

Blocked — and specifically what by

A comparison is only worth publishing if all three engines run the same rules against the same requests measured at the same point, and neither counterpart can be run here:

  • ModSecurity 3.0.16 is checked out at ../modsecurity but not built; it needs an autotools build with PCRE2, libxml2, and its own dependency set.
  • Coraza 3.7.0 is checked out at ../coraza but Go is not installed, so nothing can compile or run it.

I could write a harness that measures all three and only ever exercises the zig-waf path here. I would rather not: an unrun benchmark script is exactly the kind of artifact that looks like coverage and is not, which is the pattern I have spent this session removing from this repository. The measurement approach that a fair comparison needs — identical rule set, identical request mix, the same start and stop points around inspection only, tail latency rather than an average, and RSS after sustained traffic rather than at startup — is implemented and validated for zig-waf in benchmarks/request_path.zig, and is the template the other two would follow.

To unblock: build ModSecurity (./build.sh && ./configure && make in the checkout) and install Go for Coraza. With both present, the harness is a day's work on a foundation that already exists, and its numbers would be worth something.

Sign in to comment on this issue.