ReviewOS

zig-utils/zig-waf

[WAF-26] Implement JSON, URL-encoded, and raw body processors

#27
Closed chris opened this 9 days ago ·
9 days ago

Objective

Implement streaming JSON with depth and duplicate-key policy, ARGS mapping, URL encoding, raw-body access, processor selection, malformed-input handling, and error variables.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

WAF-08 (#9) is complete and its collection contract is now available on main at 531cc68. Hosted qualification is green: https://github.com/zig-utils/zig-waf/actions/runs/29935997519

This removes the WAF-08 dependency for this issue. Preserve the published ownership, phase-gating, origin, bounded-storage, selector/error, and derived-family contracts when implementing this scope.

9 days ago

First WAF-26 slices — the URLENCODED and JSON request body processors — landed on main (enabled by the WAF-24 body buffering):

  • c95f6ea — the URLENCODED processor parses the buffered application/x-www-form-urlencoded body into ARGS_POST/ARGS with the same pinned Coraza decoding and configured SecArgumentSeparator as the query string.
  • 160465c — the JSON processor flattens the body into dotted ARGS_POST keys rooted at json, with array indices (json.items.0) and a per-array length entry (json.items), matching the pinned Coraza flattening; invalid JSON sets the processor error flag. The RAW processor is implicit (REQUEST_BODY with no argument extraction).

Engine-integration tests cover both. Multipart (WAF-25) and XML (WAF-27) processors remain.

9 days ago

Response-body JSON processor landed, and the request-side ctl override was fixed:

  • 4218124 feat(engine): flatten JSON response bodies into RESPONSE_ARGS (opt-in ctl:responseBodyProcessor, same dotted json.* flatten as the request side, RES_BODY_PROCESSOR + RES_BODY_PROCESSOR_ERROR)
  • 266fd47 fix(engine): honor ctl:requestBodyProcessor over the Content-Type-derived processor
  • 8cbb7b0 docs(engine): record the response processor in the compatibility matrix and ctl reference

Remaining under this issue: multipart and XML/raw processors.

9 days ago

XML body processor landed, completing the JSON/URL-encoded/multipart/XML/raw processor set for this issue.

To parse XML without pulling a C dependency, I checked ~/Code/Home/lang (no reusable XML parser there, but a self-contained HTML5 entity table) and wrote a new from-scratch package zig-utils/zig-xml@0.1.0 — a lenient, allocation-light XML/HTML tokenizer (Reader/Event/decode), XXE-safe (no DTD/external entities), released to the pantry registry via the same flow as our other zig packages.

zig-waf commits:

  • 9d6d350 build(deps): consume zig-utils/zig-xml@0.1.0 from the registry.
  • e889b17 feat(engine): inspect XML request bodies — populates the XML collection under //@* (attribute values) and /* (text/CDATA), the two selectors OWASP CRS inspects, pinned to Coraza's XML processor; entities decoded, CDATA literal.
  • c73fd39 docs(engine): matrix + request-parsing reference.

Remaining body-processing work: multipart temp-file spooling (FILES_TMPNAMES), tracked on #26.

9 days ago

Complete.

Streaming JSON — the processor now streams the scanner rather than materializing a value tree (b7cbb06). That change is what makes the two policies below expressible, and it means a truncated body is inspected as far as it was read instead of hiding its contents behind a parse failure.

Duplicate-key policy — this was a bypass. std.json's default duplicate-field behavior is an error, so a body containing one repeated key was refused outright, published no arguments, and every rule inspecting ARGS saw an empty body. One duplicate key was enough to slip a payload past the entire ruleset.

Now a duplicated key publishes every value under the same argument. Every JSON parser accepts such a body and they disagree about which value wins, so inspecting only one leaves the other unfiltered for whatever the application picks. The test asserts a rule catches the privileged value whether it appears first or second, because which one a given parser keeps must not matter.

Depth — flattening costs a frame per nesting level and std.json happily accepts a hundred thousand of them, so a few kilobytes of [ from any client overflowed the stack and took the process down (5aa4ac6). SecRequestBodyJsonDepthLimit existed as a directive and was never read — which is how the ceiling came to be missing. It is now enforced, defaulting to Coraza's 1024, and a body past the limit is refused with no arguments published, since a half-flattened body would let a rule conclude it had seen everything.

ARGS mapping — dotted json.* keys, array indices, and array-length entries, matching Coraza; unchanged by the rewrite, which the existing body-processor and audit tests hold to.

URL encoding — ARGS_GET/ARGS_POST decode x-www-form-urlencoded with the pinned Coraza queryUnescape, honoring SecArgumentSeparator, and a malformed escape raises URLENCODED_ERROR (see #24).

Raw-body access — REQUEST_BODY exposes the buffered bytes regardless of which processor ran, so a rule can inspect what was actually sent rather than only what a processor made of it.

Processor selection — from Content-Type for requests (URLENCODED, JSON, MULTIPART, XML), overridable by ctl:requestBodyProcessor; opt-in only for responses, matching Coraza's deliberate absence of response Content-Type sniffing.

Malformed input and error variables — REQBODY_PROCESSOR_ERROR/REQBODY_PROCESSOR_ERROR_MSG on the request side and RES_BODY_PROCESSOR_ERROR on the response side, published as 0 from the start so a rule reading one never has to distinguish "no error" from "never parsed".

Compatibility matrix updated (ea6b247).

Sign in to comment on this issue.