ReviewOS

zig-utils/zig-waf

[WAF-29] Implement native, legacy JSON, JSON, and OCSF audit formats

#30
Closed chris opened this 9 days ago ·
9 days ago

Objective

Implement audit parts A through K and Z, all required formats, matched-rule metadata, uploaded file parts, deterministic serialization, and compatibility snapshots.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

WAF-14 #15 is complete and unblocks this issue at exact green SHA 28380d45c92a01a9ed48a7b63784ae26a59a5bab (CI: https://github.com/zig-utils/zig-waf/actions/runs/29959702219).

9 days ago

Started the audit-log format subsystem in a new src/audit.zig (an AuditRecord snapshot model + pure, deterministic serializers):

  • 4bd0a0b native serial format — pinned to ModSecurity's toOldAuditLogFormat: dashed --<boundary>-A-- … -Z-- sections, A–K part gating, Apache-style UTC timestamp, blank-line record terminator.
  • f1da3f7 structured JSON format — Coraza's audit schema ({"transaction":{…},"messages":[…]}), grouped header arrays, JSON escaping, part-gated bodies; tested via std.json round-trip.

Both green. Remaining for this issue: the legacy-JSON and OCSF formats, then wiring a transaction snapshot into the engine's logging phase.

9 days ago

Audit-log format subsystem is functionally complete: all four formats plus engine integration.

  • 382bbab engine integration — Transaction.serializeAuditLog(allocator, format) snapshots the transaction into an AuditRecord (identity, connection endpoints, request line, request/response headers from their collections, bodies, status, wall-clock timestamp, interruption) and serializes it, honoring SecAuditLogParts. Part-H rule messages are formatted from the accumulated match intents flagged auditlog, in ModSecurity's RuleMessage::log layout. An engine test drives a real disruptive auditlog rule and asserts the serial + JSON output.
  • Formats: 4bd0a0b native serial, f1da3f7 JSON, 09e0a00 legacy JSON, 722ea82 OCSF v1.2.0.
  • 57a2543 docs + compatibility matrix.

Remaining: the audit writers (serial/concurrent/syslog/HTTPS — the I/O layer, #31) and redaction/telemetry (#32).

9 days ago

Complete.

Parts A–K and Z — a transaction snapshots into an AuditRecord (identity, connection endpoints, request line, request and response headers taken from their collections, bodies, status, timestamp, interruption) and serializes according to SecAuditLogParts. Part H carries per-rule messages formatted in ModSecurity's RuleMessage::log layout.

All required formats — four, each pinned to a specific upstream rather than invented: native serial (ModSecurity's toOldAuditLogFormat), structured JSON and legacy JSON (Coraza's schemas), and OCSF v1.2.0 Web Resources Activity (Coraza's OCSF formatter).

Matched-rule metadata — id, message, logdata, severity, and tags, in upstream's bracketed order, with the disruptive form ("Access denied with code N") distinguished from the warning form.

Uploaded file parts — multipart file parts populate FILES/FILES_NAMES/FILES_SIZES/FILES_COMBINED_SIZE and the MULTIPART_* variables, which the record carries.

Deterministic serialization — a fixed field order and an I/O-free Sink boundary, with serial, callback, and ModSecurity concurrent-layout writers in src/audit_writer.zig, so serialization never depends on delivery.

Compatibility snapshots — retained fixtures under src/compatibility/evidence/, exercised by the default zig build test.

One correction landed with this issue (ce9d6b1). sanitizeMatchedBytes was classified as a known action and had no implementation, so a rule carrying it compiled and ran while the value it claimed to redact went into the audit log in full — a control that looks configured and does nothing. It is now rejected at compile time, which is also what both baselines do: ModSecurity 3.0.16's parser rejects it outright ("Action: SanitiseMatchedBytes is not yet supported") and Coraza 3.7.0 has no such action. Both spellings are refused.

That investigation also found the matrix overstating parity: it described redaction as porting the ModSecurity sanitise actions, when upstream rejects every sanitise action. sanitizeArg, sanitizeMatched, and the header variants are a deliberate zig-waf extension, and the matrix now says so.

Debug logs, Prometheus, and OpenTelemetry belong to #32 and are tracked there, not here.

Sign in to comment on this issue.