ReviewOS

zig-utils/zig-waf

[WAF-04] Establish CI, reproducible builds, SBOMs, and no-submodule checks

#5
Closed chris opened this 9 days ago ·
9 days ago

Objective

Add Pantry-only toolchain setup, format/test/fuzz/benchmark jobs, dependency pin verification, license checks, secret scanning, reproducible artifacts, SBOMs, and signatures.

Deliverables

  • Implement the complete scoped behavior described above.
  • Keep request-path work independent from UI, PostgreSQL, and blocking audit I/O.
  • Use Zig 0.17-dev and dependencies pinned through Pantry; do not add Git submodules.
  • Keep public APIs explicit about ownership, resource limits, errors, security policy, and compatibility.
  • Add focused unit, integration, differential, fuzz, benchmark, and documentation coverage as applicable.

Acceptance criteria

  • The scoped implementation and tests are merged on main in small conventional commits.
  • Zig formatting, focused tests, and the relevant CI suite pass.
  • Compatibility claims have executable evidence and no unexplained skips or silent fallbacks.
  • Untrusted inputs have deterministic bounds; hot paths have benchmark evidence.
  • Documentation and the machine-readable compatibility matrix are updated.

Relationships

Blocked by

Blocks

Baselines

  • ModSecurity 3.0.16
  • Coraza 3.7.0
  • OWASP CRS 4.28.0
  • libinjection 4.0.0
  • Zig 0.17-dev resolved and locked through Pantry
9 days ago

Complete for everything achievable in this repository; one item needs a key it does not have, and is not claimed.

Pantry-only toolchain — the runner installs Pantry and resolves the Zig compiler and every system package through it. No system toolchain is assumed.

Format, test, fuzz, and benchmark jobs — zig fmt --check over build.zig, src, benchmarks, and tools; the full test suite; zig build check; eight deterministic fuzz suites with pinned seeds (parser, plan, directives, bodies, audit, actions, transformations, operators); and parser, plan, and directive-validation benchmarks against real CRS rule files.

Dependency pin verification — this one taught me something. The obvious check, "installing must not change a manifest", cannot hold: naming a package re-adds it to deps.yaml, and a bare pantry install strips the Zig dependencies out of build.zig.zon entirely, which breaks configure. An assertion that cannot hold is worse than none, because the only ways out are to weaken it into meaninglessness or to work around it.

So CI restores the committed manifests after installing — they are the source of truth for what the build uses — and checks the property that does hold: every dependency the repository declares appears in the published SBOM. A dependency added without reaching the SBOM is precisely the gap an SBOM exists to close. Aliases are listed explicitly rather than prefix-matched, since a loose rule would also swallow a genuinely missing dependency.

SBOMs (8866a69) — zig build sbom emits CycloneDX 1.5 from the repository's own manifests: Zig packages by content hash from build.zig.zon, system packages by version from pantry.lock. Generated from the manifests rather than an environment scan, because a scan reports what happened to be installed on the machine that ran it while the manifests are what the build is defined to use — and an SBOM that disagrees with the build is worse than none, since it is trusted. Uploaded as an artifact per run.

Its first version silently skipped four dependencies whose lock entries carry source instead of resolved — the same silently-dropped-input pattern this session has been removing, in the tool whose entire purpose is completeness. It now accepts either form and fails on an entry it does not understand.

Reproducible artifacts — the SBOM's timestamp is an input rather than the clock, so two builds of the same commit produce byte-identical documents. That is the piece of reproducibility this repository controls; the compiler and its flags are already pinned through Pantry.

Secret scanning — patterns live in .github/secret-patterns.txt so the step cannot match itself, and that file is the only exclusion. A floor rather than a substitute for a scanning service: it catches the accident of pasting a key into a fixture, which is how it usually happens.

No Git submodules — asserted directly; upstream baselines are checked out beside the repository rather than vendored into it.

License checks and signatures — not implemented, and not claimed. Signing needs a key this repository does not have; adding a step that signs with a throwaway key would produce a signature that proves nothing. License verification needs per-package license metadata that the lockfile does not carry today; the honest prerequisite is that Pantry expose it, and asserting a license I inferred would be worse than not asserting one.

Sign in to comment on this issue.