ReviewOS

home-lang/home

Wire lexer + parser fuzzing into CI

#10
Closed glennmichael123 opened this 24 days ago · 0 comments
24 days ago

Context

packages/fuzz exists in the tree, but I don't see a CI job invoking it. Lexer and parser fuzzing is the cheapest, highest-value safety net for a compiler:

  • Catches infinite loops, stack overflows, and panics on adversarial input
  • Catches error-recovery bugs that produce nonsense ASTs
  • Catches O(n²) or worse pathological inputs
  • Almost zero ongoing cost once it's running — corpus grows itself

Proposal

  • Set up a structured fuzz target for the lexer (input: arbitrary bytes → expect: never panic, always make progress)
  • Set up a structured fuzz target for the parser (input: lexer output → expect: never panic, error recovery terminates)
  • Set up a structured fuzz target for the type checker on grammatically-valid inputs
  • Add a CI job that runs each fuzz target for ~5 minutes per PR (cheap budget, catches the obvious stuff)
  • Add a nightly job that runs each for ~1 hour with corpus carryover via cache
  • Commit a seed corpus from existing examples so fuzzing starts from a useful baseline
  • On crash, store the minimal repro as a regression test in tests/feature/

References

  • Zig itself uses libfuzzer-style harnesses
  • Rust's fuzz suite has caught hundreds of compiler bugs
  • cargo fuzz is a good UX model

Sign in to comment on this issue.