Automation
Bring the CI you already have
ReviewOS does not run your tests. It accepts reports from whatever does, and enforces them where it counts.
A check is reported against a commit
Not against a pull request. The same report then serves every pull request whose head is that commit, and it stays meaningful when a branch moves.
The awkward cases decide whether this is useful
Any implementation handles green and red. What separates a useful checks system from an annoying one is everything in between.
- A retried check is judged on its latest run, so one failed attempt does not keep a commit red forever
- Neutral and skipped count as passing, because a path-filtered job that had nothing to say has not found a problem
- Cancelled and stale are neither pass nor fail, because something will report again
- A check nobody marked required never blocks a merge, however red it is
Never reported is not the same as still running
A required check that has never reported means the workflow is not wired up and never will report. A pending one resolves itself. The person waiting is told which of the two they are looking at, because the actions are completely different.
Stale runs do not carry over
When a branch is force pushed, checks that ran against the old head describe code nobody is merging. Leaving them green is how an unverified commit slips past a required check.
Keep your pipeline, gain the gate
A status API is deliberately the first step. Running untrusted code is a security project of its own.