15 - Pipelines

Phase 9 builds the machinery: a check API, a durable workflow control plane, a runner contract, and a gate in front of ever executing somebody else's code. This phase is the product that machinery has to add up to.

It has two competitors rather than one, and they are not competing for the same thing. Getting this distinction wrong is the most expensive mistake available in this phase, so it goes first.

GitHub Actions is the familiarity target. It is what almost everyone arriving here already knows. They have .github/workflows/ci.yml files that work, muscle memory for runs-on and needs: and uses: actions/checkout@v6, and no appetite whatsoever for learning a second CI language in order to leave GitHub. What they want is Actions that they own: same syntax, same ecosystem, running on their hardware, with the reliability and the visibility that GitHub does not give them. If somebody cannot copy a working .github/workflows directory across and watch it go green, nothing else in this phase matters.

Buildkite is the capability target. It is what Actions turns into when a company outgrows it, and it is the reference for the engine underneath: concurrency groups, dynamic step generation, runner fleet management, signed steps, test intelligence. Buildkite sells to people who already hit the ceiling Actions has.

So: Actions syntax on the front, Buildkite-grade engine underneath. Those are compatible goals, not a compromise between two, and the rest of this file is written on that assumption. Where the two conflict, Actions compatibility wins on the authoring surface and Buildkite wins on what the engine can do once a workflow is parsed.

Why the engine is modelled on Buildkite

Buildkite's model is hybrid: they run the control plane, you run the compute. Their agent is a small cross-platform binary that you install on your own machines; it polls their API for work, runs it on your hardware, inside your network, with your secrets, and reports back. Buildkite never sees the source code, never holds the secrets, and never executes anything.

That is the shape phase 9 arrived at independently, from the opposite direction. Phase 9 splits the durable control plane from the execution plane and puts a security review in front of the second one because running untrusted repository code on instance-managed infrastructure is a separate project with its own threat model. Buildkite made that same split a business model.

Which means the expensive, dangerous half of what Buildkite sells is the half we are deliberately not building yet, and the half that is genuinely hard to copy is a control plane, an API, and a set of screens.

The last row is the one that matters and the only one a competitor cannot copy by changing a price. Buildkite has to report into somebody else's pull request through a status API, so its richest output, the annotations, the flaky test verdict, the artifact, the log, lives on a page in another tab. Actions has the pull request but spends it on a check summary and a link to a log viewer. Here it lands on the diff, on the line, in the review. Phase 9 already states the rule ("an annotation shown only in a log is a link nobody clicks"); this phase is where it gets paid off.

Vocabulary, decided once

Good news first: phase 9's chosen names are already Actions' names. A workflow is a workflow, a run is a run, a job is a job, a step is a step, and the thing that executes them is a runner. Nobody coming from Actions has to relearn a noun. The table below exists because Buildkite calls all five of those something else, and the Buildkite word must not creep in as a synonym.

• A test that greps routes, actions, models, and generated OpenAPI for pipeline, build, agent, and meta-data used in the Buildkite sense, and fails. This is the same class of rule as "never repo", and the same reason: a synonym that lands once is permanent.

  `tests/unit/pipeline-vocabulary.test.ts`, checked against the generated document, the model
  names and the action names - not against comments, because prose *about* Buildkite is how the
  decision gets explained. It also asserts the Actions nouns are the ones in use, since a guard
  that only bans things passes on an empty codebase.
  

GitHub Actions compatibility

This section is the front door, and it is load-bearing for adoption in a way nothing else here is. Phase 9 leaves the authoring contract open ("a constrained TypeScript API, a declarative format, or both. Document the portability and security costs before choosing ecosystem compatibility"). This phase closes that box: the canonical format is Actions-compatible workflow YAML. The reasoning is written down here so it does not get relitigated:

• The ecosystem is the product. actions/checkout, actions/setup-node, actions/cache and a few thousand others are what a workflow is actually made of. A format that cannot run them starts at zero no matter how good it is.
• The syntax is already an industry default. Gitea and Forgejo both chose compatibility over invention, and it is the single reason a repository can move to either of them in an afternoon.
• Everything Buildkite can express, this file has to express anyway. Almost all of it fits as additive keys on a familiar shape rather than as a different language.
• A typed authoring SDK stays on the list, further down, but it emits the same normalized graph. It is a second front door, not the only one.

The bar

• Copy .github/workflows/ to .reviewos/workflows/, push, and a normal repository's CI runs green with no edits. This is the acceptance test for the whole section, run against real workflow files from real repositories rather than ones written to pass.

  `tests/e2e/acceptance-workflow.test.ts`, over `tests/fixtures/acceptance/bumpx-ci.yml` - a
  verbatim copy of a real repository's workflow, four jobs, three of them parallel and one
  waiting on all three, with `actions/checkout`, `oven-sh/setup-bun`, `actions/cache` and a
  subdirectory reference into another repository's action. Nothing in the test edits it. Push,
  claim, run, green: the registration, the graph, the claim protocol, the steps, `GITHUB_PATH`,
  a working directory, a shell heredoc with a `timeout` and an `if:` on the last step.
  
  **What is simulated, said plainly.** This machine has no network, so the four actions the file
  names resolve through an origins map pointing at local git repositories - the configuration an
  air-gapped instance has, and one this product supports on purpose. Each mirror does what the
  hosted action does locally: the checkout has already happened, the toolchain is on `PATH` -
  written there through `GITHUB_PATH`, the same mechanism the real one uses - and there is no
  cache to restore. So the path this product owns is proven end to end; somebody else's action
  doing what it does on GitHub's runners is not, and that needs a machine with a network rather
  than more code here.
  
  `.reviewos` **wins outright over `.github` rather than merging** - merging runs every job twice
  the day somebody forgets to delete the original, and "which of these two files ran" is a
  question nobody should have to ask.
  

• .github/workflows/ is also read directly, so a mirrored repository (phase 13) runs its existing workflows without a commit that would have to be undone to go back.

  Read from the trusted ref with plumbing, nothing checked out. A repository that arrives with
  workflows registers them on its first push here, and one that later adds `.reviewos/workflows`
  switches over without the two ever running together.
  
  Testing this turned up an older bug with nothing to do with directories: **a workflow whose
  file was deleted stayed `active` forever**, and dispatch reads `state = 'active'`, so a
  repository that removed its CI kept starting runs from a definition that was no longer in the
  tree. A file that is gone now retires its workflow, in a `removed` state kept apart from
  `disabled` - a workflow somebody switched off has to stay off when the file comes back, and
  one state for both would let a revert resurrect it.
  

• A conformance suite pinned to a corpus of widely used public workflows, run in CI, reporting which constructs pass, which are unimplemented, and which are refused on purpose. The report is published. Silence about a gap is how Gitea's ignored concurrency: surprised people.

  The corpus is in `tests/fixtures/conformance/` - workflows in the shapes people actually
  write: a version matrix, a release on a tag, a container build with services, a reusable
  workflow, a nightly with a composite action, an issue labeller. Every one is parsed on every
  test run, and a failure names the file and the errors rather than a count.
  
  The report is [`docs/conformance.md`](../conformance.md), generated from a table that is the
  source of truth, with a drift test: a key whose behaviour changes without its line changing
  fails rather than misleading somebody quietly for a year.
  
  It caught its first bug immediately, which is the argument for having it: **a workflow
  triggered only on `issues` was refused as naming no event at all.** The trigger had been
  implemented, dispatched and tested, and left out of the one list that decides whether a file
  is valid - so a labeller, the second thing anybody automates, could not be registered.
  

• Where behavior deliberately differs from GitHub, it is documented per key with the reason, and the parser emits a warning naming the difference rather than quietly doing something else.

  **One table drives all three.** The published page, the conformance test and the parser's
  warnings read the same data, which is the only arrangement where a difference cannot be
  documented one way and behave another - and where adding a divergence without writing down its
  reason is impossible rather than merely discouraged. It lives in the domain rather than under
  `app/Docs/`, because it is data that happens to be published rather than documentation that
  happens to be checked.
  
  A workflow using a key that differs, or one that is not implemented yet, carries the warning
  on its version and shows it on the workflows page in the published table's own words. A
  workflow that fails to parse carries none: its author has errors to fix, and "by the way,
  `container:` behaves differently here" underneath them is noise on top of a problem.
  
  An ordinary `ci.yml` gets exactly two notes - `permissions` defaults differently here, and
  `fail-fast` is stored but not acted on yet - which is the standard this box asks for: both are
  things somebody would otherwise discover by watching a run behave unexpectedly.
  

Workflow syntax

• on: triggers: push, pull_request, pull_request_target, issues, issue_comment, release, schedule, workflow_dispatch, workflow_call, workflow_run, repository_dispatch, with branches, branches-ignore, tags, paths, paths-ignore, and types filters

  **The push filters work end to end now**, which they did not: `paths:` and `paths-ignore:`
  were parsed, stored, consulted - and handed an empty list of changed files, because the
  dispatcher never read what the push touched. Empty reads as "no information, so run", so both
  filters did nothing at all and a documentation-only push started the whole test suite. One
  `git diff --name-only` per updated ref answers it, with a new branch read as what its own
  commit introduced rather than as every file in the repository, and a push past 5,000 files
  answered as unknown rather than truncated.
  
  The negative forms are stored and consulted too. `branches-ignore` is not `branches`
  inverted - it changes the default, so a workflow with only an ignore list runs everything it
  does not name, and `tags-ignore` counts as naming tags or a workflow written to run on every
  tag but the release ones would never run on any. `paths-ignore` excludes a push only when
  *every* file it changed is ignored: one source file among a hundred documentation changes is
  still a source change.
  
  **`pull_request` starts runs now**, which it did not: the trigger was stored on every version
  and read by nothing, so a workflow that named it never ran - on a forge built around review,
  which is the wrong trigger to be missing. `DispatchPullRequestRuns` listens on `pr:opened`,
  `pr:synchronized` and `pr:ready_for_review`, with Actions' three default activity types when a
  workflow names none, drafts skipped unless the workflow asks for `ready_for_review`, and
  `branches:` filtering on the *base* branch - a workflow saying `branches: [main]` means "when
  something is proposed into main", not "when the contributor's branch is called main".
  
  Two things the fork policy decides, held by tests rather than by care:
  **the definition comes from the base branch** - the dispatcher reads registered versions and
  never parses anything from the head - and **a fork's run is recorded untrusted**, decided by
  the head repository rather than by the branch name or who pushed.
  `pull_request_target` is asked as its own question, so a workflow naming only `pull_request`
  can never be started as the trigger behind the published secret-theft write-ups.
  
  **`schedule` dispatches too**, swept every minute by `DispatchScheduledWorkflowsJob`. A sweep
  rather than a timer armed per workflow: a timer has to survive a restart and a redeploy, and a
  sweep reads what is actually due, so a process that dies between two minutes loses only the
  minutes it was dead. Runs are created on the default branch, the way Actions does it - a cron
  on a feature branch would be a job nobody is watching, from a definition nobody reviewed.
  
  What stops a cron firing twice is a **compare-and-swap on `workflows.last_scheduled_at`**, not
  the run table's unique index: a scheduled run repeats at the same ref and the same commit by
  design, so the index cannot tell a second night from a duplicate. Two sweeps racing means one
  of them updates nothing and dispatches nothing.
  
  A workflow that has never been swept records the clock and waits for the next occurrence
  rather than firing immediately, and a sweep after downtime looks back at most six hours - an
  instance that was off for a week should produce one catch-up run, not seven.
  
  **`issues`, `issue_comment` and `release` dispatch too**, which was wiring rather than
  anything new: this instance has emitted those events since
  [phase 5](./05-notifications.md) and nothing had ever read them for CI. Labelling a new issue
  and publishing on a release are the two things people automate first.
  
  Their filters are `types:` and nothing else - there is no branch on an issue and no path on a
  release. `issues` and `issue_comment` take Actions' defaults (every type, and
  created/edited/deleted). **`release` deliberately defaults to `published` only**, where
  Actions defaults to every type: a draft release starting a deployment is the surprise nobody
  wants, and `published` is what people mean when they write `on: release`. Naming the types
  opts back in.
  
  The subject goes in the run's ref - `refs/heads/main#issues/7/opened` - because the redelivery
  index is on (version, ref, head, event) and every issue event in a repository shares a head
  commit. Without it the second issue would look like the first one redelivered.
  
  **`repository_dispatch` and `workflow_run` dispatch too**, which completes the list. Both had
  been stored on every version since the parser learned to read them, and a workflow whose only
  trigger was one of them looked registered, looked correct, and would never do anything.
  
  `repository_dispatch` is the trigger for something that happened somewhere else: a deployment
  pipeline saying it finished, a package index saying a dependency moved.
  `POST /api/repos/dispatches` takes an `event_type` and a `client_payload` and nothing else -
  not the ref, not the workflow, not which repository the payload claims to be about - which is
  what makes it safe to hand to a program with a narrow token. The payload reaches the job as
  `github.event.client_payload`, capped at 64KB because it is stored on every run it starts. Two
  calls are two runs: the event type and the clock go in the ref, since the redelivery index is
  on (version, ref, head, event) and every one of these shares a head commit.
  
  `workflow_run` is the second half of a pipeline that must not be editable by whoever wrote the
  first half - a fork's pull request can change the build and cannot change what publishes it.
  Two deliberate differences from Actions. **`workflows:` is required**: a workflow that started
  after every other workflow in the repository would start after itself, and the first thing
  anybody would notice is a loop. And **a `workflow_run` run does not start another one** -
  Actions bounds the same loop with a depth limit, and there is no honest use for the second hop
  that `needs:` does not already cover.
  
  Writing them turned up the same omission the subject triggers had: both were added to the
  parser and left out of its "does this `on:` name anything I recognise" list, so every file
  whose only trigger was one of them was refused as naming no event at all.
  
  The rest are recorded as recognised-but-not-dispatched.
  

• jobs: with needs:, outputs: (resolved by the runner and handed to dependent jobs as needs.<job>.outputs.<name>, alongside needs.<job>.result), if: (decided at dispatch now - a job whose condition is false is skipped from the moment the run exists, with the reason on the row, rather than queued and quietly ignored), strategy.matrix including include, exclude, fail-fast, and max-parallel, plus continue-on-error, timeout-minutes, and outputs

  **A matrix is now four jobs in a run rather than one**, which is the half that was missing:
  the expansion existed in the parser and was dropped on the way to the version, so a matrix of
  four produced a single job. Each combination is its own `workflow_jobs` row, named the way
  Actions names them - `test (ubuntu-latest, 20)` - and carrying its own values for a runner to
  inject and a screen to show. They succeed and fail separately, which is the point: a person
  looking at a failed run needs to see *which* combination broke.
  
  **The expansion itself**, in `app/Actions/Workflow/matrix.ts`: the cartesian
  product with the last key varying fastest, `exclude` applied *before* `include` so a workflow
  that excludes a combination and includes it back keeps it, an `include` entry merged into
  every combination it fits without overwriting and appended as its own job when it would, and
  a 256-job ceiling that says what to do rather than starting 400 jobs. Object values compare by
  shape, because `{ node: 20 }` as a matrix value is idiomatic and comparing by identity makes
  every `exclude` miss.
  
  **`needs:` means every combination**, which is where a real defect lived: the graph kept its
  jobs in a map keyed by name, and a matrix puts four rows under one name, so the map held
  whichever combination was written last. A matrix whose first combination failed and whose last
  succeeded unblocked the deploy that the failure existed to stop. It also meant only one row of
  a dependent matrix was ever unblocked, leaving the rest in `blocked` until somebody cancelled
  the run. The graph groups by name now and aggregates, and the same rows carry their ids so the
  write moves the job it decided about rather than the first one with that name.
  
  **`fail-fast` works**, and defaults to true the way Actions does: one combination failing
  cancels the queued siblings and asks the running ones to stop, with the reason on each row -
  a cancelled job with nothing on it reads as "somebody pressed cancel", which is the wrong
  thing to go looking for. `fail-fast: false` leaves them alone, which is the only reason to
  write it.
  
  **`max-parallel` is honoured at claim time**, by counting the combinations already running.
  That is a check rather than a lock, stated plainly here and on the conformance page: two
  runners polling in the same instant can both take the last slot. Making it exact needs a lock
  held across every claim on the instance, which is a cost paid by every job to make one key
  precise.
  
  **`continue-on-error` at job level** does what Actions does and it reads strangely until you
  need it: the job still shows as failed, the run is not failed by it, and the jobs that
  `needs:` it are told `success`. The run page says so on the row, because a red job on a green
  run is otherwise a puzzle. The alternative - treating it as fatal - is what makes people
  delete the flaky suite instead of watching it.
  
  **`timeout-minutes` is enforced twice**, and the two halves fail differently. The runner
  checks between steps and can say *which step* the time went into; the control plane sweeps a
  job that overran whether or not its runner is still listening. Six hours when the workflow
  does not say, which is Actions' default and exists so that nothing runs forever rather than
  as an opinion about how long a job should take.
  
  One thing the sweep needed on the way: it recomputed only the run's own state, so a job it
  force-cancelled left its dependants in `blocked` and the run never reached a terminal state at
  all - a pull request holding on work that ended an hour ago. The graph settler is shared with
  the reporter now, because two things move a run and two copies of "what does this failure
  unblock" is how they end up disagreeing about the same run.
  

• runs-on:, accepting a single label, a list of labels, and a group/labels object, mapped onto queues and runner tags. Complex runs-on expressions are in scope; Gitea's not supporting them is a known migration blocker.

  All three forms parse, with the group flattened onto the labels a runner has to carry because
  a pool is a label here. The object form had been refusing a valid workflow outright - the
  parser read `runs-on` as a string or a list and a mapping came back empty, which reported as
  "does not say what it runs on". It had no test; it has one now. Expressions inside `runs-on`
  are still text, because evaluating them needs the expression engine.
  

• steps: with run, uses, with, env, id, if, name, shell, working-directory, and continue-on-error.

  All of them are read and stored. `shell` is null when the step does not say, which means
  *inherit* rather than bash - see `defaults:` below. `continue-on-error` is only a literal
  `true`: an expression there needs the expression engine, and reading `$` as
  truthy text would make every such step unfailable, which is the dangerous direction to guess
  in.
  
  **`if` is evaluated now**, by the runner rather than at dispatch - a condition reading
  `steps.build.outputs.changed` cannot be answered before the step called `build` has run, so it
  cannot be answered when the run is created at all. `steps.<id>.outputs`, `.outcome`,
  `.conclusion`, `job.status`, `needs` and `always()` all work, and a step whose condition is
  false says so in the log rather than vanishing.
  
  Two consequences worth stating. **A failing step no longer ends the job's steps**: a step with
  `if: always()` or `if: failure()` exists to run after a failure - uploading logs, posting a
  comment, tearing down a deployment - and stopping at the first failure skips exactly the steps
  written for that moment. The job still fails; the file gets to say what happens next.
  
  And **`$` in a `run:` is filled in before the shell sees it**, which is what makes
  `echo "$"` work rather than producing "bad substitution". That is
  also the well-known injection shape - a value spliced into a shell command can end it and
  start another - and Actions has the same property by design: the value comes from this run's
  own steps and event, quoting is the author's job exactly as it is there, and anything
  unresolvable is left as written rather than becoming an empty string that silently changes what
  the command means.
  

• services: on a job, with image and env, and health-checked service startup before the first step. container: is not, and the reason has not changed.

  pantry starts sixty-eight of exactly the things people put in `services:`, so the image name
  is read as *what the workflow meant* rather than as an artifact to fetch: `postgres:16`,
  `postgres:16-alpine` and `docker.io/library/postgres` are all "a Postgres, please". A step
  reaches it on loopback through `$POSTGRES_HOST`, `$POSTGRES_PORT` and `$POSTGRES_URL`, named
  after the workflow's own key.
  
  **An image nothing here can serve fails the job before a step runs**, with the image named
  and the known list printed. That is the decision worth defending: carrying on produces a
  connection refused three minutes later, in a log nobody reads to the bottom, and the person
  debugging it has no reason to suspect the `services:` line at all. A job that genuinely needs
  an arbitrary image needs a runner with a container engine, which this is not.
  
  **Started is not ready**, and the health check is the point: Postgres accepts connections a
  second or two after the process exists, so a first step that connects immediately fails on a
  fast machine and passes on a slow one. A service already running is used rather than
  restarted and is not stopped afterwards - pantry's services belong to the machine, and
  stopping one would take down the database another job on the same runner is mid-query
  against.
  
  The readiness check was verified against a live service on this machine, and the refusal path
  against a port nothing is on. Worth writing down from the same session: `pantry start
  memcached` printed `✓ Started memcached (healthy)` and `pantry status memcached` said
  `failed` a second later, with no log file written - which is the exact disagreement the port
  check exists to survive, and a pantry bug worth chasing separately.
  
  `container:` remains refused at run time with a reason rather than run on the host. Isolation
  is a separate machine, which an autoscaled fleet already gives you one of per job; and the
  common case behind `container: node:20` is a toolchain, which a pantry dependency file
  already answers with no image and no registry.
  

• concurrency: with group and cancel-in-progress, at workflow and job level. Actions has this and Gitea ignores it; the Buildkite concurrency engine in this file implements it properly rather than partially.

  **`cancel-in-progress` works.** A run records the group it belongs to, resolved against its
  own event rather than stored as written, and a new run in the same group moves the ones it
  replaces to `cancelling`. Push twice and the first run stops, which is the whole reason people
  write the key.
  
  `cancelling` rather than `cancelled`: a run already handed to a runner has to be told and has
  to acknowledge ([phase 9](./09-ci.md)), and the control plane does not get to claim an outcome
  it cannot observe.
  
  Two decisions worth keeping:
  
  - **A group whose expression cannot be resolved is no group at all.** Only the closed set of
    context values dispatch actually knows is substituted - `github.workflow`, `github.ref`,
    `github.ref_name`, `github.sha`, `github.head_ref`, `github.base_ref`, the pull request
    number. Anything else (a `||` fallback, `hashFiles`, an input) leaves the template
    unresolved, and an unresolved template would be the same literal string for every run of
    that workflow - grouping runs that should be independent and cancelling somebody's build
    under `cancel-in-progress`. Grouping too little only wastes runners.
  - **A group is not namespaced by event**, matching Actions. `group: $` is
    written precisely so a branch's push run and its pull request run do not both run.
  
  **Job-level `concurrency` works too**, which is the case the workflow level cannot express: a
  workflow whose runs may overlap, with one deployment job inside it that must not. A job's
  group is resolved against its run *and its matrix combination* - `$` is
  available - because a matrix job whose group names none of its values puts every combination
  in one group, and under `cancel-in-progress` they then cancel each other. Actions behaves the
  same way and does not withhold the values, so neither does this; there is a test saying so, so
  that nobody "fixes" it by namespacing silently.
  
  A superseded job moves to `cancelling`, and a sibling job that asked for no group is untouched.
  
  **And the other half now works too**: without `cancel-in-progress`, the second run *waits* for
  the first. A workflow that says `group: production` and nothing else is asking for one deploy
  at a time, and running both anyway is the failure the key was written to prevent.
  
  The held run sits in `waiting` - a state the model already had and nothing was using - with
  the reason on the row, because "queued" with nothing happening for twenty minutes is the most
  expensive screen in a forge and a reader should learn this is the key working rather than a
  runner that is missing. **Holding the run rather than every job** keeps it one state change,
  which means the claim is where it has to be respected: a held run's jobs are ordinary `queued`
  rows, and the claim reads the run's state. There is a test that asks a runner for work and
  watches it decline.
  
  Released by the settler when the run ahead reaches a terminal state, one run at a time and by
  id - push order, not poll order, or a deploy queue would land the older commit last. Releasing
  the whole group at once would turn a serialized queue into a stampede the first time two runs
  piled up behind a slow one.
  

• permissions: on the workflow and per job, mapped onto the fine-grained token permissions from phase 1, defaulting to read-only.

  The file speaks GitHub's vocabulary and this instance has its own, so the names are
  translated rather than adopted: `pull-requests` onto `pull_requests`, `statuses` onto the
  `checks` scope, `repository-hooks` onto `webhooks`. All three shapes are read - the mapping,
  `read-all` / `write-all`, and `{}`, which is a workflow asking for nothing on purpose and is
  not the same as the key being absent.
  
  **The default is read-only and it does not depend on a setting.** Actions' default varies with
  an organization option, which is a footgun this instance declines to reproduce: a workflow
  that says nothing gets a token that can read the repository and no more, on every instance.
  
  Two decisions the tests hold. **A job's block replaces the workflow's rather than adding to
  it** - merging is the friendlier reading and the wrong one, since it hands a job powers its
  author took away. And **neither blanket form grants `administration`**, which is the scope
  that can change branch protection and delete the repository; a workflow that needs it names
  it.
  
  A permission this instance has no scope for - `packages`, `id-token`, `deployments` - is
  **recorded and returned** rather than dropped, because a token that silently grants nothing is
  a workflow that fails at the far end with no explanation. The run detail reports the scopes, the
  level of the file that decided them, and what was refused.
  
  Nothing mints a token yet; that is the execution plane, and by the threat model it happens
  after the fork check - a fork's pull request gets no write-scoped token whatever its workflow
  declares.
  

• defaults: including run.shell and run.working-directory, at workflow and job level, with steps inheriting.

  Three levels, narrowest wins, each key falling through on its own - a step that sets only
  `working-directory` still inherits the shell. The run detail reports what a job's steps
  inherit and from which level.
  
  One difference from `env:` is worth keeping in mind: **nothing declared anywhere means the
  runner decides**, and that is reported as `runner` rather than filled in with `bash`. The
  answer depends on the platform the runner is on, which is knowledge the control plane does not
  have; inventing a default here would also be impossible to tell apart from a workflow that
  asked for it. An empty string is treated as a mistake rather than an answer, so
  `shell: ''` falls through to the level that meant something.
  

• env: at workflow, job, and step level with Actions' precedence order.

  Stored at all three levels rather than merged at parse time, because the precedence is a rule
  a reader has to be able to check and a merged blob cannot say which level a value came from.
  `app/Actions/Workflow/env.ts` resolves it - narrowest wins, names merge and values do not -
  and `explainEnv` answers the question the merge cannot: *why did my step see `staging` when
  the job says `production`*. The run detail returns that per job, with the level in effect and
  the ones it beat.
  
  The cases worth pinning are the ones people get wrong: a name with no value is an empty
  string rather than absent, because a step testing `[ -z "$THING" ]` should see one; an empty
  value at a narrower level still wins, since blanking a variable is a real thing people write;
  and names are compared exactly, so `Path` and `PATH` are two variables.
  
  Step-level `env` is stored and applied when a step runs, so the job's answer stays one answer
  rather than one per step. Secrets are not in this and never will be: they are resolved at
  injection time, after the fork check, and never written to a row.
  

• secrets: on workflow_call, including inherit.

  Declared by the called workflow, passed by the caller, and **recorded rather than resolved**.
  `inherit` stays the word it was written as: expanding it here would decide what a run may
  read before the fork check has happened, and by [the threat
  model](../ci-threat-model.md) that decision belongs at injection - a fork's pull request gets
  no secrets whatever any `secrets:` line says.
  

• workflow_dispatch inputs of every type Actions supports (string, boolean, choice, environment) and the interface form generated from them.

  **The trigger works, the inputs are checked, and the form is generated.** A repository's
  workflows have a screen now (`/{owner}/{repository}/workflows`) which answers the question the
  runs list cannot: *why did nothing happen at all*. A workflow whose file was deleted, one
  somebody disabled, one whose only trigger is an event this instance does not dispatch, and one
  that failed to parse all look identical from the runs list, which shows nothing in every case.
  Each reason is written out in words - including `unsupported_events`, recorded at parse time
  for exactly this and never shown anywhere until now - and the dispatch form is built from the
  inputs the workflow declared, posting to the same public action the API and the CLI use. `POST /api/repos/workflows/dispatch` starts a run from a workflow that names
  `workflow_dispatch`, under a new `workflow:dispatch` ability (write, mapped to `checks:write`)
  - starting a run spends the instance's runners, so seeing a workflow is not permission to run
  it.
  
  All four types are read, in the order written, because a form follows that order. Checking
  them is most of the value: a choice outside its options, a boolean that is not one, a required
  input with nothing to fall back on, and **an input the workflow never declared** are all
  refused with every problem listed at once rather than the first. That last one is refused
  rather than dropped on purpose - silently discarding `enviroment: production` is how somebody
  spends an afternoon wondering why nothing happened.
  
  A default satisfies `required`, matching Actions: `required: true` with a default means "this
  always has a value", not "the caller must always type one". The run records the values it ran
  with, defaults filled in, because "the default applied" is otherwise invisible.
  
  One thing this turned up: the run dedupe index refused a second manual dispatch, having read it
  as a redelivered event. It is partial now (`WHERE event <> 'workflow_dispatch'`) - a manual run
  is not a delivery, and pressing the button twice means two runs.
  
  **The page had never been asked what it renders**, which for stx is the gap that matters: a
  server script that throws renders every variable undefined, so a broken query produces a page
  listing no workflows and reads as a repository with none.
  `tests/e2e/workflows-page.test.ts` asks the rendered HTML instead - the four reasons a
  workflow will not run, a control per input type, no form at all for a reader who may not spend
  the instance's runners, and no em dash anywhere in the body.
  
  Two things it turned up. An `environment` input rendered as a free text field, when the whole
  reason Actions has that type is that typing the name of a protected environment wrong is a
  deploy that silently goes somewhere else; it is a select over the repository's environments
  now, and a plain field with a line saying why when there are none. And the behavioural
  differences line joined a key to its message with an em dash, which is a house rule this
  repository breaks about once a phase - so the test checks for it.
  

• environment: on a job, wired to deployment environments and their protection rules, including required reviewers and wait timers

  The key was parsed, stored, and honoured by nothing - which is worse than refusing it, since
  the workflow says the deploy is protected, the run screen shows an environment, and everybody
  involved believes the opposite.
  
  **The rules live on the repository, never in the file.** A rule a workflow author can edit is
  a rule they can remove on the afternoon they are in a hurry, so naming an environment takes
  push access and configuring one takes `repository:settings`.
  
  Three rules, each a decision a test holds. Required reviewers, where **the person who started
  the run may not approve it even when they are on the list** - a reviewer who can approve
  their own deploy is a rule that reads as two people and behaves as one. A wait timer that
  releases itself, measured from when the job was first held rather than from now (measuring
  from now restarts the clock every sweep, so the wait never ends) or from the run's start (a
  long build would eat the window). And a branch policy that **refuses** rather than holds,
  because a reviewer repeatedly asked to approve deploys from the wrong branch will eventually
  approve one.
  
  An environment the repository has not configured runs normally: `environment: staging` with
  no `staging` is documentation, and refusing it would break far more workflows than it
  protects.
  
  Found while wiring it: the settler's ready-loop hands the *graph* row to each branch, and the
  graph row carries no settings - so the first version read `undefined` and ran every protected
  deploy. `tests/e2e/workflow-environment-gates.test.ts` is what caught it.
  
  Scoped secrets are still not built, and the docs say so: there is no secret store at all yet,
  so "a deploy credential released only after approval" is not something to claim.
  

• Reusable workflows via uses: at job level, local and cross-repository, with inputs, secrets, and outputs, and the called workflow's jobs shown in the run rather than collapsed to one box.

  **Cross-repository calls work now too**, and what they were waiting on was a policy rather
  than plumbing: which repositories may be called. Answered once, in `reusable.ts`, so the
  cross-repository *trigger* can answer it the same way instead of inventing a second rule for
  one boundary.
  
  The same owner by default, needing no configuration to be safe - a repository under one owner
  is already readable by anybody who can read that owner. `workflow_call_scope` widens it to
  any **public** repository here. Another owner's **private** repository is never callable
  whatever the setting says: its jobs would run against a definition nobody outside can read,
  and "I cannot see the file that ran" is a supply-chain problem rather than a convenience.
  
  The refusal names the setting, because an administrator reading a failed run should learn
  which knob decides rather than conclude the feature is broken. An unreadable setting stays
  narrow - a database this cannot read must not widen who may call what.
  
  **Local calls work.** A job that `uses: ./.reviewos/workflows/build.yml` becomes that
  workflow's jobs, copied into the same run and named `build / compile` the way Actions names
  them, so one run still shows everything that happened. The calling job is not a row of its
  own: it has nothing to run. Inputs are checked against what the called workflow declares,
  reusing the `workflow_dispatch` validator rather than growing a second one that would have to
  agree with it forever.
  
  Three refusals, each recorded as a skipped job carrying its reason rather than as silence,
  because a run that quietly misses half its pipeline is the failure people spend an afternoon
  on:
  
  - **A workflow that did not say `workflow_call` cannot be called**, even though its jobs would
    copy in perfectly well. Calling one runs a pipeline its author never offered as an
    interface.
  - **A cycle** is caught by the trail rather than by the depth limit, which would otherwise let
    one go round three times first - and three copies of a pipeline is worse than none, since
    somebody has to work out which was real.
  - **Nesting deeper than four levels** stops there, the same limit Actions has and for the same
    reason a stack has one.
  
  What is left is cross-repository calls, which need a policy about which repositories may be
  called and a way to read a version this instance may not hold. Refused with a clear reason for
  now rather than half done. Outputs are stored as written and are not yet readable by a caller,
  which needs the jobs to have run.
  
  Found while writing the test for this: **the parser refused every reusable-workflow caller**,
  because a calling job has no `runs-on` and the validator required one. Its jobs run on
  whatever the called workflow says, which is the point of calling it.
  

• Composite and JavaScript actions, including composite actions that use other actions. Docker actions are refused, with the reason, because they need a container engine this runner does not have.

  Composite and JavaScript already ran; what was refused was *nesting* - and the code said why:
  the depth limit and cycle check were not written, so following a nested `uses:` would have
  been the version that recurses forever. Both exist now, five deep like Actions, with a cycle
  refused **naming the chain** - a cycle and a runaway depth look identical from outside, a job
  that never finishes, and neither is debuggable from a log that stops.
  
  A nested `uses:` goes through the same path as any other, so the action policy, the cache and
  the input mapping are not implemented twice.
  
  Found by the test: expressions inside a composite action were never evaluated, so
  `with: { who: $ }` reached the nested action as that literal text - it
  greeted somebody called `$`. They are evaluated against the action's own
  inputs now, which is what makes a wrapper able to pass its input down, and is most of what a
  wrapper is for.
  
  Docker stays refused rather than half-built. A container action needs an engine; the common
  case behind reaching for one - "give me node 20" - is answered by a pantry dependency file,
  which is written up in [extensions](../extensions.md).
  

Expressions and contexts

• $ expression evaluation: operators, precedence, and the function set (contains, startsWith, endsWith, format, join, toJSON, fromJSON, hashFiles).

  A lexer, a Pratt parser and an evaluator in `app/Actions/Workflow/expression.ts`. Not a
  regular expression, and never `new Function`: an expression comes out of a file in a
  repository, which on a public instance means it comes from a stranger, and evaluating it with
  the host language's evaluator would hand that stranger this process.
  
  GitHub's semantics are copied deliberately, including the parts that look wrong, because a
  workflow that behaves differently here is one somebody has to debug twice: comparison coerces
  to number so `'' == 0` is true, strings compare without case, `&&` and `||` return operands
  rather than booleans (`inputs.name || 'default'` is the fallback idiom), an unknown property
  is null rather than an error, and an empty object is truthy.
  
  `hashFiles` is **refused rather than answered**: it reads a checked-out tree this side does
  not have, and a fake digest silently restores the wrong cache - a bug people chase for days.
  
  Property reads are own-properties only. `thing.toString` is null, not the host's function;
  the grammar cannot call anything outside the closed set, but a value that leaks a host
  function into a comparison is the first half of an escape.
  

• Status functions success(), always(), cancelled(), failure(), with Actions' rule that an if: without one implies success().

  They read the job's status rather than computing anything, which is why `success()` with
  nothing yet reported is true: it is the default behaviour written out.
  

• Contexts: github, env, vars, job, jobs, steps, runner, secrets, strategy, matrix, needs, inputs. A reviewos context is the canonical name and github is an alias, which is the approach Forgejo took and it works.

  **Nine of the twelve are readable now**: `github` (with `actor`, `workflow`, `head_ref`,
  `base_ref`, `ref_name`, `ref_type`, `run_id`, `run_number`, `run_attempt`, `repository_owner`,
  `server_url`, `api_url`, `event` and `event_path`), plus `env`, `job`, `steps`, `needs`,
  `matrix`, `inputs`, `runner`, and now `vars` - resolved across instance, owner, repository and
  the workflow file at claim time, so the runner merges nothing. `reviewos` is the same object
  under this forge's own name.
  
  `secrets` is populated now too: encrypted at rest, chosen per job at the claim, withheld from a
  fork entirely and from a deploy job until its environment's gate has opened.
  
  **`strategy` and `jobs` complete the twelve.** `strategy` carries `fail-fast`, `max-parallel`,
  and which of a matrix's jobs this one is: `job-index` is counted over the run's rows in
  position order rather than stored, because a matrix of four is four rows under one `job_id` and
  that order *is* the expansion order. `fail-fast` and `max-parallel` had been copied onto every
  run, read by the graph, and readable by nothing a workflow could see - the recurring shape of
  this phase.
  
  `jobs` is the called workflow's own view of itself, and it exists in exactly one place:
  `on.workflow_call.outputs.<name>.value`. The prefix is stripped, because a called workflow
  cannot know it was called `deploy / build` and an expression written against `jobs.build` has
  to keep working when it is; a workflow that workflow called in turn belongs to *its* context
  rather than this one's.
  
  Making `jobs` real found a defect underneath it, and a bad one. **A call job had no row**, so
  `needs: [call]` in the caller named a job that was not in the run: the graph read it as
  missing, the settler swept the dependent as unreachable, and the run went **green having
  skipped the job after the call**. A deploy behind a called build is exactly that shape, and
  nothing about the run said so. A call is a `wait` barrier now - finished when its jobs are,
  never handed to a machine - which is also where the called workflow's declared outputs live, so
  the caller reads them as `needs.<call>.outputs.<name>`. And the call's own `needs:` is grafted
  onto the called workflow's root jobs, because a called workflow was starting immediately
  however much the caller said it should wait.
  
  `tests/e2e/workflow-call-graph.test.ts`. One assertion in `workflow-push.test.ts` had codified
  the defect - "the calling job itself is not a row" - and now says why it is one.
  
  They are built in one function rather than assembled per call site, because a `run:` and a job
  output that interpolate the same expression have to see the same value. Half of `github` was
  being read by the expression evaluator and sent by nothing: `github.workflow` resolved to an
  empty string on every run, which is the kind of defect that looks like a workflow bug.
  

• The expression evaluator is sandboxed and total: no host access, no unbounded evaluation, and a documented failure mode for an expression that cannot be resolved.

  The failure mode is written down and tested in both places it matters. **An `if:` that cannot
  be evaluated does not run the job**, because the other direction deploys somebody's code
  because their condition had a typo; the reason is recorded on the job, since a skipped job is
  the one outcome with nothing else to look at. **An interpolation that cannot be evaluated
  stays as written**, rather than becoming an empty string somebody has to explain.
  

• Tests: an expression suite ported from Actions' own documented examples, including the ones that are surprising

  `tests/unit/workflow-expression-parity.test.ts`, and it is deliberately the *documented*
  examples rather than more cases of our own: a compatibility claim is worth what somebody can
  check, and the cheapest check is to take the expressions out of GitHub's documentation and
  assert what the documentation says they produce. Nobody's workflow breaks on `1 == 1`.
  Workflows break on `''  0` being true, on `'ABC'  'abc'` being true, on `&&` returning an
  operand rather than a boolean, and on `format('')` meaning something.
  
  **It found a real one immediately.** An `if:` that names no status function carries an implied
  `success() &&`, and a step with no `if:` at all is exactly that case - this instance applied
  neither, so every condition was evaluated as though nothing had failed. That was invisible
  while the runner stopped at the first failing step, and became load-bearing the moment it
  stopped stopping: without the rule, a job whose build broke went on to run its deploy step.
  The reason is said on the skipped step rather than left for somebody to work out from the
  file.
  

The runner protocol Actions expects

This is where compatibility is actually won or lost. A workflow file that parses but whose steps cannot talk back to the runner is a workflow that fails on its second line.

• Workflow commands on stdout: ::error::, ::warning::, ::notice:: with file, line, col, and endLine, plus ::group::, ::endgroup::, ::debug::, ::add-mask::, ::add-matcher::, and ::stop-commands::

• ::error file=...,line=...:: becomes a check annotation, which becomes a comment on the diff.

  The whole path is held by a test: a step prints the line, the runner parses it, the annotation
  endpoint records it against a check named for the *job* - "greet failed" is useful, "CI
  failed" is what the reader already knew - and the diff renders it in the gutter. That is the
  reason to implement the format exactly rather than approximately: every linter, compiler and
  test runner people already use has an Actions reporter, and honouring it means those reporters
  work here unchanged.
  
  Read defensively at both ends. A path is a string a step printed rather than a file this
  instance verified, and the count is capped at two hundred in the runner *and* in the endpoint,
  because the runner is the part this instance does not control. Annotations replace rather than
  append, so a retried job does not leave two of everything on one line.
  line. This is the sentence where the Actions ecosystem and this project's whole premise meet,
  and every linter, compiler wrapper and test reporter already emits it.
  

• File-based protocol: GITHUB_OUTPUT, GITHUB_ENV, GITHUB_PATH, GITHUB_STATE, and GITHUB_STEP_SUMMARY, with the multiline delimiter form, under both GITHUB_* and REVIEWOS_* names

• Step summaries render as markdown on the run and, where they belong to a check, on the pull request

  A step summary is the one part of a run written *for a reader* rather than printed for a log:
  the table of what was built, the diff of what changed, the three numbers somebody actually
  wanted. It was being collected, filed on the check, and shown nowhere on the run - a page with
  ten thousand lines of output and not the paragraph the job wrote has the two the wrong way
  round. On the pull request it was printed as text, so a markdown table arrived as literal
  pipes and dashes.
  
  Rendered through the same constructed-HTML renderer as an issue body, because it comes from
  whoever can push to the repository: nothing is sanitized, the HTML is built tag by tag from a
  closed set. Headings get a per-job id prefix, since a summary titled "files" must not be able
  to take the id the page's own tab uses.
  

• The default environment variable set: GITHUB_REPOSITORY, GITHUB_SHA, GITHUB_REF, GITHUB_REF_NAME, GITHUB_HEAD_REF, GITHUB_BASE_REF, GITHUB_WORKSPACE, GITHUB_ACTOR, GITHUB_RUN_ID, GITHUB_RUN_NUMBER, GITHUB_RUN_ATTEMPT, GITHUB_EVENT_NAME, GITHUB_EVENT_PATH, GITHUB_SERVER_URL, GITHUB_API_URL, and the rest, each aliased

  All of them, plus `GITHUB_REF_TYPE`, `GITHUB_REPOSITORY_OWNER`, `GITHUB_TRIGGERING_ACTOR` and
  the `RUNNER_*` set, and **every one is also `REVIEWOS_*`**. Aliased rather than chosen: a
  script that reads one and a script that reads the other are both right, and a forge that only
  answers to somebody else's name cannot be described in its own terms. `RUNNER_*` keeps its
  prefix, because it describes the machine rather than the forge.
  
  Three decisions worth keeping. `GITHUB_SERVER_URL` is **the address the runner actually
  reached**, not a configured one: a configured URL is the one behind the proxy as often as not,
  and every action that builds a link from it would produce a link nobody can follow.
  `GITHUB_BASE_REF` is **empty rather than absent** outside a pull request, because
  `if [ -n "$GITHUB_BASE_REF" ]` is how much of the ecosystem asks "am I on a pull request".
  And `RUNNER_TEMP` and `RUNNER_TOOL_CACHE` are **inside the workspace**, not the host's `/tmp`:
  a step that writes to a shared temp directory can read what the last job left there, and on a
  single-tenant box the last job may have been somebody else's branch.
  
  Deliberately not the whole of the control plane's environment, which holds the database
  credentials. A runner that passed its own environment through would be a way to read every
  repository on the instance.
  

• GITHUB_EVENT_PATH contains an event payload matching the shape of the webhook payloads from phase 5, because half the ecosystem parses it

  Written per job into the workspace's runner directory, so it goes when the workspace goes: a
  payload left in the host's temp directory outlives the job that owned it. The envelope is
  `Webhooks/payloads.ts`'s - `event`, `repository`, `sender`, and one key named after what
  happened - with `ref`, `after` and `pull_request.base.ref` under the names the ecosystem's
  scripts already reach for.
  
  Built from what the run recorded rather than from the repository as it is today, so a re-run
  of an old run sees the commit it was created for. **Nothing in it carries a URL**: this
  instance cannot know its own public address from inside a job, and a payload with a URL that
  does not resolve is worse than one with none - the environment is where a runner is told,
  by whoever configured it.
  
  One thing it cost: the runner directory is created *after* the checkout, because `git clone`
  refuses a directory that already holds anything. Writing the payload first turned every job
  into "destination path '.' already exists".
  

• An automatic per-job token, scoped to the run and the repository, expiring with the job, honouring the permissions: block, and never granted to a fork run by default. This is GITHUB_TOKEN and the ecosystem assumes it exists.

  `permissions:` had been parsed, stored and shown on the run screen since the beginning and
  acted on by nothing - the same defect as `fail-fast` and `timeout-minutes` before it: a key a
  reviewer reads as a control that controls nothing. Now it decides what the token carries.
  
  **Scoped to one repository**, `selection: selected` with exactly one row attached. A job that
  can comment on the repository it is building must not be able to comment on every repository
  its actor can reach, and a token that could is why people are afraid of CI holding
  credentials at all.
  
  **A fork's pull request gets read access whatever its own workflow file declares**, because
  the workflow in a fork's branch is the fork's code. That rule lives in one pure function with
  a test on it, since it is the one most likely to be lost in a refactor.
  
  Revoked when the job reports and expiring within the hour regardless - the expiry is the
  backstop for a runner that dies without reporting, not the mechanism. It travels with the
  secrets rather than beside them, which buys `$` working as written
  and the value being masked by the same pass that masks every stored secret.
  
  Found writing the test: `where('revoked_at', 'is', null)` compiles to a bound parameter in
  this builder and matches nothing, so the revocation silently did nothing. `whereNull` is the
  spelling, and two other files in this codebase already carry that note.
  

• The API endpoints that automatic token is used against by common actions, at GITHUB_API_URL, in the same shapes, so actions/github-script and friends work

  **Three endpoints, deliberately, not the API**: check runs, commit statuses, and comments -
  what CI *writes*. What an action reads it already has, in the event payload and the checkout.
  A layer that half-implements two hundred endpoints is worse than one that implements three
  and says so: the first fails at a random depth inside somebody's action, the second fails at
  the call with a message naming what exists.
  
  Everything else answers 404 **with the list**, because an action told "Not Found" by an API
  it believes in retries, blames the token, and eventually blames the forge.
  
  Each call is this instance's own API with the caller's token, which is the shape the MCP
  surface uses and for the same reason: no second permission check to disagree with the first.
  `permissions:` decides the write, from the ordinary gate.
  
  Two deliberate differences, both documented: `state: error` becomes a failure, since this
  instance has three status states rather than four and inventing one to preserve a distinction
  nothing acts on is compatibility as decoration; and issue and pull request numbering, which
  GitHub shares and this instance shares too because comments live in one table.
  
  Found writing it: the action read the resource from a route parameter that only one of the
  three routes has, so `statuses` and `comments` arrived empty and got the "not implemented"
  answer *from the action that implements them*. The router was matching all along - four
  segments included - which is worth recording, because the first diagnosis was a router
  limitation and it was wrong.
  

• Secret masking in logs, including values registered at runtime with ::add-mask::.

  **Masked in the runner, before anything crosses the wire.** A value masked server-side has
  already been written down by the time it is hidden, which is a masking feature that does not
  mask. The command's own line is dropped rather than logged, since `::add-mask::hunter2`
  contains the secret it is asking to hide.
  
  Longest secret first, so one containing another leaves no fragment behind; values under three
  characters are not masked at all, or every log becomes asterisks.
  

Resolving uses:

• Local actions: uses: ./.reviewos/actions/thing.

  Composite actions run properly - their steps in order, with `with:` arriving as `INPUT_*` the
  way every action written against Actions reads it, and **in the caller's workspace rather than
  the action's own directory**, which reads as wrong until you write one: an action's steps
  operate on the repository that called them, and `GITHUB_ACTION_PATH` is how it reaches its own
  files. That covers most of what an action is in practice, since repositories' own actions are
  nearly all composite.
  
  JavaScript actions run their `main:` with Bun, and the log says so rather than pretending to
  be node - an action that depends on something Bun does not implement fails in a way its author
  can read. A path that climbs out of the workspace is not a reference at all: on a host runner,
  the directory above the checkout is the rest of the machine.
  
  Nesting - a composite step that itself `uses:` something - is refused with a message rather
  than followed, because doing it without the depth limit and cycle check the
  reusable-workflow path already has is the version that recurses forever.
  

• Container actions: uses: docker://registry/image:tag

  The argv is built by a pure function and the execution is three lines, which is the right way
  round: a container run is a long command line where every mistake is silent. A missing `--rm`
  leaks containers until a build machine has no disk; a mount at the wrong path makes an action
  see an empty workspace and do nothing; an environment variable still carrying a host path
  sends a tool at a directory that does not exist inside, so the action writes its outputs to
  nowhere and the job goes green with empty results. All of those are testable without a
  runtime, and the test asserts the command line rather than the outcome.
  
  The workspace mounts at `/github/workspace`, every path under it is rewritten to what the
  container will see, and every variable is passed as `--env KEY=value` rather than `--env KEY` -
  the second form reads the value out of the *runner's* environment, which would be a way for a
  workflow to ask for a variable this process holds and the job was never given. `args` is split
  the way a shell would split it and passed as an argv, so a quote in a workflow file is not a
  place to inject a command. `--security-opt no-new-privileges`, always.
  
  Off by default, twice: the instance's action policy has to allow containers, and the machine
  has to have docker or podman. A runner with neither says so by name rather than failing on a
  command not found, because a machine without a container runtime is an ordinary machine.
  
  **An action whose `image:` is a `Dockerfile` is refused**, by name. Building it would make the
  runner a build host for arbitrary images from the repository whose workflow it is running - a
  bigger decision than "may containers run here", with its own cache, its own disk budget and
  its own supply chain.
  
  Found on the way, and fixed here: a `uses:` step was never given the files a `run:` step gets,
  so `GITHUB_OUTPUT` was absent from every action's environment and **no action could set an
  output at all**. Nothing failed - the action wrote to a path that was not there, and the step
  after it read an empty value, which reads as "that action is broken on this forge".
  

• Remote actions by owner and name with a configurable default host, plus fully qualified URLs.

  An origins map decides where a host's repositories actually live, so `actions.example` can
  point at an internal mirror, at this instance, or at a directory on disk. That is the same
  mechanism the mirroring box below wants rather than a second one - and it is what lets the
  test suite exercise real fetching over `file://` on a machine with no network.
  
  A reference naming no host is refused with a reason rather than resolved against github.com.
  That guess is the one the policy layer already declines to make, and making it here instead
  would have moved the decision somewhere nobody would look for it.
  

• Ref resolution by tag, branch, and commit sha, with sha pinning enforceable by policy.

  The reference forms are read and the policy decides: **the default is closed** - local actions
  always, and nothing from anywhere else until an operator names a host. An unqualified
  `actions/checkout@v4` means nothing without a configured default host, and says so rather than
  guessing at github.com.
  
  Only a full forty-character sha counts as pinned. A short sha is ambiguous by construction and
  a seven-character prefix can be brute-forced onto a different object, so accepting one would
  make `requirePinnedSha` a setting that reads as protection and is not.
  
  **Fetching works now.** A remote reference is fetched with a shallow fetch of the one object
  the reference names - which is the only form that handles a tag, a branch and a sha without
  guessing which it was - and checked out into a cache keyed by the resolved commit.
  

• An action cache on the instance, so a fleet of runners does not each fetch the same action, and so an instance can keep working when the upstream host does not.

  **Keyed by the resolved commit rather than by the reference**, so `@v4` and the sha behind it
  are one entry - which is what makes the second job's fetch free rather than a fetch. A pinned
  reference is answered from the cache without touching the network at all, because a sha *is*
  the identity and a directory named after one cannot be stale; a tag is resolved every time,
  because it moves, and only the resulting commit is reused.
  
  Two jobs fetching the same commit at once is a race with no loser: the second finds the first
  one's directory already there and drops its own copy, since same sha means same bytes.
  
  **The instance-side cache exists now too**, and it is the same thing as mirroring: the
  instance keeps bare mirrors under `storage/actions/{host}/{owner}/{name}.git` and serves them
  over the ordinary git protocol, so a runner points its origins map at
  `https://instance/actions/github.com` and everything else about fetching stays exactly as it
  was. Ten runners then fetch from here instead of from the internet.
  
  Read-only and unauthenticated on purpose: what is here is public code mirrored from a public
  host, carrying no repository's contents and no user's data, and requiring a credential would
  mean every runner in a fleet holding one to fetch things anybody can download.
  `git-receive-pack` is not served at all - a mirror that could be pushed to is a supply chain
  with a hole in it, and the whole value of mirroring `actions/checkout` here is that what this
  serves is what upstream had.
  

• Mirroring of the actions a repository actually uses into the instance (phase 13 already mirrors repositories), so an air-gapped install is a supported configuration.

  **What is mirrored is what is used**, read out of the workflow versions this instance has
  already parsed rather than from a list somebody maintains: a list drifts the moment a workflow
  changes, and the failure of a stale one is a build that breaks because the single action
  nobody added is the one it needed. Only the newest version of each active workflow counts, and
  only references the policy would actually allow - mirroring an action nobody may run is a
  network request with no possible use.
  
  Swept hourly. `git remote update --prune` on an unchanged repository transfers nothing, and a
  tag deleted upstream disappears here rather than being served forever; the reason to run it
  when everything is fine is that the day it matters is the day the upstream is down, and a
  mirror last updated a week ago is missing exactly the tag somebody pushed yesterday.
  
  The test deletes the upstream before fetching through the instance, because a cache that only
  works while the thing it caches is reachable is not a cache.
  

• An allowlist policy at instance and owner level over which action sources may be used, since uses: is arbitrary code selection by anyone who can edit a workflow file

• Tests: every resolution form, a pinned sha that does not match, an action outside the allowlist, an unreachable upstream with a warm cache, and a local action outside the repository

  All five, against real git over `file://` rather than against a mock, which is the only way
  the resolution cases mean anything - tag, branch, commit sha and subdirectory each resolve
  differently and the differences are the whole feature. A branch is the form people reach for
  without noticing (`@main` means "whatever is there today") and it had no test at all.
  
  The warm-cache case removes the upstream rather than misconfiguring it, because a cache that
  only works while the thing it caches is reachable is not a cache. Its other half is that a
  **tag fails honestly** with the upstream gone: a tag moves, so answering one from the cache
  would mean serving whatever it pointed at last time and calling it current. A pinned sha is
  different in kind - a sha *is* the identity, so a directory named after one cannot be stale.
  

First run

Actions users do not provision anything. They push a file and it runs. That expectation does not survive contact with a self-hosted forge unless we make it.

• A single documented command brings up a runner and registers it with the instance

  `./buddy runner:local`, and that is now the whole of it. It used to be two: register, copy the
  credential out of the output, start with `--token`. The second step was friction with no
  safety behind it - the same operator, at the same shell, on the instance's own machine - and
  the argument that actually matters is unchanged: **this instance runs nothing until somebody
  types this**, and typing it is that. The credential is kept in
  `storage/framework/runtime/runner-local.token` at mode `0600`, because a file that can claim
  any job on the instance is not a world-readable one.
  
  `--register` stays, for the case it is really for: a runner on a *second* machine, where the
  credential has to be carried over. Re-registering rotates rather than duplicating, so a leak
  is fixed by one command. Documented in [self-hosting](../self-hosting.md#running-ci), with the
  no-isolation tradeoff stated where somebody deciding will read it.
  

• A default queue exists on a new instance, so runs-on: ubuntu-latest resolves to something without configuration

  `ubuntu-latest` is in the local runner's default labels, alongside `self-hosted` and `local`.
  It is what every workflow copied from Actions says, and a new instance where the first
  `runs-on:` anybody writes matches nothing is an instance where CI appears to be broken rather
  than unconfigured. Naming it is a claim about what the label *means* here - "the machine the
  instance is on" - which is the honest reading for a single-tenant install and is printed on
  every start rather than assumed.
  
  The other half is the empty state. A job that can never be taken already said why; it now also
  says **what to do about it**, and only to somebody who could - the ability the cancel control
  asks for. A shell command in front of every visitor is noise; a paragraph that stops one word
  short of useful is worse.
  

• Optionally, the instance ships with one local runner enabled for single-tenant installs, off by default for multi-tenant ones, with the tradeoff stated plainly rather than buried.

  `./buddy runner:local` - **a command rather than a setting**, which is the whole safety
  argument: this instance does not execute repository code unless an operator has said where,
  and typing it is that. `--register` makes the credential once; re-running it rotates the
  credential rather than making a second runner, so a leak is fixed by one command.
  
  The tradeoff is printed on every start, not buried here: **no isolation**. A step runs as the
  user who started the runner, on the host the control plane is on, with that user's files and
  network. Right for one team on one box running code they wrote; wrong for anything else.
  
  **A fork's pull request is refused by the runner itself**, not by a flag somebody can set at
  three in the morning: untrusted code on the control plane's own host is the one combination
  that turns CI into somebody else's shell. It is reported as failed rather than dropped, so the
  run still reaches a terminal state instead of holding a pull request's checks open.
  
  It speaks the ordinary HTTP protocol - claim, logs, report - rather than reaching into the
  database, so the lease, job-token and late-report rules are exercised rather than bypassed.
  Two things it does that a remote runner cannot: it clones from the bare repository on disk
  (`--no-hardlinks`, so a step running `git gc` cannot write into the instance's own objects),
  and **it checks out before the first step**. Actions leaves that to `actions/checkout`, which
  needs action resolution this phase has not built - without the checkout every copied workflow
  would run in an empty directory and read as a broken product rather than an incomplete one.
  
  A step's environment is the documented default set, deliberately *not* this process's
  environment: the control plane's own variables include the database credentials, and handing
  those to a repository's script would make the runner a way to read every repository on the
  instance. `uses:` steps are skipped with a line saying so, since resolving an action is the
  next box rather than this one.
  

• The interface says clearly when a run is queued because no runner matches, and which labels would have matched, instead of a spinner.

  A run sitting at "queued" with a spinner is the most expensive screen in a forge: it looks
  like the instance is thinking, so people wait, then wait longer, then ask in a chat channel -
  and the instance knew the answer the whole time. Five answers, and each sends somebody
  somewhere different:
  
  - no runners are registered at all;
  - none of them reaches this repository, which is a scope problem rather than a label one;
  - none has the labels this job asked for - and the message names **what the runners that
    could take it do have**, which is the half that tells somebody what to write instead;
  - every runner that matches is disabled, which means "turn that one back on" rather than
    "change your `runs-on`";
  - a runner matches and will take it on its next poll, which is a real answer rather than an
    absence.
  
  Computed from the same rules the claim protocol uses to hand work out, because a screen that
  explains a decision the dispatcher did not make is worse than a spinner. On the run page and
  in the run detail, and read only when something is actually waiting - a finished run has
  nothing to explain.
  

• A repository with no workflows offers starter templates that are real Actions workflows.

  Six of them, on the workflows page's empty state, ordered by the languages this instance has
  already measured - and nothing hidden, because a suggestion that removes options is one that
  is wrong at somebody's expense.
  
  Every starter is parsed *and dispatch-checked* by a test: it is not enough that a template
  reads well, it has to be a file this instance would actually run, and one whose triggers
  nothing dispatches would sit there saying "runs on nothing". They are deliberately small and
  carry no placeholder to fill in - a starter that needs editing before it works is an editing
  task, an editing task is a decision, and a decision is where people stop.
  

Where the compatible forges stop, and we do not

Gitea and Forgejo both chose Actions compatibility and both proved it works. The places they fall short are documented, and they are precisely the Buildkite capabilities in the rest of this file, which is the whole argument for this phase existing:

• Each row above has a test proving the difference, because a comparison table in marketing that no test defends becomes false without anybody noticing

  `tests/unit/comparison-claims.test.ts`. Every row is either a **live claim**, checked by
  exercising the capability rather than importing the module - a concurrency group actually
  resolved, a tag selector actually matched and actually refused, an annotation landing on the
  row a reviewer is reading - or a **pending claim** naming the roadmap box that would make it
  true.
  
  **It ratchets both ways.** A new row with no claim fails; so does a pending claim whose box
  gets ticked, because at that point the promise should be replaced by a check.
  
  Writing it did the job immediately: two rows were not true. `environment:` was parsed and
  wired to nothing, and signed dispatch has no key for a pool to trust. Both were marked
  *(planned)*, which is the correction the test existed to force - and the first of them was
  then built, so the ratchet fired and the promise was replaced by a live check.
  

The engine: the step model underneath

Everything above describes what an author writes. This describes what the control plane can do once it has parsed it. The Actions surface normalizes into this model, and the Buildkite capabilities here are reachable from additive keys on Actions syntax rather than from a second language.

Step kinds

Actions has one step kind and expresses the rest through job structure. Buildkite has six, and the extra five are genuinely useful, so the engine carries all of them. In Actions syntax the last four appear as job-level keys rather than as new step types, which is the pattern for everything in this phase: familiar surface, larger engine.

• Command step. One or more shell commands, or a uses: action. The only kind that consumes a runner, and the only one an Actions workflow writes directly.

  The default, and the only kind the claim will hand to a machine - which is a rule rather than
  an optimisation. The other three are the control plane's own work, and a runner deciding a
  deployment approval would not be a scheduling mistake, it would be the gate not existing.
  

• Wait step. A barrier. Everything before it must finish before anything after it starts, with a variant that continues on failure. needs: covers most of this; an explicit barrier covers the rest.

  `reviewos: { wait: true }`, **normalized into `needs:` at parse time** so the graph a reader
  sees is the graph that runs: the barrier needs every job declared before it, and every job
  after it that named no dependencies of its own needs the barrier. A job with an explicit
  `needs:` keeps it, because that is a statement about the graph and a barrier must not quietly
  widen it.
  
  `continue-on-failure: true` is the variant, and it is the graph-level twin of `if: always()`:
  the dependencies still have to be *finished*, since the point of a barrier is that everything
  before it is over - only their verdict stops mattering. It reaches the graph as
  `allow_dependency_failure`, which is the attribute two sections down.
  

• Block step. Pauses the run until a human unblocks it. The approval gate, and the primitive phase 9's "waiting steps" already describes. Reached from Actions syntax through environment: protection rules.

  `reviewos: { block: 'Deploy to production?' }`. The job sits in **`paused`**, which is its own
  job state rather than `blocked`: `blocked` means "waiting for another job" and the graph
  resolves it on its own, where nothing resolves this but somebody deciding. A screen that
  cannot tell the two apart cannot show the button, which is the whole difference between a gate
  and a hang. The run reads `waiting` rather than `running`, because nothing is running.
  
  **Its own ability, `workflow:approve`.** Not `workflow:cancel`: stopping a build is safe and
  approving a release is not, and folding them together would mean anybody who can stop a run
  can also ship one. Who opened it is recorded on the job, because "who approved this
  deployment" is asked while looking at the run.
  

• Input step. Pauses and collects typed fields (text, select, boolean) from the person unblocking it, which become available to later steps.

  **Folded into the block step rather than given a row of its own**, because a block with fields
  *is* an input step and two names for one row is how a model grows a spelling problem. The
  typed values become the job's **outputs**, so a later job reads
  `needs.approve.outputs.version` exactly as it reads any other job's output - inventing a
  second mechanism for "values a person typed" would be a second thing to learn for a value that
  behaves identically.
  
  `string`, `boolean` and `select`, and a `select` declares its options. A value outside them is
  refused **with the options listed**: the entire reason to declare them is that somebody can be
  told which ones there are rather than reading "invalid input".
  

• Trigger step. Starts a run of another workflow, in this repository or another, passing commit, branch, environment, and metadata. Async by default, awaitable on request. Actions reaches this through workflow_call and repository_dispatch.

  In this repository, for now: a cross-repository trigger is the same policy question as a
  cross-repository `uses:`, and answering it here would answer it in the wrong place. Async by
  default, which is Buildkite's default and the right one - a trigger that waits turns one stuck
  run into two - with `await: true` keeping the job running until the run it started finishes
  and **carrying that run's verdict back**, since a trigger that waited and then reported
  success whatever happened would be a gate that is not one.
  
  Three rules that are not optional. A triggered run is **trusted because the run that triggered
  it was**, so a fork's pull request cannot trigger a trusted one: a trigger cannot raise its own
  trust level. It carries a **depth with a ceiling of five**, because a workflow that triggers a
  workflow that triggers the first one is a run factory and nothing else in the model would
  notice - every trigger makes a *new* run, so there is no row to catch the loop. And a trigger
  that cannot resolve **fails, with the reason on the job**: a pipeline whose deploy stage
  silently did nothing, and a green run to go with it, is the failure this phase exists to
  avoid.
  

• Group step. Nests steps under one label so a run with two hundred jobs reads as eight things. Groups carry their own dependency edges and rollup state.

  **A label rather than a container**, which is a deliberate narrowing of what the line asks for.
  Nesting jobs inside jobs would change every query that reads a run, and it would let a screen
  re-sort the jobs - and a run page that disagrees with the order in the file is a page nobody
  can check the file against. The heading prints once, over the jobs that share it, in declared
  order. Dependency edges stay between jobs, where `needs:` already puts them.
  

Step attributes

Every one of these has to survive normalization into rows (phase 9: no workflow-sized JSON blob) and has to be expressible in the validator that runs before a definition ever reaches a runner. The names below are the internal model's; the Actions key that maps onto each is noted where it is not obvious.

• key, label, and a stable identity that survives re-uploads within a run

  Actions already has both: the job id is the key and `name:` is the label, so adopting a
  second vocabulary would mean two names for one thing in every message, screen and error.
  
  The identity half is real work and it is done in `Workflow/upload.ts`: a generated job whose
  key is already in the run is **refused with the name**, rather than merged or renamed.
  `needs:` is by name and two jobs sharing one is how a matrix is expressed, so silently adding
  a second `build` would change what every existing `needs: build` waits for - a graph nobody
  wrote.
  

• depends_on, accepting several keys, plus allow_dependency_failure so a step can run after a failure on purpose

  `needs:` *is* `depends_on` and takes a list, so the first half was Actions compatibility
  already. The second was reachable only from a `wait:` barrier, which meant the
  "publish the results whatever happened" stage had to be written as a barrier even when it was
  an ordinary job.
  
  `reviewos: { allow-dependency-failure: true }` now says it on any job, and reaches the graph
  as the **same flag** the barrier sets rather than as a second mechanism - one rule with two
  spellings is one that disagrees with itself a year later.
  
  It is the graph-level twin of `if: always()`, and the distinction is kept: the dependencies
  still have to be *finished*, only their verdict stops mattering. A job that needed a failed
  one and did not ask is skipped, as before.
  

• if, a conditional expression over a documented, sandboxed variable set: branch, tag, commit message, trigger source, changed paths, prior step outcomes, and declared inputs. No arbitrary reads of control-plane state.

  The evaluator was already sandboxed; what was missing was half the set. **`ref_type` and the
  commit message** were the two people reach for on the first day and did not have:
  `if: github.ref_type == 'tag'` is how a release job is written, and
  `contains(github.event.head_commit.message, '[skip ci]')` is the filter `on:` cannot express
  per job. Declared inputs and the changed paths join them.
  
  The message is the **whole** message rather than the subject, because half the people who
  write `[skip ci]` put it on the second line - and a condition that only saw the subject would
  be right most of the time, which is the worst kind of wrong.
  
  `reviewos.changed` sits under this instance's own name rather than inside `github`: a
  workflow reading it would not run on GitHub, and a reader deserves to see that in the
  expression rather than discover it on migration.
  
  Prior step outcomes stay unavailable at dispatch on purpose - nothing has run - and the
  evaluator refuses rather than guessing, so the job is skipped with the reason recorded. The
  test asserts the *shape* of the context as well as its contents, because a sandbox is only a
  sandbox if something checks its edges.
  

• branches, including negation, as the shorthand for the most common if

  `reviewos: { branches: [main, '!wip/*'] }`, decided when the run is created. It exists
  because `if: github.ref == 'refs/heads/main'` is the expression everybody writes and a good
  share get wrong - `refs/heads/` is easy to forget, and the failure is silent, since a
  condition nobody matches is a job that simply never runs with no error anywhere.
  
  **An exclusion beats an inclusion**, which is what every other tool carrying both does and
  the safer reading: `['*', '!release/*']` means *not release*, and the other way round runs a
  deploy on exactly the branch it was told to avoid. A list of exclusions alone means
  everywhere else.
  

• skip, boolean or a reason string that shows on the run

  And `skip: true` still gets a sentence, because the reason is the whole difference between a
  skipped job somebody can decide about in three weeks and a commented-out block nobody reads.
  
  Applied at dispatch with `branches`, so a job that will never run is `skipped` from the first
  second rather than sitting in the queue looking like work nobody has got to - which is how
  somebody ends up investigating a runner that is behaving perfectly.
  

• soft_fail, boolean or a list of exit statuses that report failure without failing the run

  `reviewos: { soft-fail: [1] }`, decided **at the report** rather than at dispatch, because it
  keys on an exit status that does not exist until the job has failed.
  
  The list is what earns the attribute its place over `continue-on-error`, which is
  all-or-nothing: a linter exiting 1 on findings is a soft failure, and the same linter exiting
  127 because it is not installed is a broken pipeline wearing a green tick. A failure with no
  exit status at all - a lost runner, a timeout - is **not** tolerated by a status list, the
  same rule `retry` follows.
  
  The job still records `failed` and the graph is told not to count it. Rewriting it as a
  success would be the version where nobody finds out the linter has been failing for a
  month.
  

• retry, automatic and manual. Automatic retries key on exit status, a lost runner, and a timeout, with limits and constant, linear, or exponential backoff. Manual retry can be permitted, forbidden, or forbidden with a reason.

  **Automatic, on exit status and on a lost runner.** `reviewos: { retry: 2 }` is two extra
  attempts; `retry: { attempts: 2, exit-status: [137] }` narrows it to the failures worth
  repeating, because a suite that exits 1 on a failed assertion is not one and a step killed for
  memory at 137 is. A named-status retry does *not* fire on an unknown status: retrying on "we
  do not know why it failed" is how a narrow retry becomes a blanket one.
  
  **The cap is required and there is a ceiling of five**, refused with the reason rather than
  clamped: a job that fails five times in a row is not flaky, it is broken, and the retries are
  spending machines to postpone the moment somebody looks at it. Every attempt shows on the run,
  because a job that quietly ran twice is a flaky test nobody ever fixes.
  
  **This found an unbounded loop.** The lease sweep requeued *every* job whose runner stopped
  responding, with nothing counting - so a job that kills the machine it runs on, out of memory
  or out of disk, was handed to every runner in the fleet in turn, for as long as the fleet
  existed. Recovery is right; recovery without a limit is one job taking down a fleet one
  machine at a time. Three attempts now, and then a failure that says three different runners
  went quiet on the same work rather than reading as an ordinary one.
  
  Backoff and manual-retry policy are not implemented: a retry that waits needs a scheduled
  wake-up rather than a state change, and the manual half is a re-run button that does not
  exist yet.
  

• timeout_in_minutes, per step, with a workflow default and an instance ceiling

  `timeout-minutes:` on a step, which is Actions' own key and was parsed by nothing until now:
  the runner applied its own two-hour ceiling to every step and the workflow's number went
  nowhere. The narrow one is the useful one - a job allowed sixty minutes that hangs in a
  thirty-second health check spends fifty-nine of them proving nothing - and a step stopped this
  way **says so**, because a killed process exits with a signal and otherwise reads as an
  ordinary failure in a command that was working fine.
  
  The ceiling stays underneath as the default, so nothing hangs forever whatever the file says.
  

• priority, so a deploy jumps a queue full of pull request checks

  `reviewos: { priority: 10 }`, read by the claim on every poll - which is why it is a column
  rather than a value in the settings blob. Equal-priority jobs stay first in, first out,
  because a queue that reorders equal work is one where somebody's build can starve.
  
  **It orders a queue, it does not preempt.** A job already running is not stopped for a more
  important one: that would mean killing work somebody is waiting on to start work somebody else
  is waiting on, which needs a policy rather than a number.
  

• parallelism, expanding one step into N identical jobs that differ only by index and total

  `reviewos: { parallelism: 5 }`, five rows in the run named `test (1/5)` through `test (5/5)`,
  each told which it is through `REVIEWOS_PARALLEL_JOB` and `REVIEWOS_PARALLEL_JOB_COUNT`.
  
  **The index counts from zero and the name counts from one**, deliberately: the number exists
  to be handed to `/api/repos/tests/split`, which indexes from zero, and a person reading a
  failed run is not indexing an array. Buildkite makes the same split. Written down in both
  places because it is the one thing to get wrong here.
  
  A hundred copies is the ceiling, and a file asking for more is refused rather than clamped: a
  suite quietly running twenty of the two hundred shards it asked for reports a tenth of itself
  as green. A barrier, a gate or a trigger cannot have copies - five copies of a gate is five
  approvals for one deploy.
  

• matrix, expanding across named dimensions, with adjustments that add a single combination, skip one, or soft-fail one, because the useful matrix is never the full cross product

  The matrix itself is Actions' `strategy.matrix`, `include` and `exclude` included, with the
  expansion order that decides how a run reads. `reviewos: { adjustments: [...] }` is the
  additive half.
  
  **Soft-failing one combination is the part Actions cannot express**: `continue-on-error` is
  per job, so tolerating the nightly Node version means tolerating every version, and a matrix
  that tolerates everything cannot fail a build. `skip` is not a duplicate of `exclude` either -
  an excluded combination never existed and cannot explain itself, where a skipped one is a row
  carrying its reason.
  
  Last match wins, so a broad adjustment followed by a narrow one reads the way it looks. An
  entry with no `with:` is ignored rather than applied to every combination, which is the
  failure that would quietly tolerate a whole matrix.
  

• concurrency and concurrency_group, a named limit shared across runs and workflows, which is how a shared staging environment or a deploy lock gets serialized

  `reviewos: { concurrency-group: production }`, enforced at the claim: at most N jobs wearing
  that name run at once, across every run and every workflow in the repository. A deploy that
  two workflows can start cannot be serialised by a run-level group - the two runs are not in
  the same group and never will be - and `max-parallel` keys on the job's own name, so it cannot
  say that a smoke test and a deploy share one environment.
  
  Repository-wide rather than instance-wide: a `production` group in one repository is not the
  same lock as another team's. Counted rather than locked, the same trade `max-parallel` makes -
  two runners polling in the same instant can both take the last slot, and making it exact costs
  a lock on every claim on the instance.
  

• concurrency_method: ordered (a FIFO queue) or eager (whoever is ready). The difference is whether a deploy queue preserves commit order.

  **`ordered` is the default**, because that difference is the reason to reach for the feature:
  whichever-is-ready makes the state of production depend on runner timing. `eager` is there for
  a group that is a resource limit rather than a sequence - four jobs sharing one licence
  server. Ordered goes by job id, which is dispatch order, which is push order.
  

• agents, a key=value tag query selecting which runners may take the job

  Shipped with the fleet work: `reviewos: { agents: { gpu: a100 } }`, matched against the tags a
  machine reported at registration. `runs-on:` is a set membership test, which is right for
  `ubuntu-latest` and wrong for anything with a value in it - a fleet with four GPU models ends
  up with labels called `gpu-a100`, and a label means whatever the person who typed it was
  thinking.
  

• env, per step, over a workflow-level env, over runner environment

  Actions' own key, with Actions' precedence - workflow, then job, then step, narrowest wins -
  resolved in one place so the runner and the screen that explains where a value came from read
  the same answer. The runner's own environment is the base underneath all three.
  

• secrets, naming secrets to inject rather than embedding them

  `reviewos: { secrets: [DEPLOY_KEY] }`, narrowing what a job receives to what it asked for. A
  trusted job used to receive every secret in scope, which is Actions' behaviour and is fine
  until a test job's dependency is compromised and reads a deploy key that job never needed.
  
  **Saying nothing is not saying none.** No key means everything in scope, which is backwards
  compatibility rather than advice; `secrets: []` is a job that has decided, and is worth being
  able to say about a job that runs somebody else's code.
  
  Resolved at the claim rather than at dispatch, deliberately and unlike the line above: the
  claim is the last moment both facts are known - whether the run is trusted, and whether the
  environment's gate has opened - and deciding earlier would mean handing a deploy credential to
  a job that is still waiting for a reviewer. Naming a secret is not a way around either rule.
  

• artifact_paths, globs uploaded automatically when the step ends, pass or fail

  `reviewos: { artifact-paths: [screenshots/**] }`, collected by the runner after the steps and
  before the conclusion. **Pass or fail is the feature**: the step-based alternative has to be
  written `if: always()`, and the run where somebody forgot is always the run with the
  screenshot in it.
  
  Paths stay inside the checkout, checked twice - when the file is parsed, and again on each
  match, because a symlink the build created can point anywhere and a job that can publish
  `/etc/` can put the machine's secrets on a page. A glob that matches nothing says so in the
  log rather than failing a job that had already finished.
  

• plugins, the extension point, with its own section below

  `reviewos: { plugins: [acme/docker-login#v1.2.0: { registry: ghcr.io }] }`. Parsed here for
  shape only: whether the plugin exists, what its parameters are, and whether this instance
  permits it are answered at dispatch, where the repository and the policy can be read.
  

• cancel_on_build_failing, so long jobs stop when a sibling has already sunk the run

  `reviewos: { cancel-on-build-failing: true }` - the forty-minute browser suite still going
  when the unit tests have gone red. Nobody will read its result and the machine it holds is one
  nothing else can use.
  
  **Off unless the job asks**, which is the opposite of `fail-fast` and deliberately so: a job
  that publishes results, tears down a preview or reports the failure exists *because*
  something failed, and a run-wide default would stop exactly the jobs people lean on hardest
  on the day a build breaks. A failure the workflow tolerates does not count, and a running job
  is asked to stop rather than declared stopped.
  

• checkout options: submodules, clone depth, LFS, sparse paths, clean behavior, and skipping checkout entirely

  `reviewos: { checkout: { depth: 1, sparse: [packages/api] } }`, and `checkout: false` for a job
  that needs no code. The commands are built as data in `Runner/checkout.ts` rather than
  assembled inside the executor, because a checkout is a shell command built from user input and
  quoting tested by running builds is quoting nobody tested.
  
  **A depth on this instance's own machine clones through `file://`**: git ignores `--depth` on
  a local-path clone, hardlinks the whole object store, and prints a warning most people never
  read - so a workflow that asked for a shallow clone would silently get ten years of history.
  
  No `clean`, and that is a property of this runner rather than an omission: every job gets a
  workspace of its own. The parser says so in the error rather than leaving somebody to guess.
  

• if_changed, path-glob gating evaluated against the run's diff. The monorepository primitive, and the one that decides whether a big repository is usable here at all.

  `reviewos: { if-changed: packages/api/** }` on a job. Actions filters the *whole workflow* on
  `on.push.paths`, which is the wrong grain for a repository with twelve packages in it: the
  workflow has to run, and what it runs is the question.
  
  Decided at dispatch, like `if:`, so a job that is not going to run is `skipped` **with the
  reason on the row** from the moment the run exists rather than queued and quietly ignored. The
  reason names the globs, because the value of skipping a job in a monorepository is being able
  to see why without opening the file.
  
  **Unknown paths mean the job runs.** The changed list is empty when the instance could not work
  out what moved - a force push, a first push, a rewrite past the ceiling - and the two failures
  are not equal: a job that runs when it need not have costs a few machine-minutes, and a job
  skipped when it should have run is a broken commit nobody noticed.
  
  A barrier or a gate cannot carry it, and that is refused rather than allowed: a barrier that
  only sometimes exists changes the shape of the graph, and a deployment gate that vanishes
  because no file matched is a gate that approves itself.
  

• notify, per step, distinct from workflow-level notification

  `reviewos: { notify: [{ user: alex, if: failure }] }` on a job. The case watching a repository
  cannot cover: a nightly run with forty green jobs and one red deploy is a notification nobody
  reads unless it names the job.
  
  **People on this instance, never an address.** A workflow file is editable by anybody who can
  push, so a `notify:` that took an email address would make every repository here a mail relay
  with a spam problem. Naming a user means their own preferences decide the channel, which is
  also the answer to "how do I get a text message": that is a per-person setting rather than a
  workflow one.
  
  Delivered from the settler rather than from the report endpoint, because a job reaches a
  terminal state by four routes - a runner reporting, a lease lapsing, `fail-fast`, a
  cancellation - and a notification wired to one of them silently does not happen for the other
  three. `notified_at` is claimed with a guarded write, so running on every pass costs a query
  rather than a duplicate.
  
  Two refusals hold it: somebody who cannot read the repository is told nothing, and **a fork's
  run notifies nobody** - a stranger who can open a pull request should not be able to make this
  instance message a maintainer on demand. A cancelled job counts as a failure, because the
  deploy did not happen either. `tests/e2e/workflow-notify.test.ts`.
  

• Tests: every attribute above round-trips definition to normalized rows to execution, and the validator rejects each one's malformed forms with a file location and a fix

  `tests/e2e/workflow-attribute-round-trip.test.ts`: one table entry per `reviewos:` key,
  checking the file -> definition row -> run's job row, and the malformed forms with a line and
  a fix on each error. The keys a machine acts on are checked at the claim in the suites beside
  the features themselves.
  
  **The table is held against the parser's own key list**, so adding a key without saying where
  it is stored and what reads it breaks this test. That is the failure this phase keeps
  producing - `fail-fast`, `timeout-minutes` and `permissions:` each shipped parsed and read by
  nothing - and it was found by accident every time until now.
  
  It earned its place immediately: an error about a key written inline (`checkout: { clean: true }`)
  carried line 0, because the line search looks for a key at the start of a line. An editor
  cannot jump to line 0 and a reader reads it as "somewhere in this file"; it now falls back to
  the job's own line.
  

Which of those an Actions author already has a word for, so the engine does not grow a second spelling for a thing people can already say:

• Every additive key above is documented as an extension, in one place, with what happens when a workflow using it is taken back to GitHub. An extension nobody can find, or that silently breaks portability, is worse than not having it.

  [`docs/extensions.md`](../extensions.md), and there is exactly one key to document because
  **everything additive lives under `reviewos:` on a job**. An extension spread across five new
  top-level keys is five things to find when a workflow moves; this is one to delete and one
  word to grep for when somebody asks what in a repository is not portable.
  
  What happens on GitHub is stated first rather than in a footnote: **the file is refused**,
  since GitHub does not accept a job key it does not know. That is the right failure. A key
  GitHub silently ignored would mean a `block:` gate that is simply not there on the other
  side - a deployment approval that approves itself.
  

Dynamic definitions

Buildkite's most important feature and its largest security surface. A job can generate steps and upload them into the run it is already part of, so a workflow can decide what to do after looking at the repository. Actions has only a shadow of this: a matrix built from fromJSON of a prior job's output, which covers the common case and nothing else.

• A runner-side upload command that appends steps to the current run, validated by the control plane before any of them become eligible

  `reviewos-upload generated.yml`, on the PATH of every step, put there by the runner. One line,
  because the alternative is every generating job carrying a curl invocation with a job
  credential in it - and a credential in a repository's own script is a credential in its own
  log. The credential lives in the script, outside the checkout, so a step that prints its
  environment or tars its workspace does not carry it away.
  
  Validated by **the same parser a workflow file goes through**, with the run's existing job
  names handed to it so a generated job may depend on the one that generated it. A second,
  laxer validator for uploaded steps would be the one an attacker reads.
  

• Uploaded steps are attributed to the job that uploaded them, and the run records the full resulting graph rather than only what was declared at the start

  `uploaded_by_job_id` and `upload_depth` on the row. A run's graph is what it *became*: a
  screen showing only the original file would be describing a run nobody had.
  

• An uploaded step cannot raise its own trust level: it cannot grant itself secrets, target a queue the parent could not, or turn a fork run into a trusted one

  Structural rather than checked: everything that decides what a job may reach is inherited from
  the run and the uploading job, and **there is no field in the document for any of it**. A
  fork's run stays untrusted, the repository is the same repository, and the pool serving it is
  the pool that already served it. Priority is inherited too, which is the one field a generated
  job could otherwise use to jump a queue full of other people's work.
  
  Secrets are not in this list because they are not implemented anywhere yet; when they are, the
  rule is already written down in the threat model and this is the shape it has to take.
  

• An upload budget: maximum steps, maximum depth, maximum total uploads per run, so a loop is bounded by the control plane rather than by a quota nobody set

  Four limits, because each one alone is a loop somebody can still write: three uploads deep,
  twenty uploads per run, fifty jobs per upload, five hundred jobs per run - plus a 200KB ceiling
  on the document itself, which bounds how much text one request can make the parser look at.
  Reaching a limit stops the *next* upload rather than unwinding the last one.
  

• Signature verification. When signed workflows are enforced (below), an uploaded step must be signed by a key the runner pool trusts, or refused.

  Generated work is signed at the claim exactly like written work, and a pool with
  `require_signed_steps` refuses it the same way - the signature is over the steps as the runner
  receives them, whatever table they came out of. No separate path, which is the point: a second
  rule for generated steps is the one an attacker reads.
  
  Writing this found the defect underneath it. **An uploaded job reached a runner with no
  steps at all.** Its rows were created, the graph was right, the settler released it, and the
  claim read `workflow_version_steps` - which has nothing for a job that was in no workflow
  file. So it ran nothing and reported success. A generated job now carries its own commands on
  its step rows, and the claim reads those for exactly those jobs.
  

• Tests: uploading a step that targets a forbidden queue, an upload loop, an upload from a fork, an unsigned upload under enforcement, and an upload after the run reached a terminal state

  All five now. The first four in `tests/e2e/workflow-uploads.test.ts`, plus the ones writing it
  turned up: a name the run already has, a `needs:` naming nothing, a document the parser
  refuses, and a priority a generated job tried to give itself. The fifth is in
  `tests/e2e/workflow-signed-steps.test.ts`, where a generated job is claimed and its signature
  checked against the published key - which is also the test that would have caught the empty
  step list.
  
  The forbidden-queue case is covered by construction rather than by a check: an uploaded job
  cannot name a pool at all, and the claim already refuses a repository a pool does not serve.
  The end-to-end test runs the real runner, whose step writes YAML and calls `reviewos-upload`,
  and then claims the generated job with a second poll.
  

Definition management

• Workflow templates, owner-managed, so an organization can require a starting point. This is phase 9's "owner-managed reusable workflows" from the governance side rather than the reuse side.

  A reusable workflow is called by a repository that decided to call it; a template is what an
  organization puts in front of every repository that has not decided anything yet - which is
  where CI conventions are actually set, and where "copy it from the last repository" is how
  they drift.
  
  Two refusals carry the feature. **Publishing validates**, because a template is copied into
  repositories by people who did not write it, and one that fails on their first push is a
  support ticket from somebody with no way to know where the file came from. And **applying
  never overwrites** without being told to: a template that silently replaced a repository's
  own workflow is governance deleting the exception somebody made on purpose.
  
  Applying writes a **real commit** through the same plumbing a web edit uses, and registers
  the workflow itself - this commit is written by the instance, so no push hook fires, and a
  template that landed as text nothing had read would not exist until somebody happened to push
  again.
  
  Found doing it: `app/Actions/Workflow/templates.ts` already existed and holds the starter
  workflows for an empty repository. The owner-managed ones are `ownerTemplates.ts`, which is
  the honest split - a starter is what this instance suggests, and a template is what an owner
  requires.
  

• Schedules in cron syntax, per workflow, each with its own branch, commit, message, and environment, plus enable and disable without deleting

  The schedules were built earlier; **enable and disable were not, and that is the part worth
  writing down.** `disabled` has been a state on the workflow row since the beginning, and
  every dispatch path already refused to run one - the push dispatcher, the schedule sweep and
  the manual dispatch all check it. Nothing could ever *set* it, so the check was dead code and
  the state was a lie: a reader of the model would conclude a workflow could be turned off, and
  no path existed. The same shape as `fail-fast`, `timeout-minutes` and `permissions:` before
  it, from the other direction.
  
  Off rather than deleted, because deleting is a commit, a review and a revert for something
  that is usually temporary - a nightly job failing while an upstream service is down, a deploy
  paused during a freeze. `removed` stays the third state and keeps meaning the file is gone
  from the branch, so enabling one of those is refused with what actually has to happen instead.
  
  It takes `workflow:dispatch` rather than repository administration: the person who needs to
  stop a workflow failing at three in the morning is the person on call, not the owner.
  

• Skip intermediate runs and cancel intermediate runs, per workflow: when three commits land in a minute on the same branch, do not run all three

  `reviewos: { intermediate: skip | cancel | run }` at the top level - the only extension that is
  not on a job, because it is a statement about the *workflow's* runs.
  
  `cancel` is `concurrency.cancel-in-progress` said in one word. **`skip` is the third thing
  neither Actions nor Gitea offers**, and usually the one people mean: let the build that has
  already started finish, and drop the ones that have not. The run in progress will produce a
  result somebody reads; the queued ones would produce two nobody does.
  
  A skipped run is `cancelled` outright rather than `cancelling`, because nothing had taken it -
  there is no machine to tell and no acknowledgement to wait for, which is the whole difference
  from cancelling in progress. Its jobs go with it, rather than sitting queued for a runner to
  take work from a run nobody will read.
  
  One thing writing it turned up: the value was being validated *after* the parser's
  `errors.length > 0` gate, so `intermediate: maybe` was silently read as `run`. A validator
  whose complaints are discarded is worse than none - it reads like a check.
  

• Merge queue support: a run against the prospective merge result rather than the branch tip, so a queue of pull requests is tested in the order it will land

  The engine and its rules, not the screen. An entry is tested on the base with everything
  ahead already merged, built with `merge-tree` and written to `refs/merge-queue/<number>` so a
  runner can check it out - and the base branch does not move while it is tested.
  
  **Landing is a ref move, not a second merge.** Merging again at that moment produces a
  different commit from the one the run went green on - same tree, different parents - and the
  thing that was tested would never exist. That is the whole failure this feature is built to
  prevent, so it moves the branch to exactly the tested commit, guarded by where the branch
  was; a branch that moved underneath puts the entry back in the queue rather than forcing it.
  
  **A failure ejects and re-queues everything behind it**, because those entries were tested on
  top of a commit that is now never going to exist. Not re-testing them is precisely how a
  merge queue lands the change that breaks main. `ejected` rather than `failed`: the pull
  request did not fail, it did not land this time in this order.
  
  Found doing it: `performMerge` had no dry run, so the first version would have moved `main`
  to the prospective commit before anything tested it - the exact bug the feature exists to
  prevent, in the feature that prevents it. `dryRun` is now a real option rather than something
  smuggled past the type with `as any`.
  
  Deliberately not done: parallel speculative testing (double the machines to save latency,
  before anybody has asked), the stall policy, and the page.
  

• A workflow can live at a path other than the repository root, and more than one can live in one repository

  `.github/workflows/*.yml` and `.reviewos/workflows/*.yml`, any number of them, each its own
  workflow with its own runs and its own schedule. `.reviewos/` wins outright when present -
  not merged, because two directories quietly contributing to one list is how somebody ends up
  running a file they thought they had replaced.
  

• Environment variables and settings at instance, owner, repository, and workflow level, with a documented precedence order and a screen that shows where a value came from

  Narrowest wins - the workflow file's `env:`, then the repository, then the owner, then the
  instance - and that half is what everybody expects. **The screen is the feature.** Four
  places can set `REGISTRY`, so a value can be wrong at a level nobody is looking at, and "it
  is us-east-1" is not the answer somebody needs at that point: the listing says which level
  answered and what it overrode, widest last.
  
  Setting a value something narrower already overrides says so in the answer, because
  otherwise it looks like it worked and the question comes back three days later.
  
  The resolution runs once, at claim time, and the page reads the same function: a precedence
  rule implemented twice is a precedence rule that disagrees with itself about why a value is
  what it is.
  
  An instance variable needs an administrator and an owner variable needs the owner -
  administering one repository is not permission to change every repository an organization
  has. And they are **variables, not secrets**: readable by anybody who can read the
  repository, in the logs, handed to a fork's job. There is no secret store yet and the docs
  say so rather than approximating one with a `secret: true` column on a plain-text table.
  

• Tests: schedule fires once per window, intermediate cancellation leaves exactly one run, a template change does not retroactively alter a finished run

  The first two were already covered - `tests/e2e/workflow-schedule.test.ts` races two sweeps
  and gets one run, and holds the window open at both ends; `tests/e2e/workflow-push.test.ts`
  lands two commits under `intermediate: skip` and finds one cancelled outright with the reason
  and one queued.
  
  The third is `tests/e2e/workflow-run-immutability.test.ts`, and it is the property the whole
  copy-on-dispatch design exists for: a run keeps the jobs, the commands and the limits the file
  had when it started, a **new** push runs the rewritten file - immutability that never updates
  is a cache nobody invalidates - and a re-run re-runs what ran rather than what the file says
  now. Reading the newest version is always the shorter query, which is why this needs a test
  rather than a convention.
  

The runner fleet

Buildkite's agent is the part of it that is open source, and the part people trust it for. Phase 9 defines the protocol; this is the fleet management around it.

The protocol has been run as a fleet, on machines that are not the instance's, which is worth recording because everything below is easy to design against a runner that only ever runs next to the control plane. ./buddy build:runner --target linux-x64 compiles the same executor runner:local uses into one file with no runtime to install - it compiles at all because nothing under app/Actions/Runner/ touches the framework or the database, and a runner that needed a database connection is one you could only run on the instance's own box.

What a real run showed, with two runners on a Linux x86-64 machine in another country claiming from a macOS arm64 control plane:

• A three-way matrix spread across both runners, with max-parallel: 2 holding the third combination back until a slot opened.
• The checkout over HTTP, since there is no storage/repos on a fleet machine. A same-host runner still clones from disk; which one happens is a fact about where the runner is rather than a setting.
• RUNNER_OS=Linux and RUNNER_ARCH=X64 on the runner while the control plane was macOS/ARM64, which is the environment set being about the machine that runs the job rather than about the instance.
• A wait barrier resolving with no runner at all (runner_id null), and the job after it running only once the barrier had.
• Job outputs read back from $GITHUB_OUTPUT across the wire, and a step-level if: gated on one.

The credential is made on the instance and carried over, because a fleet machine has no database to register itself in - which is exactly what a registration token is for.

The compiled entry point now only runs when it is the program. standalone.ts executed on import, and everything that walks app/Actions/ imports every file it finds - so a CLI command that happened to load it printed the runner's usage and exited zero. buddy docs:reference did nothing, successfully, and the generated pages went stale with no error to explain why. import.meta.main guards it.

• Runner pools: a named group of queues plus the workflows permitted to use them. A workflow in one pool cannot dispatch to, read artifacts from, or trigger a workflow in another unless a rule says so.

  **A pool serves the repositories it lists, and every repository when it lists none.** The empty
  list is what every existing install has, so nobody is quietly given a boundary they did not
  ask for; adding one repository is the act of drawing it. The refusal names the pool but *not*
  its other repositories - on a shared instance that list is the map of who is working on what.
  
  The narrowing from the line above: permission is per *repository* rather than per workflow, and
  it governs which machines take the work rather than artifacts and triggers. A workflow-level
  rule needs a name for a workflow that survives being renamed, and artifact and trigger scoping
  is a second boundary with its own failure modes - both are worth doing after somebody has run
  the first one.
  

• Queues within a pool, named for infrastructure rather than for teams, with pause and resume so an operator can drain one without deleting it

  Draining is the operation the row exists for. Every other way to take machines out of service
  loses something: deleting runners loses their identity and their history, disabling them one
  at a time is a list somebody has to keep, and turning them off leaves jobs waiting on a
  machine that is not coming back. Pausing says "no new work here", lets what is running finish,
  and is undone by one call - **the jobs stay queued**, which is the difference between a drain
  and an outage. The reason travels to the run page, because the person who returns to a stuck
  queue is usually not the person who drained it.
  

• Registration tokens scoped to one pool, rotatable, revocable, with a first-use and last-use record. Registration credentials never enter a job environment (phase 9 rule).

  **The credential a fleet machine should actually carry.** Before this, an autoscaler needed an
  *administrator's* token to create runners - which puts the widest credential on the instance
  into a userdata blob on every machine it starts. A registration token can do one thing, add a
  machine to one pool, and that is the whole blast radius when a blob leaks.
  
  `POST /api/runner/register` **exchanges it**: registering mints a per-runner credential and the
  machine uses that from then on, the same shape as the job token one layer down. That is how the
  phase 9 rule is kept rather than promised - the thing running jobs is holding something else by
  then, and a test asserts the registration credential is refused at the claim.
  
  Revoked rather than deleted, because "which token did that machine register with" outlives the
  token and is asked at exactly the moment a machine did something surprising. Revoking stops
  *new* machines joining and does not interrupt a build already running on one that joined.
  First use and last use are recorded because they answer the two questions asked about a
  credential nobody remembers making: has this ever been used, and is it still being used.
  

• Runner tags, set at registration and by a startup hook, queried by a step's agents selector. Unmatched selectors leave a job queued with a visible reason rather than silently forever.

  `reviewos: { agents: [gpu=a100] }`, matched against `key=value` tags a machine reports about
  itself at registration. Labels are a set membership test, which is the right shape for
  `ubuntu-latest` and the wrong one for anything with a value in it: a fleet with four GPU models
  grows labels called `gpu-a100`, and a label means whatever the person who typed it was
  thinking.
  
  A selector that is not `key=value` is **refused rather than read as a label** - one that
  silently became a label would match a different set of machines than the file says, and a job
  running somewhere it should not is invisible. A machine that reported no tags satisfies no
  selector, which is the safe direction.
  
  An impossible selector is told apart from a label mismatch on the run page, because the two
  look identical from outside and have opposite remedies: a label is changed in the workflow, and
  a tag is set on the machine by whatever knows it has a GPU.
  

• Runner lifecycle visible in the interface: connecting, idle, accepted, running, stopping, stopped, lost, with the job it is on and the time in state

  Six states, derived rather than stored: `never-seen`, `idle`, `running`, `stopping`, `lost`,
  `disabled`. A status column has to be written by whoever causes the change, and **the one
  change nobody causes - a machine going quiet - is the one that matters**, so the lifecycle is
  the lease, the last poll and the stop somebody asked for, read together.
  
  `lost` outranks `stopping` and `running` on purpose: a machine asked to stop that then goes
  quiet has stopped without saying so, and a machine holding a job whose lease has lapsed is
  exactly what the reclaim sweep is about. `never-seen` is the first-run confusion made visible -
  a credential somebody made and a command nobody started.
  

• Ephemeral runners: disconnect after one job, or after an idle timeout, which is what makes an autoscaling group safe

  `--jobs 1` and `--idle-timeout <seconds>`, both on the runner itself. That placement is the
  point: **the runner knows whether it is mid-job** where a scaler outside it has to guess, and
  guessing wrong means killing a machine in the middle of somebody's build. The idle clock runs
  from the last *job* rather than the last poll, so a runner that has been busy does not shut
  down because the queue emptied for one cycle.
  
  Verified on a real machine: a runner started with `--idle-timeout 6` against an empty queue
  exited on its own.
  

• Graceful stop that lets the current job finish, and a forced stop that does not, both from the API

  Both on `/api/instance/fleet`. They differ in one thing: what happens to the job the machine
  is holding. Graceful takes no new work and lets it finish; **forced puts the job back in the
  queue rather than cancelling it** - the work is fine, it is the machine that is going away,
  and somebody watching a pull request should not see their build fail because an autoscaler
  shrank the fleet. It counts as an attempt, so a machine force-stopped repeatedly cannot hand
  one job round a fleet forever.
  
  The machine is told **when it next asks for work**, because that is the only moment this
  instance can tell it anything: a runner is somebody else's machine, possibly behind a
  firewall, and there is no connection to send a signal down. The request is cleared when it is
  acknowledged, or a machine an operator brought back would stop again immediately.
  

• A metrics endpoint reporting queue depth, waiting jobs per queue, and runner counts by state, in a shape an autoscaler can poll. This is the whole interface an autoscaler needs.

  On the existing `/api/metrics`, in Prometheus exposition format, because that is what every
  scraper and every autoscaler already reads - a JSON shape of our own would be a format each
  operator has to write an exporter for. `reviewos_ci_jobs_waiting`, `_jobs_running`,
  `_jobs_oldest_waiting_seconds` and `reviewos_ci_runners{lifecycle}`, all per queue.
  
  **Every series is emitted at zero**, which is the detail that decides whether the contract
  works: a gauge that disappears when it reaches zero is how a scaler concludes there is no work
  when what happened is that nobody reported any. `unassigned` is a real queue name, carrying
  the machines nobody put in a queue and the jobs whose `runs-on:` matches no runner anywhere -
  on an instance that has started using pools, that bucket is where the surprises are.
  

• Reference autoscaler for at least one substrate, plus documentation of the polling contract for the ones we do not write

  [`docs/autoscaling.md`](../autoscaling.md): the contract, what a scaler has to do itself, and a
  hundred-line shell script against Hetzner Cloud that is deliberately boring.
  
  **The interesting part is that it has no scale-down path.** The runner exits on its own when
  the queue has been empty for five minutes and the machine shuts itself off, so nothing outside
  has to answer "is it mid-job" - which is the question a scaler cannot answer and the reason
  autoscaled CI kills builds. `stop-runner` exists for the cases the runner cannot know about: a
  spot instance being reclaimed, a queue drained for maintenance.
  
  Machine preparation is **pantry, not a container image**: `pantry install git node@22` on a
  general-purpose machine, and a machine that needs a different version tomorrow installs it
  rather than being rebuilt. There is no Dockerfile anywhere in that document, which is the
  point.
  
  The binary comes from the instance itself - `GET /api/runner/download?target=linux-x64`,
  public and uncredentialed because the file holds no secret and does nothing until it is given
  a URL and a token. That makes the version question answer itself: the binary a machine fetches
  is the one built for the instance it is about to talk to.
  

• Pool maintainers: a role that can manage queues, tokens, and workflow assignment without being an instance administrator

  The role exists because of what happens without it: the person who looks after the build
  machines is made an instance administrator - draining a queue needs it - and now they can read
  every private repository on the instance.
  
  Per pool rather than a global "fleet operator", because a fleet with two pools usually has them
  because two groups own different machines, and a role spanning both puts each group's
  credentials within reach of the other. Two verbs stay administrator-only: creating a pool is
  creating a boundary, and appointing maintainers is handing out the power to manage one - a role
  that can appoint itself sideways is not a narrower role at all.
  
  A maintainer acting on a pool they do not maintain gets the same 404 a stranger gets: the
  existence of somebody else's pool is not theirs to learn.
  

• Tests: a job with an impossible selector, a runner lost mid-job, a drained queue, a revoked token mid-job, and a runner claiming work from a pool it is not registered to

  All five, against the real claim rather than against the rules in isolation - a boundary the
  dispatcher does not enforce is documentation. The runner-lost case is in
  `runner-reclaim.test.ts`, where it also proves the attempt cap: three machines going quiet on
  one job stops it being handed out rather than passing it round the fleet forever.
  

Runner hooks

Buildkite's hook set is the extension point that makes the agent adaptable without a plugin, and the list is worth copying wholesale because each entry exists to solve a problem people actually have.

• Fleet lifecycle: runner-startup, runner-shutdown

  Either side of the poll loop, in the runner's own directory with no job and no workspace -
  because there is neither. A failure changes nothing but the log: there is no job to fail, and
  a machine that refuses to take work because a warmup script exited 1 is one an operator has to
  notice before anything happens at all.
  

• Job lifecycle, in order: pre-bootstrap, environment, pre-checkout, checkout, post-checkout, pre-command, command, post-command, pre-artifact, post-artifact, pre-exit

  All eleven, wired into the local runner. **A job hook that fails fails the job**, which is the
  point: a fleet that must inject a proxy or refuse untrusted work is one where the hook not
  working means the job must not run either. `post-command`, `post-artifact` and `pre-exit` run
  whatever happened, so a teardown still runs after a failed build - and the conclusion is read
  after them, so a `pre-exit` that cannot put back what it set up fails a job whose steps
  passed.
  
  `$REVIEWOS_ENV` carries values from a hook into the hooks after it and into the steps, the
  same channel `GITHUB_ENV` gives a step.
  

• Three scopes with a documented precedence: runner hooks (on the machine, outside repository control), repository hooks (in the checkout), and plugin hooks

  Two of the three: the machine's, from `--hooks`, and the repository's, from `.reviewos/hooks/`
  in the checkout. Both run for an ordinary stage, the machine's first, so the second reads what
  the first exported. **Plugin hooks are not built** - there are no plugins yet - and they slot
  between the two when there are.
  
  A file without an execute bit is not a hook: a `README` in a hooks directory would otherwise
  be run as a shell script and its failure reported as the job's. A fork's pull request gets no
  repository hooks at all, which is a second line behind this runner refusing untrusted runs -
  and it is there so the day the first is relaxed for a sandboxed runner, this is not relaxed
  with it.
  

• pre-bootstrap can refuse a job before any repository code is fetched. This is how an operator keeps a trusted runner from running an arbitrary workflow, and it must be runner-scoped only.

  A non-zero exit fails the job with the reason, before the checkout runs - the e2e asserts that
  the log has no checkout group in it at all. Runner-scoped, because a repository hook here
  would be the code deciding whether to trust itself.
  

• checkout and command are overridable, so a fleet can substitute its own clone strategy or execution wrapper

  Both replace the built-in behaviour rather than adding to it, and both are runner-scoped for
  the same reason: a repository that could replace the command would not be running its own
  steps any more, and a fleet's profiler wrapper would be removed by the first repository that
  did not want it.
  
  Writing this found a real bug: a hook that writes into the workspace before the checkout - the
  whole point of `environment` - broke the built-in clone, because `git clone` refuses a
  directory with anything in it. The checkout now uses the fetch shape when the workspace is not
  empty.
  

• Tests: a refusing pre-bootstrap, a repository hook attempting to override a runner hook, hook failure at each stage, and the environment a hook can and cannot see

  The precedence is a pure function over a directory listing, so the rule that matters - a
  repository hook is never consulted for the three deciding stages - is tested directly rather
  than inferred from a run. The e2e runs real hooks against a real checkout: a refusal, an
  exported variable a step reads, a `command` hook that replaces the steps, and a `pre-exit`
  failure that fails a job whose steps passed.
  

Plugins

Actions is the primary extension mechanism. uses: is what an author reaches for, it is what the ecosystem is made of, and nothing here replaces it. A plugin is the second mechanism, for the thing an action structurally cannot do: hook into the job around the command, before checkout or after artifact upload, on every step in a pool without being written into each workflow. Buildkite's plugins and Actions' actions are not competitors; they sit at different points in the job lifecycle.

• The distinction above is documented on one page with a decision rule, or every workflow author will pick by coin flip and half of them will be wrong

  [Plugins](../plugins.md), and the rule is one line: **a plugin wraps a job, an action runs as
  a step**. With the tiebreak that matters when somebody is unsure - write the action, because
  a plugin is the answer to "the workflow file is the wrong place for this".
  

• A plugin is a versioned, self-contained repository providing hooks and a declared parameter schema, referenced by a step or attached to a pool

  `plugin.yml` plus a `hooks/` directory, in a repository here or vendored in the one using it.
  Referenced per **job** rather than per step, which is the honest mapping: a Buildkite command
  step is a job here, and the hooks are job-scoped because that is where a lifecycle is.
  Attached to a pool with `attach-plugin`, which runs it on every job that pool takes -
  including the ones already queued, and no repository can remove it.
  
  Only the stages the manifest names are read, so a file appearing in `hooks/` cannot quietly
  become a hook - and a plugin cannot take part in `pre-bootstrap`, `checkout` or `command`
  whoever attached it, because those three are the machine's alone.
  

• Parameters are validated against the plugin's schema before dispatch, not by the plugin at runtime

  Types, `enum`, `required`, and defaults filled in - and an **unknown parameter is an error**,
  which is the rule worth defending: the failure this catches is a typo, and a typo silently
  ignored is a plugin running with its default while somebody reads the line they wrote and
  believes it took effect.
  

• Pinning by commit or tag, and an instance policy that can require pinning

  A tag counts, and the plugin is recorded as the commit that tag pointed at when the run was
  created - so a tag moved afterwards does not change what a re-run executes. A tag and a branch
  are the same string in a workflow file, so the difference is decided by resolving the ref
  rather than by reading it.
  

• An allowlist policy at instance, owner, or pool level, because an unrestricted plugin reference is arbitrary code selection by whoever can edit a workflow file

  `plugin_policies`, one row per subject, and **each level only narrows**: allowlists intersect,
  capabilities intersect, and a pinning requirement anywhere applies. A level that could widen
  what the level above allowed would make the level above decorative, which is the failure mode
  of every allowlist that merges by union.
  

• Vendored plugins: a plugin resolved from the repository itself rather than fetched

  `./.reviewos/plugins/<name>`, read out of the commit the run is for. Pinned by construction:
  there is no version to write down because it travels with the code that uses it.
  

• A plugin can be marked as requiring elevated capability (docker socket, host network), and a pool can refuse those

  `requires:` in the manifest, granted per pool and refused by default. Checked at the **claim**
  rather than at dispatch, and that is the only place it can be: a capability is a statement
  about a machine, and dispatch does not know which machine will take the job. A refusal fails
  the job with the reason rather than leaving it queued looking like work nobody has got to.
  

• Documented authoring path, a local test harness, and a small first-party set that covers the cases every fleet needs

  The authoring path and the harness are on [the page](../plugins.md), and the harness is
  honestly small: a hook is a program that reads environment and exits non-zero, so
  `REVIEWOS_PLUGIN_X_Y=... ./hooks/pre-command` is the whole of it. **No first-party set yet**,
  which is the part of this box that is a promise rather than a fact - there is no registry
  either, and a plugin is found the way any repository is.
  

• Tests: an unpinned plugin under a pinning policy, a schema violation, a plugin outside the allowlist, and a plugin hook attempting an escalation the pool forbids

  All four, against real bare repositories: `tests/e2e/workflow-plugins.test.ts` builds a plugin
  repository with a tag and a vendored plugin that declares a capability, then dispatches
  through the real path and claims through the real endpoint. The rules themselves are pure and
  tested at their edges in `tests/unit/plugins.test.ts` - a policy that could only be exercised
  by dispatching a run is one nobody would test the edges of.
  

What a run looks like

The screens. Buildkite's advantage here is a decade of small decisions, and most of the list is small decisions.

• Log output streamed live, with collapsible groups the job itself opens and closes, per-line timestamps, ANSI colour, links, and images

  **Images, last of the six.** An `image` event names an artifact of its own run - there is no
  URL field and there will not be one. A URL would let a build put a picture served from
  anywhere onto a page a colleague opens, which is a request their browser makes to somebody
  else's server every time the log is read: a tracking pixel a build can install in a page other
  people look at.
  
  What renders in place is decided by the bytes rather than by the `Content-Type` of the upload,
  which is whatever the machine typed. PNG, JPEG, GIF and WEBP, at most 8MB, served with
  `nosniff` and a `Content-Security-Policy` of `default-src 'none'; sandbox` so a wrong sniff has
  nowhere to go. **SVG is refused**: it is the one image format that is a document, and rendering
  it in place means running it. Everything else stays downloadable from the run's artifacts and
  reads in the log as a line saying so - which is also what an expired artifact shows, because
  the log outlives the bytes and a broken image with no explanation is the worst way to say it.
  
  The rest, from before: groups, timestamps, colour and links are done, on top of structured
  log events: an append may carry `line`, `group` and `endgroup` events instead of bytes, and
  the four things text cannot carry stop being guesses. `::group::` is a marker one CI product
  uses and a string somebody's build may legitimately print; the time a chunk arrived is not the
  time a line was printed, since a runner batching a hundred lines has them all land in one
  millisecond; a chunk carries one stream where a job interleaves two; and escape bytes shown as
  text are noise nobody can turn off.
  
  A group is a `<details>`, so folding works with no script - the run screen carries almost
  none. The last group of a *failed* job is open, because a job that groups its output is
  usually grouping the parts nobody reads and the exception is the one the failure is in; that
  is decided from the job's state rather than by searching the output for the word "error".
  
  Colour is classes rather than inline styles, so a theme decides what red is on the background
  the reader actually has. Links are `http` and `https` only, `rel="noreferrer nofollow
  noopener"`, with trailing punctuation left out of the href - a link that 404s because it
  swallowed a full stop teaches people not to click them.
  
  Text is not deprecated: a runner that sends what its build printed still works, renders through
  the same path, and gets the text stored beside any events rather than having to send both.
  
  Images are the one left. They need a way to reference bytes a job produced - which is now
  possible, artifacts exist - plus a content policy for rendering them, and an image a build can
  put on a page somebody else loads is a decision rather than a feature.
  
  **The streaming half, from before:** The run screen follows a job from the
  sequence the page was rendered at - `resources/functions/runlive.ts` - so a job that has
  already printed two megabytes costs nothing to follow and a reader never sees a line twice.
  New output is appended after what the server rendered rather than replacing it, because a
  reader mid-line in the failure of job three should not be moved; the run's own state is
  reported in a line at the top, and a run that finishes offers a reload rather than taking one.
  
  It stops when the run does. The log endpoint reports the job's state beside its chunks for
  exactly that: a job quiet for a minute in the middle of a step and one quiet forever look
  identical from the output alone, so a follower without it either gives up early or polls a
  finished job until the tab closes. A hidden tab backs off to a minute rather than stopping,
  since coming back to a stale page is the thing this exists to prevent.
  
  Groups, timestamps, ANSI and links wait on structured log events - the plain-text half is what
  exists, and a group marker parsed out of plain text is a guess about somebody's build output.
  

• Log search that works during streaming and on a finished run, with deep links to a line

  A GET form on the run page and a server-rendered list of matches, each linking to the line
  itself - `#log-{job}-{line}`, with the id on the line the renderer emitted. Landing on the job
  is what a reader could already do by scrolling; landing on the line is the feature.
  
  No script, so it works on a finished run and while one is still streaming, and a result is a
  URL somebody can paste into a conversation - which is most of the value, because "it is
  failing" and a link to the line are different messages.
  
  Plain text rather than a pattern: a regular expression box on a log search is a way to hang
  the server on something somebody pasted, and what people type is a symbol name or an error
  code. Fifty matches at most, and a long line is clipped *around* the match - clipping at the
  end usually cuts off the part somebody searched for.
  

• Log redaction applied before persistence, driven by the secrets the job was given, with a visible marker where something was removed rather than a silent gap

  The second line behind the runner's own masking, and it exists because the first is somebody
  else's program: a runner that is old, patched or hostile is still one this instance accepts
  logs from, and "we asked it to mask" is not a property of the stored log. The value, its
  base64 form and its percent-encoded form are each replaced with `[redacted]`.
  
  The job's secrets are memoized per job rather than decrypted per chunk - a job's secrets do
  not change while it runs, and putting the instance's key work on the hot path of a streaming
  log would be a cost paid on every line.
  
  **A value split across two writes survives this pass**, and that is stated in the docs rather
  than hidden: it sees one chunk at a time, and holding the tail of every chunk to check the
  join would mean buffering a log meant to be streamed. The runner's masking covers it, because
  the runner sees the stream. A value shorter than five characters is also left alone - a secret
  of `dev` would blank a word everywhere it appears.
  

• Log size ceiling with a documented truncation behavior, and backpressure that slows a runner rather than dropping the middle of the log

  The ceiling is two megabytes a job, truncating at the *end* with a line saying so - a visible
  loss a reader can act on.
  
  The backpressure is `log_bytes_per_second`, an instance setting: past it a chunk is refused
  with a wait, and the runner sends the same one again. The chunk is idempotent on its sequence,
  so the retry costs nothing and the log stays whole and in order. **Dropping the middle would
  be worse than truncating the end**, because a reader cannot tell it happened.
  
  Instance-wide rather than per job, because that is where the problem is: one job is bounded by
  its own ceiling anyway, and what makes every other write on the box slow is forty jobs
  flooding at once. A setting rather than a constant because the right number is a property of
  the disk underneath, which the operator knows and this code cannot. Zero is off, and a
  settings table this cannot read is treated as off - refusing every chunk because a lookup
  failed would lose the output of every job on the instance.
  
  The runner's retry is bounded: after a few refusals it lets the chunk go and carries on.
  Losing a line is bad; a machine stalled on a server that keeps saying no is worse.
  

• Annotations: markdown, with a level (success, info, warning, error), a context key so a rerun replaces rather than appends, and append semantics when asked for

  Three levels rather than four - `notice`, `warning`, `failure`, which is the set the diff
  renders and the set the check model already had. A fourth called `success` would be an
  annotation nobody puts on a line.
  
  **`context:` names the group**, defaulting to the job's own name because that is what a reader
  wants on a pull request: "typecheck failed" is useful where "CI failed" is what they already
  knew. Naming it is for the job that reports two independent things - a lint pass and a type
  pass - which without a key would replace each other, so the last tool to finish would be the
  only one anybody saw.
  
  Replacing is the default, so a re-run does not double every finding on a line. `append: true`
  is for a suite that streams findings as it goes: it has nothing to send twice, and holding
  everything until the end would mean nothing on the diff until the job finished. An appended
  context is capped, oldest first, and says in the answer how many it dropped - a diff that has
  to render nine thousand annotations stops being a diff.
  

• Annotations render on the diff, on the file and line they name, on both sides. This is the row in the table at the top of this file, and it is the reason to build any of this.

  Done in phase 9, where the checks half of it lives: `app/Actions/Pull/annotations.ts`, hung on
  the diff through an `annotationsAt` slot beside the one review threads use. Both sides, on the
  line the tool named, and a finding spanning five lines is placed once rather than five times -
  repeating it would turn one warning into five and give a reviewer counting them a number the
  tool never reported.
  

• Artifacts: uploaded by glob, content-addressed, downloadable individually and as a set, searchable within a run, with retention policy and expiry visible before it happens

  The glob is `reviewos: { artifact-paths: [...] }`, the addressing is the SHA-256 the store
  names them by, and the expiry is on every listing and on the run screen before the date
  arrives rather than after.
  
  **As a set** is `/api/repos/workflow-runs/artifacts/archive`: a tar, uncompressed, written by
  hand in `Artifact/tar.ts` rather than pulled in - the format is a 512-byte header and padding,
  where zip needs a compressor and a central directory, and every machine that runs CI has
  `tar`. Compressing artifacts that are usually compressed already is the wrong trade.
  Assembled in memory with a ceiling, which is honest about what it is: streaming entry by entry
  is the change to make when somebody has a gigabyte of output, and refusing with the size beats
  an instance that falls over.
  
  **Searchable within a run** is `?q=` on the listing, filtered by name. A matrix of twenty
  writes twenty artifacts, and finding the one from the combination that failed means reading
  twenty near-identical names. The total stays the run's rather than the filter's: what a run is
  holding does not change because somebody typed in a box, and a total that moved with the
  filter is a number nobody could use for a retention decision.
  

• Artifacts are downloadable by later steps in the same run by name, which is the only reason most artifacts exist

  `reviewos-download built.tar` in a later job, over `POST /api/runner/artifacts/fetch`. By name
  rather than by id, because a later job knows what the earlier one called its output and does
  not know a database id - through the same name cleaning the upload applied, so a job asking
  for what it uploaded finds it.
  
  The run comes from the **job token**, never from the request: a runner that could name the run
  could read another run's build output, and on a fork's pull request that output belongs to
  somebody else's commit. Expiry is checked here as well as by the sweep, because the promise a
  retention date makes is about availability and honouring it only when a background job
  happened to have run is not a promise. A row whose bytes are missing answers 410 rather than
  404, so an operator does not go looking for a typo.
  

• Run metadata: string key/value pairs any job in a run can read or write, with compare-and-set so two parallel jobs cannot lose a write

  `reviewos-meta set version 1.4.2` in one job, `reviewos-meta get version` in another, and
  `POST /api/runner/metadata` underneath. A job that computes a version number, a preview URL or
  a decision has to hand it to a job that runs later, and both alternatives are worse: an
  artifact is a file for a string, and an output only reaches jobs that declared `needs:` on the
  producer - so a value cannot travel sideways or reach a job that was generated after the fact.
  
  **The compare-and-set is the design.** Without it two parallel jobs each read a list, each
  append, each write it back, and the second write lands on top of the first with nothing
  anywhere saying so. The guard is a `WHERE` clause on the version rather than a read followed
  by a write, because two jobs writing in the same instant would both pass a look-then-write.
  `if_version: 0` is "only if nobody has set this", which is a lock in one write.
  
  Scoped to the run rather than the repository: two runs of the same workflow are different
  commits, and a deploy that read the other run's version number would ship the wrong build. A
  value over ten kilobytes is refused rather than truncated - that is a file, and a file is an
  artifact.
  

• Run and job state machines exposed exactly as phase 9 defines them, with the interface, API, and webhooks reading the same states rather than three vocabularies

  The run list and the run page, at `/{owner}/{repository}/runs` and `/run/{number}`. One
  mapping in `resources/functions/runs.ts` turns a state into a word and a tone, and **the word
  is the state, capitalised** - the way this goes wrong is not a disagreement about data, it is
  a screen inventing a friendlier synonym, and then "Stopping" is in the interface while
  `cancelling` is in the API and somebody has to know they are the same thing.
  
  Colour is the second signal and never the only one: the state is written beside the dot,
  because a green dot and a red dot are the same shape to a reader who cannot tell those two
  colours apart.
  
  A blocked job says **what it is waiting for**, from the row it already has. "Blocked" alone
  sends somebody to open the workflow file to find out something the page knows. And the two
  commits are shown when they differ - what the run is about, and where its workflow came from -
  because a reader who cannot see that difference cannot tell a run of their code from a run of
  their code by somebody else's workflow.
  
  Webhooks for run transitions are not wired; the box's third vocabulary has no consumer yet.
  

• A dependency graph view: what ran, what is running, what is blocked and on what, and the critical path through the run

  Layers on the run screen - which jobs could have run at the same time - and the longest chain
  through them, named in order with the split between working and waiting.
  
  **The critical path is the half that pays for itself.** Adding runners does nothing for a run
  whose length is one chain of dependent jobs, and speeding up the slowest job does nothing when
  it is not on that chain; every other CI system leaves this to somebody with a stopwatch. The
  cost of a job is its wait plus its run, because an hour spent queueing is an hour of the run
  either way - a path that ignored the wait would point at the wrong job on a busy fleet, and
  pointing at the wrong job is worse than not pointing at all.
  
  Server-rendered with no JavaScript, per the phase 14 rule, and computed from the same rows the
  job list is built from so the two cannot disagree about a duration. A `needs:` cycle read out
  of the database is bounded rather than recursed: the parser refuses one, and a stack overflow
  is a worse answer than a wrong layer somebody can see.
  

• Timing on every job: queue time, run time, and the difference between them, because a slow run is usually a queue problem and the graph should say so

• Rerun a whole run, rerun failed jobs only, and rerun one job, each recording that it is an attempt rather than overwriting the first

  `POST /api/repos/workflow-runs/rerun` with `scope: all | failed | job`, and two buttons on the
  run screen. A re-run is a new *attempt* of the same run rather than a second run: the commit,
  the workflow version and the number are unchanged, and two rows would leave a reader guessing
  which was the answer.
  
  **Re-running the failed jobs carries the jobs that were skipped because of them.** Without
  that the second attempt finishes green with half the pipeline never having run, which is worse
  than not having the button.
  
  **The failing attempt stays readable**, which is the whole point - somebody re-runs a job to
  compare it against the failure. Log chunks now carry the attempt that wrote them, and the
  unique index is keyed on it: sequence numbers restart with each attempt, so without that the
  first chunk of a re-run collides with the first chunk of the attempt it is meant to be
  compared against and is silently dropped as a duplicate.
  
  A run that has not finished is refused: two attempts of one job in flight is exactly what the
  lease exists to prevent. And `GITHUB_RUN_ATTEMPT` is a real number now - it was a literal 1
  carried all the way to the runner and computed by nothing.
  

• Cancel a run and cancel a job, cooperative first and forced after a deadline (phase 9)

  The run has been cancellable since phase 9. `POST /api/repos/workflow-runs/cancel-job` is the
  other half, and the case the first cannot serve: one job stuck on a machine that has gone
  quiet, and nine others that are work nobody wants to throw away.
  
  Every row under the name goes, because a matrix is several rows under one - a button that
  stopped a quarter of what it says would be worse than none. Cooperative in the same shape as
  the run: a running job goes to `cancelling` with its lease revoked in the same write, and its
  dependants stay blocked until the machine acknowledges, because until then the job might yet
  report a success and skipping them first would be the control plane deciding an outcome it
  cannot see. A job that never started is cancelled outright and its dependants skipped in the
  same pass - there is nobody to wait for.
  
  Forcing after a deadline is the sweep from phase 9, which already reclaims a lease nobody has
  answered for.
  

• Unblock a block step from the interface, the API, and the CLI, recording who did it, and collecting input fields where declared

  The interface and the API have had it since the step model landed - the gate's fields are
  rendered as a form and become the job's outputs. `buddy ci:unblock <run> <job>` is the third,
  with `--input key=value` for each declared field.
  
  Who opened it comes from the credential rather than from an argument: a CLI that sent a name
  would be a CLI that could send somebody else's.
  

• A run's provenance is always visible: which workflow version, which commit, which trigger, which actor or token, which runner, and which pool

  A paragraph under the run's heading - the file, the definition commit it came from, the event,
  the ref, who started it, and whether the run was trusted - and, on each job, which machine ran
  it and which pool that machine belongs to.
  
  Scattered across four screens these are an investigation; together they are the answer to
  "what produced this". The definition commit is separate from the head commit because two runs
  of one commit can have run different files, and on a fork's pull request they always do. A run
  with no person behind it says `the schedule` rather than leaving a space that reads as a
  missing name.
  
  Writing this cost a lesson: both new blocks read values declared further down the script, and
  stx renders a page with every binding undefined rather than reporting the error - so the run
  page went blank and the failure surfaced as thirteen unrelated assertions.
  

• Keyboard navigation through jobs and log sections, and a run page that is readable with no JavaScript for the finished case, in line with the phase 14 rule

  **Links rather than key bindings**, because a binding needs script and the rule is that a
  finished run reads without any. The run's graph is the navigation: tab through the jobs in
  dependency order, press enter, land on one - each job is an anchor, and the log's folds are
  `<details>`, which the keyboard already operates. A focus ring appears for a keyboard and not
  for a mouse.
  
  The log search's results are the same idea one level down: a link per matching line, landing
  on the line rather than on the job.
  
  A finished run carries no script at all - the live region is the one the page has, and a run
  that has ended does not get it, because a page polling forever for a run that finished costs
  the instance a request a second for nothing.
  

• Tests: a run page rendered server-side for a finished run, redaction of a secret that appears in a log line split across two writes, an annotation replaced by context key, metadata written by two parallel jobs, and artifact download authorization from a different repository

  All five. The split-across-two-writes case is two tests rather than one, because the answer
  has two halves and either alone reads as a hole: **the runner catches it**, since a process
  writes bytes and the runner buffers to a newline before masking the joined line, and **the
  server does not**, because it sees one stored chunk at a time and holding the tail of each to
  check the join would mean buffering a log meant to be streamed.
  
  That is the documented limitation, and it has a test asserting the gap rather than a paragraph
  claiming there is none - a redaction feature people believe is total is worse than one whose
  edge they know.
  

Security

Phase 9's execution-plane gate covers sandboxing. This section is the part that applies even when every runner is somebody else's machine.

• Signed workflows. The control plane signs each step it dispatches, over the command, environment, plugins, and matrix values; the runner verifies before executing. Without this, anyone who can write to the control plane's database can execute arbitrary code on every runner in the fleet.

  `app/Actions/Workflow/stepSignature.ts`, signed at the claim and carried with the work. The
  signature is over a **canonical** encoding - keys sorted, no whitespace - of the run, the job,
  the matrix combination, and every step's command, `uses:`, environment and working directory.
  Signing the command alone would leave the environment as the way in, one indirection along:
  `NODE_OPTIONS`, `LD_PRELOAD`, a `PATH` with somebody's directory first.
  
  A separate key from the identity one, `purpose: 'steps'` in the same `instance_keys` table,
  published at `/.well-known/reviewos-step-keys.json` rather than in `jwks.json`. The two say
  different things - one vouches for *who a job is* to somebody outside, the other for *what a
  runner should execute* - and one set holding both invites a verifier to accept either
  statement in place of the other.
  
  The private half is encrypted with `APP_KEY`, which is why this is worth anything: a writer
  who has the database and not the process cannot mint a signature. A key that could not be read
  yields **no signature rather than no work**, because failing the claim would take a fleet down
  over a feature most instances do not enforce.
  

• Verification is enforceable per pool, and a pool can be set to refuse any unsigned step

  `require_signed_steps` on the pool, sent to the machine with the claim rather than configured
  on the machine: an operator turning it on covers every runner in the pool, not the ones whose
  config file somebody remembered to edit. `POST /api/instance/fleet` with
  `operation: require-signatures`, audited, and the answer says the consequence in words -
  including the honest one, that a runner older than this feature ignores the field.
  
  The runner checks before the workspace exists and before the first hook, against keys it
  **fetches**: a signature checked with a key from the same message proves only that the sender
  can do arithmetic. Keys it cannot fetch are a refusal, not a pass - the pool asked for signed
  work, and "I could not check" is not "it was fine".
  
  Off by default. A fleet that started refusing every job the day it upgraded is a fleet nobody
  upgrades. See [signed work](../signed-work.md).
  

• Key management: generation, rotation, and multiple active verification keys during a rotation

  `instance_keys`, generated on first use rather than configured - a key an operator has to
  create is a feature that stays off, and there is nothing to decide since RS256 is what every
  verifier reads.
  
  Rotation is both halves: a new key signs from that moment, and **the old one keeps
  verifying**. Without the second a rotation is an outage, because every token signed a minute
  ago becomes unverifiable - and a rotation that takes an outage with it is one nobody performs.
  Every token carries the `kid` of the key that signed it. The private halves are encrypted with
  `APP_KEY`: a backup that leaks one is somebody able to mint a token for any repository here.
  
  This covers the OIDC keys. Signing dispatched *steps*, the box above, uses the same table
  with `purpose: 'steps'` and a separate published key set.
  

• OIDC. A job can request a short-lived token, scoped to the run, repository, workflow, and branch, to authenticate to an external service without a stored credential. This is how a deploy stops needing a long-lived cloud key.

  `reviewos-oidc [audience]` in a step, `POST /api/runner/oidc` underneath, and the two
  documents a cloud reads at the root: `/.well-known/openid-configuration` and
  `/.well-known/jwks.json`. Not under `/api`, because the path is fixed by the specification and
  a document AWS will never ask for is a document that does not exist.
  
  **Every claim comes from the run, not the request.** The only thing a caller chooses is the
  audience. A token whose `repository` came from the body would be one any job could mint for
  any repository, which is precisely the thing this replaces.
  
  Fifteen minutes, and the e2e verifies a real token the way the other side would: fetch the
  discovery document, take the JWKS, check the signature with WebCrypto. A token only this
  codebase can check is not worth minting. See [identity tokens](./oidc.md).
  

• OIDC claims are documented and stable, so a cloud trust policy written against them keeps working

  **The names are GitHub's**, deliberately: a trust policy is a document somebody writes once
  and forgets, and the ones people already have are written against `repository`,
  `repository_owner`, `ref`, `workflow` and a `sub` of `repo:owner/name:ref:refs/heads/main`.
  Inventing better names would mean every user rewriting a policy to gain nothing.
  
  An environment makes the subject more specific - `repo:acme/api:environment:production` -
  which is what somebody means by "only the production deploy may assume this role". The full
  table is in [identity tokens](./oidc.md), and the discovery document lists them too, because
  "documented" that lives only in prose is documentation a machine cannot check.
  

• Secrets stored encrypted, scoped to a pool, repository, or environment, injected only into the steps that name them, never listed in plaintext after creation

  Five scopes now, narrowest wins: environment, repository, owner, pool, instance. The pool
  scope is the one this box was missing, and it is the credential that belongs to the machines
  rather than to the code - a registry token that exists because *these* runners are allowed to
  publish is not a fact about any repository, and writing it into each repository that needs it
  is how one credential ends up in twenty places and is rotated in three.
  
  Set through the fleet endpoint, so it takes `fleet` at admin on the token and instance
  administration or maintaining that pool on the person: this is drawing the pool's boundary,
  not administering a repository. A job receives it because a machine in that pool took it, so a
  run on anybody else's machines never sees it however the workflow asks - and neither does a
  runner in no pool, which is what every installation had before pools existed. A repository's
  own secret beats a pool's: the pool says where work runs, the repository says what is running,
  and the second is the more specific statement.
  
  **Named at the job rather than at the step**, and the wording above overstates what a claim
  can do. A step is not a boundary the control plane can enforce anything at - the whole job's
  environment is handed to one machine at one claim - so `secrets:` on a job is where "only what
  names them" is real, and pretending otherwise would be a permission that exists only in the
  documentation. A job that names what it needs is a job whose compromised dependency cannot
  read the deploy key it never asked for, which is the property this clause is for.
  

• The recommended path stays an external secret store, with first-party support for fetching from one, because the best secret is one we never held

  A secret's stored value may be a reference - `store://prod/secret/data/publish#TOKEN` - and
  then this instance holds a path rather than a credential: a copy of its database is a list of
  names and locations. Two drivers, both first-party because most instances already have one of
  them: a directory the platform mounted (Docker secrets, a Kubernetes volume) and HashiCorp
  Vault KV v2, whose token is read from a file per request so a rotated one is picked up without
  a restart and never appears in `ps`.
  
  **A reference names a store the operator configured, never a URL.** That is the whole
  boundary: "read this from the store you set up" is a different sentence from "fetch this from
  an address a repository administrator typed", and the second is a request this server makes
  from inside the network on somebody else's say-so. A path that climbs out of a file store is
  refused before and after normalising it.
  
  **A reference that cannot be read fails the job by name**, at the claim, before the first
  step. A store that is down or a path that moved otherwise becomes an empty credential and a
  failure forty minutes later against somebody else's API, with an error that says nothing about
  this instance. The same reporting caught an older silence: a value this instance could no
  longer decrypt - a rotated `APP_KEY` - used to be skipped, which looked exactly like a secret
  nobody had set.
  

• Fork policy: a run triggered from a fork gets no secrets and no OIDC by default, cannot supply the workflow definition it runs under, and requires approval to run at all. Phase 9 states this rule; this is where it is enforced in the dispatch path.

  Three of the four clauses were already enforced and tested: secrets are chosen at the claim
  where the trust flag is, `POST /api/runner/oidc` refuses an untrusted run outright, and
  `dispatchPullRequest` reads *registered versions* so a head branch's own files are never parsed
  into a definition.
  
  The fourth was missing. A fork's pull request ran the moment it was opened, on machines an
  operator provided - and while it got nothing, running at all spends the fleet and reaches
  whatever those machines reach. `fork_run_approval` decides: `first-time` by default, matching
  what GitHub learned the hard way (ask about a contributor whose work has never landed here,
  stop asking once it has), with `always` for an instance whose runners sit somewhere expensive
  and `never` for the behaviour of an instance that has not thought about it.
  
  **A held run is `waiting`, not `queued`**, which is what keeps it away from the claim - there is
  no second place to get the hold right - and its jobs read as blocked with the reason, because a
  list of queued jobs under a run nothing will claim is somebody investigating their runners for
  an hour. `POST /api/repos/workflow-runs/approve-fork` takes `approve` or `refuse` under
  `workflow:approve`, the same ability a deployment gate needs: stopping a run is safe and
  starting a stranger's code is not.
  
  **Approving does not make the run trusted.** It runs; it still gets nothing. Conflating "you
  may run" with "you are ours" is the mistake behind the published secret-theft write-ups, and
  the answer says so in words as well as leaving the flag alone. A refusal cancels the run and
  keeps it, with who decided: the next person to look at the pull request needs to see that a
  decision was made rather than that nothing ever ran.
  
  A contributor who can push here is not asked - a push from them would run without asking - and
  that is the real permission rather than a row in `repo_collaborators`, since the repository's
  owner is not a collaborator of their own repository.
  

• Fine-grained token permissions for reading runs, dispatching runs, managing workflows, administering pools, and reading logs, each separable, per the phase 1 rule that there is no fallback token type

  Two repository scopes and one instance scope, none of them implied by anything else.
  `actions` at read reads runs and jobs, at write starts, stops and approves them, and at admin
  turns a workflow on and off - which sits at admin because a disabled workflow is a required
  check that quietly stops appearing on pull requests, and that is a way around a protection
  rule rather than a build. `actions_logs` is read-only and separate on purpose: redaction
  covers the secrets this instance knows about and cannot cover one a script assembled, so
  "watch my builds" and "read every line my builds printed" are different sentences.
  
  `fleet` is instance-wide, because the machines belong to no repository. Before it, a token
  issued to a deployment script - carrying `contents` and nothing else - could create pools,
  appoint maintainers and mint registration tokens the moment its owner was an instance
  administrator: the endpoint asked who was asking and never what they were holding. The scope
  check runs after the person check, so somebody with no standing over a pool still gets the
  404 that keeps its existence private, and somebody who may act but whose token was issued for
  something else gets a 403 naming the scope.
  
  Three things this moved. Reading runs and logs used to ride on `repository:read`, so every
  token that could clone could read every run and every log. Dispatching, cancelling and
  approving used to ride on `checks: write`, so an external CI issued a token to post results
  could also start runs on this instance's machines. And managing workflows used to be
  `workflow:dispatch`, so anything that could start a run could disable one.
  

• Every state-changing operation is in the audit log from phase 11, attributable to a token as well as a person

  Thirteen verbs: dispatching a run (by hand and by `repository_dispatch`), cancelling a run or
  one job of it, re-running, opening a gate, enabling and disabling a workflow, writing and
  removing a secret, a variable, and an environment. Every one of them spends the instance's
  machines, changes what a branch rule sees, or hands a job a credential, and until now none of
  them was in the log at all - "who turned that workflow off" had no answer, and "who started
  this deploy" had only the run's `actor_id`, which a token cannot be traced through.
  
  A secret's row carries its name and its scope and never its value: an audit log that recorded
  the value would be a second place the secret lives, with weaker protection than the first. A
  variable's row does carry the value, because a variable is not a credential and "who pointed
  the deploy at production" is the question the log is for. The end-to-end test asserts both,
  through a real boot with a bearer token rather than by calling the actions - `actor_id` is
  filled either way, and `access_token_id` is only filled when the request carried one.
  

• Tests: a forged step signature, a rotated key mid-run, an OIDC token used against another repository's trust policy, a fork run attempting secret access, and a token with dispatch but not admin attempting each admin route

  `tests/e2e/ci-security.test.ts`, against a real boot rather than over plain values, because
  every one of these fails quietly if it fails: a forged step runs, a rotation kills a run in
  flight, an identity token opens somebody else's account, a fork reads a deploy key, a narrow
  token turns out to be wide.
  
  The forgery is done the way a database writer would do it - payload intact, real signature,
  one command somebody else's - and the environment variant too, since a step whose command is
  untouched still runs with whatever it was given. The rotation asserts both directions: work
  signed before it still verifies, because a rotation that killed every dispatched job is a
  maintenance task nobody dares run, and a runner holding a stale key set refuses work signed
  after it rather than treating what it cannot check as consent. The fork claims its own run by
  id rather than whatever was queued first, and is handed no secrets and refused an identity
  token - after a trusted claim proves delivery works, so a broken secret path cannot pass as
  the feature working. The narrow token dispatches successfully first, which is what makes the
  four refusals that follow about the permission rather than the path.
  

Test intelligence

Buildkite Test Engine is a separate product and, for a lot of their customers, the reason they are there at all. It ingests test results from any CI, not just their own, which is the shape to copy: it should work for a repository that has not moved its CI here yet.

• TestSuite, TestRun, TestExecution, and ManagedTest models. A test is identified by suite, scope, and name; scope is what separates two tests with the same name.

  A rename makes a new test, deliberately. Guessing that `renders the header` and
  `renders a header` are the same test is guessing about intent, and being wrong loses the
  history of the test that still exists - which is the history somebody is about to decide
  from. `tests/e2e/test-intelligence.test.ts` says so out loud.
  

• Ingest JUnit XML and a documented JSON format over an authenticated endpoint, from any CI, with the run tied to a commit and optionally a pull request

  `/api/repos/tests/ingest`, with `check:report` - the ability a CI integration already has to
  say a commit passed. Reporting *which* tests passed is the same act at a finer grain, and a
  new scope would mean every existing integration asking for one more permission to tell you
  more. The reporter's own `key` makes the run idempotent, because every collector retries and
  a doubled history is one flake detection then answers from.
  
  **JUnit is read with a scanner, not an XML library.** The input is a file from a machine this
  instance does not control; a reader that cannot be made to resolve an external entity or
  allocate a gigabyte is worth more here than one that handles namespaces.
  `tests/unit/test-junit.test.ts` proves `&xxe;` stays text, and that a truncated report keeps
  the cases before the cut rather than being thrown away whole.
  

• First-party collectors for the frameworks people actually use, starting with the ones this repository could use on itself, and a documented protocol so the rest are writable by anyone

  `./buddy tests:report --url ... --repository owner/name --suite unit`, which is the one this
  repository can point at itself. Bun already emits JUnit, so the collector carries the four
  facts a report needs and a test runner does not know - repository, commit, branch, and an
  idempotency key - and nothing else. A collector that reimplemented the reporting would be a
  second thing to keep working.
  
  The key is the run **and the attempt**, because a rerun is a different report of the same
  commit and keying on the run alone drops exactly the results that show a flake.
  
  **It exits with the test runner's status, not the endpoint's.** A network failure while
  reporting must not turn a passing suite red, and a failing suite stays red when the report
  never arrives: losing the evidence is not the same as the commit being wrong.
  `--use-verdict` inverts that on purpose, which is how a muted test stops failing a build.
  
  Verified against a stub instance: seven tests in, seven `<testcase>` elements out, posted
  with the credential, the suite, the sha and the key, and the answer printed back including
  the newly-flaky list.
  

• Per-execution: result, duration, retries, failure message, stack, and the job it ran in

• Tags as dimensions on an execution, for filtering and aggregation

  Both are why the JSON format exists: JUnit cannot carry retries or a dimension without
  somebody inventing an attribute, and a failure that only happens on one browser or one shard
  is the most useful thing a suite can say.
  

• Ownership: map a test to a team or a path, so a failure has an addressee

• Flaky detection: a test that passed and failed on the same commit, or that changes verdict across reruns, over a configurable window

  Two shapes, over the last twenty executions: disagreeing about one commit, and passing only
  after a retry. **The second is the one tools throw away** - a reporter that stores the final
  verdict has already lost the fact that the test failed twice first.
  
  One failure is a failure, not a flake. Calling it flaky there is telling somebody to ignore a
  broken test.
  

• Test states: enabled, muted, skipped. A muted test still runs and still reports, but does not fail the run. A skipped test does not run. The difference matters and most tools conflate it.

• Quarantine is auditable and expires: who muted it, when, why, and a review date, so quarantine does not become a graveyard

  A mute needs **both** a reason and a review date or it is refused, and the listing marks the
  ones whose date has passed `overdue`. The friction is the point: thirty seconds against a
  test that would otherwise be off forever.
  
  Muted failures are counted, kept in the test's history, and shown - they are only set aside
  when the endpoint reaches a verdict. So the day the test starts passing again is visible,
  which is exactly what skipping it destroys.
  

• Monitors and actions: a rule that watches a test over time, raises an alarm when a condition holds, recovers when it stops, and fires an action once per transition rather than per run

  Three conditions - `fail_rate`, `flaky`, `duration` - and no expression language, because a
  general one is a second product to document, test and get wrong.
  
  **The state lives on the monitor, and that is the whole design.** "Is the failure rate above
  five percent" is true every hour it is true, so a rule that acted on the answer would send
  the same alarm twenty-four times a day - and the channel it arrives on is the one that has to
  work the day it matters. A monitor in alarm for a month sends one message.
  
  Three decisions the tests hold. A **measurement it could not take is not a recovery**: a
  suite nobody reported for would otherwise clear an alarm because the reporting broke, which
  is when the alarm matters most. A **muted test cannot cause one**, since its failures are set
  aside everywhere else and counting them here would alarm on exactly the tests somebody
  already decided about. And **exactly at the threshold is not over it**, because "above five
  percent" is what somebody wrote down.
  
  The threshold for `fail_rate` is a percentage rather than a share, which is a decision about
  a trap rather than about taste: `5` typed at a field wanting a share is five hundred percent,
  a monitor that can never fire, and it reads as covered.
  
  Found on the way: `schema.double()` is an alias for `float` in ts-validation, so it generates
  a four-byte column that hands `2.5` back as `2.5000000596046448`. Thresholds are `decimal`,
  which is exact - and percentages fit its two decimal places, where shares would not.
  
  Transitions leave as the `test:monitor` webhook with `alarm` or `recovered` in `action`.
  Webhook-only, like `check:reported`: nobody wants an inbox entry each time a suite wobbles.
  

• Reliability and duration trends per test, per suite, and per branch, with the slowest and least reliable surfaced without a query

  A Tests tab on the repository, beside Runs. "Without a query" is the whole requirement: every
  number on it is derivable from the execution table by anybody willing to write SQL, which
  means in practice nobody looks, and the slow test that got slower over four months stays
  invisible until somebody wonders why CI takes eleven minutes.
  
  **Ranked by total time, not by the slowest single run.** A 40ms test that runs in every one
  of two thousand executions costs more than the one nine-second test, and the total is what
  the wall clock feels.
  
  The page states its own evidence: a test with fewer than five runs in the window is not
  ranked, and the number left out is shown. A reliability figure computed from four executions
  is not a measurement, and presenting it as one teaches people to distrust the rest of the
  page. `?branch=` narrows it, which is what makes "per branch" true rather than claimed.
  

• Test splitting: a client that distributes a suite across parallel jobs using historical timing, so parallelism stops meaning "split alphabetically and hope"

  Longest-processing-time-first, which is within 4/3 of optimal and enough - the input is
  estimates, so chasing an optimal partition of approximate numbers buys nothing. What matters
  is that the big items are placed first: placing them last is how one node ends up eleven
  minutes long while another finishes in forty seconds.
  
  **Two properties matter more than the quality of the partition, because both are silent when
  they break.** Every item lands on exactly one node - a test that runs twice wastes a machine,
  a test that runs nowhere stopped being run and nothing says so. And every node computes the
  same partition without talking to any other, which makes determinism load-bearing down to how
  ties are broken.
  
  For a job here the runner writes `reviewos-split` onto the PATH beside `reviewos-upload`, so
  a sharding job needs no repository credential: the job token it already holds can read the
  timings. `tests/unit/runner-shell-commands.test.ts` executes both generated scripts, which is
  how the first one's broken escaping was found - a `\n` in a template literal became a real
  newline inside a JavaScript string, and nothing type-checks a shell script.
  

• Splitting degrades honestly with no history: deterministic partition, and a note saying it had nothing to work with

  And a file nobody has timed is assumed to cost what a typical file costs, not nothing. Zero
  is the obvious default and it hides: adding zero never changes which node is cheapest, so
  every new file lands on the same node - the pull request that added twelve test files would
  put all twelve on one.
  

• Test results appear on the pull request, and a newly flaky test introduced by a branch is distinguishable from one that was already flaky on the base

  A Tests panel on the checks tab: which tests failed and what they said, the per-suite counts,
  and the sentence nobody else writes - "this branch made 1 test flaky", against "6 were
  already flaky on main, so this branch did not cause them".
  
  **The distinction is measured, not read off a flag.** Flakiness elsewhere is a property of
  the test, so a test unreliable on main for a month decorates every pull request that touches
  nothing near it, and "there are seven flaky tests" becomes a sentence reviewers skip. Here
  the same rule runs twice, over this branch's history and over the base's, and the difference
  is the only part anybody has to act on.
  
  A commit nothing has reported on says so rather than showing green, which is the rule the
  checks rollup beside it already follows: green for unmeasured is how a misconfigured
  collector goes unnoticed for a month. `tests/e2e/pull-tests.test.ts`.
  

• Retention policy on execution data, configurable, with the storage cost stated

  `test_retention_days`, an instance setting, default 90, swept daily. Executions are the one
  table in this product that grows with how often *machines* run rather than with how much
  people do - two thousand tests reported on every commit is two thousand rows per push.
  
  The cost is stated rather than left to be discovered: about 220 bytes per execution with
  indexes, so two thousand tests at ten pushes a day for ninety days is roughly 400MB. Zero
  keeps everything and the setting's own description says the cost is then unbounded, because
  finding that out from a full disk is the failure worth spending a sentence on.
  
  **Tests, suites, mutes, owners and reasons are never swept**, only executions and the runs
  they belonged to. Those are decisions somebody recorded rather than data that accumulated,
  and a sweep that took the mute with the history would silently un-quarantine a test. Batched
  at two hundred runs, because the first sweep on a year of history is otherwise one statement
  against millions of rows holding a lock long enough for pushes to time out.
  

• REST API, webhooks, and generated OpenAPI for suites, runs, executions, and states

  `GET /api/repos/tests?view=suites|runs|executions|states`, because those are four different
  questions: which suites exist and how the last run went, which runs a suite has had, what
  happened to individual tests in one run, and what this instance currently believes about a
  test - steady, flaky, muted, owned, and whether its quarantine review is overdue.
  
  A page is not an API, which is what this box was really about. A team's own dashboard, a
  release script that refuses to ship while a suite is red, an agent asking whether the test it
  is about to change is already flaky: each of those had to scrape HTML or query the database,
  and both are ways of depending on something nobody promised to keep.
  
  Reading takes `workflow:read` rather than `contents:read`. Test results say which tests exist
  and which are failing, which is a shape of a private repository's contents - so they sit with
  the runs rather than with permission to clone. A public repository answers anonymously, which
  is what makes a public dashboard possible at all. Executions carry the failure message and not
  the stack: the stack is the larger half of the row and the half a dashboard never renders.
  
  The webhook is `test:recorded`, carrying the suite, the branch, the totals and the run id -
  the totals rather than the executions, because a report of two thousand tests is a delivery
  that times out on exactly the repositories that matter, and the id is how a receiver asks for
  the detail. It sits beside `test:flaky` and `test:monitor`, which are thresholds where this is
  the plain fact that a run happened.
  

• Tests: ingestion of a malformed report, the same run reported twice, a test renamed between runs, flake detection across a rerun, and muting that does not hide the result

  Fourteen in `tests/e2e/test-intelligence.test.ts`, twelve in `tests/unit/test-junit.test.ts`.
  The malformed case is the load-bearing one: a collector posting an HTML error page, because a
  proxy answered instead of the file it meant to send, must not read as a suite with no tests -
  which is indistinguishable from a suite that passed.
  
  Splitting has its own twelve in `tests/unit/test-split.test.ts` and three more over HTTP,
  including the partial-history case.
  

Delivery

Phase 9 owns deployments. Two Buildkite capabilities sit next to them and belong here.

• Preview environments linked on the pull request, expiring on merge or close, using phase 9's deployment model rather than a second one

  `app/Models/Deployment.ts` is that model, and it is one model: a preview is a deployment with a
  pull request on it. That is what makes expiry a fact rather than a feature - the thing it
  belongs to closed, so it is no longer current - where a previews table of its own would need a
  sweeper somebody has to keep in step.
  
  This instance never deploys anything. A job does, with the credentials its environment released
  to it, and records what happened: which commit, which environment, what URL came out. So the
  row is provenance, and the history is what somebody reads when a page is wrong on a Monday and
  nobody remembers what shipped on Friday. Recording one takes the ability that opens a gate,
  because writing "production is on this commit" is saying where the product is - and the audit
  row names the token as well as the person.
  
  A second push replaces the first rather than adding to it: a branch pushed to five times would
  otherwise have five live previews, four pointing at URLs that no longer answer, and the page
  would show whichever the query ordered first. Expiry marks rather than deletes - "what was on
  this URL last Tuesday" is a question people ask - and it says which way it ended, because
  merged and closed mean different things. A preview recorded *after* the merge is swept on the
  next deployment: a slow deploy finishing after the pull request landed is the ordinary case.
  

• macOS runners as a first-class case in documentation and pool configuration: they are how mobile delivery works and they are the case every CI product handles worst

  In [autoscaling](./autoscaling.md), with the reason they are handled worst: every design
  assumes a machine is disposable, and Apple's licence ties macOS to Apple hardware - so a mac is
  bought or rented by the hour rather than created in twenty seconds. Three things follow, and
  they are the difference between a fleet that works and one that produces a mystery a
  fortnight: their own pool, because these machines hold signing material and a pool that also
  takes every repository's pull request checks is one where somebody else's dependency runs
  beside the keychain; a `cleanup` hook, because `--jobs 1` is not available to a machine that
  lives a year; and the Xcode version in `--tags`, because a build that needs 16.2 landing on
  15.4 fails halfway through with an error about a Swift version.
  

• Signing material and store credentials as environment-scoped secrets released only to the publish step, never to build or test steps

  The environment scope already did this; what was missing was saying so where somebody shipping
  an app would look, and proving it. A build job runs whatever the dependency tree brought with
  it, so a certificate in a repository secret is a certificate any of it can read.
  

• Tests: a preview expiring, a build step attempting to read a publish secret

  `tests/e2e/previews.test.ts` and the release-path block in `tests/e2e/workflow-secrets.test.ts`.
  The build job is tested twice: not receiving the publish credentials, and **not receiving them
  when it asks for them by name** - naming a secret narrows what a job gets rather than widening
  it, and a `secrets:` list that could widen would be the feature undoing itself.
  

Insight

Buildkite sells reporting on the fleet, and it is the thing an operator opens on a Monday.

• Per workflow and per repository: run count, success rate, duration percentiles including p95, failure by step, and retry rate, over a selectable window
• Queue wait time by queue and by pool, which is the number that tells an operator to add runners
• Runner utilization and idle time, which is the number that tells them to remove some
• Cost proxies: total run minutes by repository, owner, and queue. We do not bill, but somebody self-hosting this pays for the machines and should be able to see where they went.
• Flaky test impact: runs failed by a test that was already known flaky, which is the argument for fixing it
• The whole surface is available through the API in the same shape as the screens

Clients

Buildkite's surface is reachable from a terminal, from Terraform, and from a program, and phase 12 already commits us to the principle. These are the pipeline-specific pieces.

• CLI: validate a workflow, dispatch one, follow logs, inspect a run, unblock a step, cancel, and retry from a step, as a client of the public API only (phase 12)

  `buddy ci:validate`, `ci:runs`, `ci:run`, `ci:logs --follow`, `ci:dispatch`, `ci:unblock`,
  `ci:cancel` and `ci:rerun`.
  
  **A client of the public API and nothing else**, which is the constraint that makes it worth
  having: a command that reached the database would work on the instance's own machine and
  nowhere else, and would stop being a test of whether the API is usable by anybody. The e2e
  runs the real binary against a served instance with a token, the way an operator would - and
  it found the log endpoint takes a job id where a person has a job name, so the command
  resolves it and lists the names when it cannot.
  
  `ci:validate` is the exception that needs no instance and no credential: parsing is this
  repository's own code, and asking somebody to push a broken file to find out it is broken is
  the loop it removes.
  
  Failures say which kind they are. A CLI that prints `{"error":"Not found"}` and exits 1 has
  told somebody nothing: they cannot tell a wrong token from a wrong repository from an instance
  that is not running, so each of those says so in words.
  

• Workflows as code: a typed SDK, in the shape Cloudflare's @cloudflare/ci demonstrates, where the workflow is a program and ordinary control flow expresses the graph. It runs as an orchestrator job under the durable-execution rules in phase 9, never in the control plane, and it produces the same normalized rows as a YAML workflow. This is the second front door, not a second product.

  `defineWorkflow` and `buddy workflow:build`, documented in
  [workflows as code](../workflows-as-code.md). What it buys is what YAML cannot express: twelve
  jobs over a list of packages is a loop rather than twelve copies somebody keeps in step, and
  `needs: workflow.ids()` is "everything above" without a list to maintain.
  
  **It emits the YAML this instance already reads**, and that is what keeps it a front door.
  The parser, the conformance table, the extension rules and every refusal are shared, so a
  program cannot quietly express something a file may not - a `block:` gate with steps under it
  is the same error through both doors. The test asserts it directly: the same workflow written
  both ways normalizes to the same rows.
  
  Nothing here runs a workflow and nothing reaches the control plane. The program produces a
  document, which is also what keeps the determinism problem small: the only thing that has to
  be the same twice is the shape, and the shape is written down.
  

• The SDK's determinism rules are enforced by its own types and a lint rule where they can be, and by the replay check where they cannot. An author should learn about a forbidden clock read from an editor, not from a diverged run three weeks later.

  Three layers. The **types** hand an author the builder and nothing else - no clock, no
  environment, no fetch - so most of the rule is not reachable. The **check** reads the source
  at build time and names what it found with the line: `Date.now()`, `Math.random()`,
  `process.env`, `fetch()`, a directory listing, a fresh identifier. Comments and strings are
  ignored, because a rule that fires on an explanation of why not to use `Date.now()` is one
  people work around by deleting the explanation. A file that reads any of them is **refused,
  not warned**. And the **replay** builds twice and compares, which is the layer neither of the
  others can replace: a program reading something nobody thought of still produces two different
  documents.
  
  Honestly short of the box in one respect: the check runs at build time rather than in an
  editor. The editor half needs the rule to live in pickier's own rule set, which is a separate
  package with its own release - and a rule that lands in three weeks is not a rule that helped
  the person writing a workflow this afternoon. The messages are the ones an editor would show
  when it gets there.
  

• Terraform provider covering workflows, schedules, pools, queues, tokens, and secrets, because a fleet that cannot be declared is a fleet that drifts

  **The declarative half is done and the provider is not**, and the split is deliberate rather
  than a stopping point. `buddy fleet:apply fleet.yml` converges pools, their queues - including
  a queue declared paused, with its reason - and which repositories each pool serves, with
  `--plan` printing what would change and touching nothing. It is idempotent by construction, so
  it is safe to run from a pipeline, and **it never removes what the file does not mention**: a
  partial file applied on the wrong afternoon would otherwise drain the fleet, so anything on the
  instance and absent from the file is reported as drift for a person to decide about.
  
  What is left is the provider binary itself. Terraform providers are Go programs published to
  their registry from their own repository with their own release, and this machine has neither
  a Go toolchain nor Terraform - so writing one here would mean shipping several hundred lines
  nobody had compiled, which is the thing this roadmap keeps refusing to do. The endpoints it
  would drive exist and are idempotent; the work is a repository, not a feature.
  
  Two of the six named resources are not the provider's to own, and that is worth writing down
  before somebody tries: **workflows and schedules are files in git**. A provider that wrote
  them would be a second source of truth for something the repository already holds, and the
  first force-push would decide which of the two was real.
  

• MCP surface for runs, logs, and test results, so a coding agent can read a failure without scraping a page

  Four tools beside the review ones: `list_workflow_runs`, `read_workflow_run`, `read_job_log`,
  `read_tests`. Each is a call to this instance's own API carrying the token the connection
  authenticated with, like every other tool here - so there is no second permission check to
  disagree with the first.
  
  `read_job_log` takes the cursor from the previous call, because an agent re-reading a
  hundred-thousand-line log to find the four new lines is an agent spending its context on
  nothing.
  
  `read_tests` earns its place with one sentence in its description: check whether a test is
  already flaky before blaming the diff in front of you. A test unreliable for a month is not
  evidence about this change, and an agent without that fact writes a confident and wrong
  review.
  
  The test that named tools one at a time now walks every tool's path against `routes/api.ts`.
  A tool whose endpoint was renamed always 404s, and a model handed one of those does not
  conclude the tool is broken - it concludes the task is impossible and abandons the work.
  

• Webhook events for every run, job, and test transition, redelivered through phase 5

  `run:transitioned` and `job:transitioned` carry the new state in `action`, so one
  subscription covers a whole lifecycle. `test:monitor` carries `alarm` or `recovered`.
  `test:flaky` fires when a test crosses from steady to unreliable - **once**, not on every
  run, because the test that has been flaky for a month is not news and a receiver told about
  it every time writes a filter that hides the one that broke today.
  
  Emitted where the crossing is known rather than by the caller: the row said steady a line
  ago and says flaky now, and reconstructing that afterwards means asking the database what it
  used to think. Never awaited for its effect, and unable to fail an ingestion - a webhook is a
  consequence of the result being recorded, not a condition of it.
  
  All four ride phase 5's delivery, so redelivery, signing and the delivery log come for free.
  

• Notifications on run outcome, per workflow and per step, to the channels phase 5 already delivers, with a rule set rather than an on/off switch

  A rule is a sentence: *this workflow, this branch, this job, this outcome, this person*.
  Globs for the workflow and the branch, so `release/*` is one rule rather than one per release
  branch, and a workflow can be named by path, by file name or by its own name - all three are
  what somebody has to hand. The switch this replaces is the reason CI notifications get muted:
  a repository running twelve workflows on every push produces twelve messages about things that
  were always going to pass.
  
  **`recovery` is the condition that earns the feature**, and the one every implementation gets
  wrong: the first success *after* something that was not one. Not "not failed" - a first run
  going green is not a recovery, nothing was broken, and treating it as one is how a new
  workflow's first success wakes somebody up.
  
  **One person hears once about one run**, however many of their rules matched, and the narrowest
  match decides what it says: a rule naming a job beats one naming the run, because whoever wrote
  the narrower rule said what they cared about. Four messages about one push is how somebody ends
  up muting the repository.
  
  A rule names somebody on this instance and never an address - the channel is the recipient's
  own setting, which phase 5 already holds. Subscribing yourself needs read access; subscribing
  somebody else needs `repository:settings`, because signing a colleague up for an alert is not a
  thing a passer-by may do. Access is checked again at delivery, since a rule outlives the access
  that justified it, and a fork's run notifies nobody: the workflow is the base branch's, the run
  is a stranger's code, and a pull request must not be a way to make this instance message a
  maintainer on demand. A rule naming a workflow that does not exist is refused when it is
  written rather than silently never matching.
  

• A status badge endpoint, cached, for a workflow on a branch

  `GET /api/repos/badge?owner=&repo=&workflow=&branch=`, drawn rather than fetched: no external
  font, no external anything, since a badge is served into somebody else's page. The label and
  the message are each sized against a per-character width table, because `passing` and
  `illiiil` are the same length and not the same width, and a badge sized by character count
  clips one and pads the other.
  
  It never fails and it never discloses. A repository that does not exist, one the reader may
  not see, and one that has never run this workflow all get the same grey `unknown` pill with a
  200 - byte for byte, which the test asserts, because a difference in width or colour is a
  difference somebody can measure and measuring it is how a private repository gets confirmed.
  A 404 would be the same disclosure through a broken image.
  
  Cached against the run rather than the clock: the `ETag` is the workflow, the run and its
  state, so a repeat ask costs a query and no rendering, and a build finishing invalidates it
  at once rather than a minute later. The badge reports the newest *finished* run, so it does
  not flicker between `running` and the answer while a build is going - the question it answers
  is whether the branch is good.
  
  The workflows page carries the line to paste, with the picture it produces beside it. The
  path is relative: this instance is not `reviewos.org`, and a snippet carrying somebody else's
  hostname is the one bug a badge can have that nobody notices until the README is public.
  

Arriving from somewhere else

A migration path is a feature. There are two of them and they are not the same shape.

From GitHub Actions there is no migration, and that is the point of the compatibility section: the workflow files are the workflow files. What is still needed is everything around them.

• Phase 8's importer carries workflow files across untouched, and reports which constructs the conformance suite says will not run yet, before the move rather than after

  A `ci` stage on the import, after the releases. The files are already here - they came with the
  clone - so the stage registers them and produces the report, which is the actual deliverable: a
  file that copied cleanly and does not run is the worst outcome of a migration, because
  everything looks moved and the first push is green in the file and red in reality.
  

• Repository and organization secrets, variables, and environments import as part of the same operation, since a workflow without them is green in the file and red in the run

  Variables come across with their values - they are not credentials, and the endpoint that lists
  them returns them. Environments come across with their protection, because a deploy gate that
  silently did not move is a rule somebody believes is on. **Secrets come across as names only,
  and cannot come across any other way**: no forge hands a secret's value back, ours included, so
  the import names them and says they have to be set here. "You have eleven secrets to set",
  written down before the move, is the difference between a planned afternoon and a broken
  deploy.
  
  Nothing overwrites what is already here. An import is re-run, and a second pass that replaced a
  value somebody had corrected would undo the work between the two.
  

• Self-hosted runner labels are preserved, so runs-on: [self-hosted, gpu] keeps meaning what it meant

  Preserved by not touching the file, and *checked*: the report counts how many active machines
  on this instance answer to each label a workflow asks for, and names the ones nothing answers
  to. `runs-on: [self-hosted, gpu]` keeps meaning what it meant only if a machine carries both,
  and the difference between "queued" and "queued forever" is worth a sentence before the move.
  

• A per-repository report after import: workflows found, constructs unsupported, actions referenced that the instance cannot resolve, and what to do about each

  All four, and the third is the one that surprises people: an unqualified `actions/checkout@v4`
  resolves to nothing here, because this product refuses to guess github.com. The report says so
  with the fix beside it rather than leaving it to the first red run.
  

• Tests: import a real repository's workflow directory and assert the run graph matches what Actions produced for the same commit

  `tests/e2e/import-ci.test.ts`, over the conformance fixtures - the shapes real repositories
  have rather than files written to pass. It asserts the graph: which workflows a push starts,
  the matrix expanded into one job per combination, and a tag starting only what filters on tags.
  
  **It found a real one.** `on: push: tags: ['v*']` ran on every push to every branch, because
  the rule that tags are opted into had no mirror: a workflow naming tags and not branches was
  read as naming no branch filter. So a release workflow that publishes on a tag published on
  each commit to main - the exact failure this section exists to prevent, and one nobody notices
  until it ships.
  

From Buildkite it is a translation, and it is cheap because their format is public.

• An importer that reads a pipeline.yml and emits workflow YAML, reporting per step and per attribute what translated, what translated with a change in meaning, and what has no equivalent. The report is the deliverable; a silent partial translation is worse than a refusal.

  `buddy import:buildkite .buildkite/pipeline.yml --out .reviewos/workflows/ci.yml`. The workflow
  goes to the file and the report to the terminal, on purpose: caveats written into the file as
  comments are caveats nobody reads twice. Every attribute lands in one of three buckets, and an
  attribute this instance has never heard of is named rather than dropped.
  
  Two things the translation gets right that a naive one would not. **Steps between barriers stay
  parallel** - chaining each to the one before would serialise a pipeline that was not, which is
  a translation slower than the original and reads as this product being slow. And a `wait` is a
  job here rather than a separator, so the graph is said in `needs:` without changing shape.
  
  The output is checked by parsing it: the test asserts this instance would register the file it
  produced, because an importer that emits something almost valid has moved the problem rather
  than solved it.
  

• A documented mapping table from their vocabulary to ours, which is the table at the top of this file plus the attribute list

  [`docs/from-buildkite.md`](../from-buildkite.md), and the importer reads the same table it
  documents - a mapping that is true in the documentation and different in the code is exactly
  the failure this arrangement prevents.
  

• A stated position on plugin compatibility: their plugin interface is hook scripts plus a parameter schema, which is close enough that compatibility is a decision rather than a rewrite. Decide, write it down, and do not leave it implied.

  **The decision is no**, and the reason is one sentence: `BUILDKITE_PLUGIN_*` and their agent's
  lifecycle are an interface, and implementing half of it produces plugins that mostly work. A
  plugin that mostly works is worse than one that does not - it fails on the day its author used
  the half nobody implemented, in somebody else's build, with no error naming the cause.
  
  What to do instead is a three-line list: a plugin that runs a command is a step, a plugin that
  wraps the whole job is a runner hook, and a plugin your organisation wrote is a plugin here.
  The importer names every plugin it finds, so that list is a list of decisions somebody makes
  with the pipeline in front of them.
  

• Test result import so history survives the move, since the flaky verdict is the part that took months to accumulate

  Their export is one JSON object per execution, which is close enough to ours to be a rename and
  a unit conversion - and the conversion is the part that would be silently wrong. Duration is
  seconds there and milliseconds here, so a suite of four-second tests imported as
  four-millisecond ones looks like a suite that got faster. An execution whose result is
  `unknown` is dropped rather than counted as a pass: importing it as one is how a green history
  gets invented.
  

Not copying

Named so they stop being re-proposed, in the manner of the index's deferred list.

• Seat pricing, managed test pricing, and compute minutes. There is no meter. Several Buildkite features exist to make a meter legible and have no purpose here.
• A hosted execution plane, for now. Unchanged from phase 9: it does not begin until the threat model, isolation boundary, secret flow, cache policy, and quotas pass review. Everything in this file is deliberately useful with only self-hosted runners.
• A package registry, for now. Buildkite sells one and it is a real gap against them. It stays on the deferred list in the index until the forge is good, with the standing condition that when it lands, packages:read and packages:write are fine-grained token permissions from the first commit (phase 1). Writing that condition down now is the entire point of naming it here.
• A second receive pipeline. Push triggers consume the push:received event from phase 2, as phase 9 already says.
• A better workflow language. There is a good argument that Actions YAML is not a good format, and it does not matter. It is the format people have, the format the ecosystem targets, and the format a repository can leave with. The typed SDK in the clients section exists for anyone who disagrees, and it emits the same graph. Inventing a third language is how a CI product acquires users it already had and loses the ones it wanted.
• Bug-for-bug fidelity with GitHub. Compatible means a normal workflow runs unchanged, not that every undocumented behavior is reproduced. Where we differ, the conformance report says so and the parser warns. The line between those two is a judgement call, made per construct, written down.