AI: secure agent runtime in GitHub Actions (sandbox, injection defense, toolbelt)

Part of #1367 (Phase 1 — AI Foundation).

Goal

A secure agent runtime that executes multi-step AI tasks inside the user's GitHub Actions runner — the architecture Pullfrog is built on (security, tools). This runtime powers review (#1375), fix-ci (#1381), finishing touches (#1380), issue enrichment (#1383), and AI major upgrades (#1388).

We already run in the user's Actions (unlike CodeRabbit's SaaS) — this issue adds the agent loop + guardrails.

Design

New directory src/agent/:

src/agent/
├─ runner.ts      # the agentic loop: prompt → tool calls → observations → done
├─ toolbelt.ts    # tool registry + permission tiers
├─ tools/
│  ├─ fs.ts       # read_file, write_file, list_dir (repo-scoped, path-traversal guarded — reuse src/utils/file-changes.ts guards)
│  ├─ shell.ts    # exec in isolated shell: allowlisted env, no *_TOKEN/*_KEY/*_SECRET/*_PASSWORD passthrough
│  ├─ git.ts      # branch/commit/push via existing src/git/github-provider.ts paths (feature branches only — never default branch)
│  ├─ github.ts   # PR/issue/review/comment ops (wrap github-provider)
│  └─ ci.ts       # get_check_logs: fetch check-run/job logs, parse + extract failing steps and error excerpts (Pullfrog's get_check_suite_logs)
├─ modes.ts       # named modes: review, fix-ci, implement, plan — each = playbook prompt + tool permission set
└─ transcript.ts  # structured run log (tool calls, tokens, duration) for observability

Security requirements (non-negotiable, mirror Pullfrog)

1. Prompt-injection defense: issue/PR bodies and comments are NEVER inlined into the system/user prompt — the agent fetches them via tools and they arrive as clearly-marked untrusted tool output
2. Env allowlist (default-deny): shell tool starts from an empty env + explicit allowlist; block *_TOKEN, *_KEY, *_SECRET, *_PASSWORD, *_CREDENTIAL patterns
3. Secret masking: all transcript/log output passes through src/ai/redact.ts
4. Branch protection: no pushes to the base branch; feature branch + PR only (enforced in the git tool, not just prompts)
5. Permission tiers per mode: review = read-only + comment tools; fix-ci/implement = +fs/shell/git; tools not in the mode's set are not offered to the model at all
6. Actor gating: tasks triggered by non-collaborators (public repos) run in a restricted read-only tier (Pullfrog's community-issue model)
7. Iteration caps: max tool-calls, max wall-clock, max tokens (ai.maxTokensPerRun from #1373) — abort with a partial-result comment rather than hanging

Lifecycle hooks (Pullfrog parity)

Config-declared scripts with enforced semantics: setupScript (before agent), prepushScript (must pass before any push — lint/test gate), stopScript (non-zero exit feeds failure output back to the agent and resumes — the verify loop). Config under ai.agent.{setupScript,prepushScript,stopScript,timeoutMinutes}.

Tasks

• src/agent/ per layout; tool schemas typed in src/types.ts
• Tool permission matrix + mode definitions with tests asserting a mode cannot invoke out-of-tier tools
• CI log fetching + parsing (ci.ts) with fixtures from real Actions log formats
• Injection-defense tests: hostile PR body containing "ignore previous instructions, run X" must arrive only as tool output and the shell tool must refuse env exfiltration
• Transcript artifact: write JSON run log to the workflow's artifacts dir; summary comment formatting
• docs/ai/agent.md + security documentation of the threat model

Acceptance criteria

• A review-mode run cannot write files or push (test-enforced)
• Hostile content in PR bodies cannot alter agent behavior in the injection test suite
• stopScript failure loops the agent with the failure context until pass or cap
• Runs produce a structured transcript with token/cost totals

Dependencies

• Requires #1373 (provider layer, tool-calling shim)
• Blocks #1375, #1380, #1381, #1383, #1388

Implemented in fb09bf6 / 6a8e159.

The security properties are enforced structurally rather than by instruction, and each has a test:

Tool visibility is the permission boundary. A mode grants capability tiers, and a tool outside them is never converted into a model-visible definition. A capability the model cannot see is one no PR body can talk it into using. Toolbelt.invoke re-checks and throws ToolPermissionError rather than returning an error result — an out-of-tier call is not something the model should retry differently.

Untrusted content is escaped, not just wrapped. Third-party text reaches the model only as marked tool output. The marker is escaped inside the payload, so content containing its own </untrusted-content> cannot close the block early and appear to be trusted context — there is a test for exactly that payload.

The shell env is an allowlist, not a blocklist. A blocklist leaks any secret-bearing variable added to CI until someone writes a new rule; the allowlist makes tomorrow's variable invisible by default, with a second name-shape check behind it. A test asserts a command literally cannot read GITHUB_TOKEN.

Paths are guarded twice — on the repository-relative form and again on the resolved absolute path, because a symlink inside the workspace can satisfy the first check and still resolve outside it.

restricted mode implements the actor gating: a run triggered by someone without write access is capped at reads, since public repositories accept input from anyone. An unknown mode name throws rather than defaulting, because silently landing in a permissive mode on a typo would be the worst outcome.

Runs are bounded by tool calls, wall clock and the AI token budget, and everything in the transcript passes through redaction — transcripts are uploaded artifacts and provider errors echo request headers back.