ReviewOS

stacks/buddy

AI: finishing touches — docstrings, unit tests, autofix, simplify

#1380
Closed chris opened this 8 days ago ·
8 days ago

Part of #1367 (Phase 2 — AI Review Parity).

Goal

CodeRabbit's "Finishing Touches" (docs) as agent-runtime tasks — on-demand agentic actions on a PR, each invocable via a @buddy-bot command (#1376) and optionally offered as one-click checkboxes under the review summary:

  1. generate docstrings — add missing doc comments for changed public APIs (JSDoc per repo convention; respect CLAUDE.md JSDoc style). Commit to the PR branch or a stacked PR (config)
  2. generate unit tests — generate bun:test tests for changed functions, following the repo's test/ structure and describe/it conventions; verified by actually running them in the sandbox before committing (agent stopScript-style loop from #1374)
  3. autofix — apply all committable suggestions from the current review (and fixable linter findings from #1378) as one commit
  4. simplify — refactor pass on changed code for clarity without behavior change; must pass existing tests before committing
  5. Custom finishing touches — config-defined named prompts (CodeRabbit caps at 5 recipes / 10K chars; we can be more generous): ai.finishingTouches.custom: { name, instructions, tools?: 'read-only' | 'write' }[]

Design

  • src/agent/tasks/ — one module per task (docstrings.ts, unit-tests.ts, autofix.ts, simplify.ts, custom.ts), each a mode definition (playbook + tool tier) for the #1374 runner
  • Output modes per task: commit (push to PR branch — requires the PR author to be the bot or explicit maintainer invocation), stacked-pr (branch off the PR branch, open PR targeting it — safest default for human PRs), or suggestion (post as committable suggestions only)
  • Verification gates: unit-test task runs the generated tests; simplify/autofix run the repo's test command (from learnings #1379 or package.json scripts) before pushing; failures → post partial result as suggestions instead
  • Checkbox offering: review summary (#1375) appends an offer section with <!-- buddy-bot:touch=<task> --> markers; checked boxes handled by the same comment/edited machinery as #1370

Config

ai.finishingTouches?: {
  docstrings?: boolean | { output?: 'commit' | 'stacked-pr' | 'suggestion' }
  unitTests?: boolean | { output?: ..., pathInstructions?: {...}[] }
  autofix?: boolean
  simplify?: boolean
  custom?: { name: string, instructions: string, tools?: 'read-only' | 'write' }[]
}

Tasks

  • Task modules + mode/permission definitions; command wiring in #1376 dispatcher
  • Stacked-PR mechanics in src/git/github-provider.ts (branch-off-branch, PR targeting PR head branch)
  • Verification gates with sandboxed test runs + partial-result fallback
  • Checkbox offer/consume flow
  • Per-task fixtures: repo with missing docstrings, uncovered function, failing lint, convoluted function
  • docs/ai/finishing-touches.md

Acceptance criteria

  • Each of the four built-in tasks produces a correct commit/stacked-PR/suggestion in fixture repos
  • Generated tests actually run green before being committed; red generations degrade to suggestions
  • Custom recipes execute with their declared tool tier only

Dependencies

  • Requires #1374 (runtime), #1375 (review offers), #1376 (commands); autofix consumes #1378 linter findings
8 days ago

Implemented in src/agent/tasks.ts.

Docstrings, unit tests, autofix, simplify and plan as named agent tasks, each pairing a playbook with the capability tiers it may use, offered through the same checkbox markers the dashboard already uses (#1370) so ticking one is handled by machinery that exists.

Two properties worth recording. The three tasks that change code require verification before reporting success — a change that looks right and breaks the build costs more than no change at all, and a generated test that does not pass is worse than none. And the plan task runs in review mode, which has no write tier, so it structurally cannot modify anything rather than merely being told not to.

Scope note: the task definitions, the offer/selection mechanism and the output-destination model are done. Not implemented: stacked-PR mechanics on the provider (branch-off-branch plus a PR targeting the PR head) and the sandboxed verification gate that decides between committing and degrading to suggestions. Both are provider and runner plumbing against the shapes defined here.

Sign in to comment on this issue.