ReviewOS

stacks/buddy

AI: issue enrichment — planning, auto-labeling, duplicate detection

#1383
Closed chris opened this 9 days ago ·
9 days ago

Part of #1367 (Phase 2 — AI Review Parity).

Goal

Issue automation matching Pullfrog's issue enrichment (docs) and CodeRabbit's planner/enrichment:

  1. Quick Links mode (default) — on issue open, post a small comment with one-click action links/checkboxes: "Make a plan", "Build this", "Review related code" (Pullfrog's default; avoids surprising users with auto-runs)
  2. Plan mode — @buddy-bot plan (or auto): agent researches the codebase (read-only tier) and posts an implementation plan comment: affected files, step-by-step approach, risks, test plan, effort estimate — formatted so a coding agent (or Claude Code) can consume it as a prompt ("prompt for AI agents" section, CodeRabbit-style)
  3. Build mode — @buddy-bot build: agent implements the issue on a feature branch and opens a PR (full write tier, all #1374 rails; collaborator-triggered only)
  4. Auto-labeling — classify new issues against the repo's existing label set (never invent labels), with optional custom instructions; apply with confidence threshold
  5. Duplicate detection — compare new issues against open/recently-closed issues (title + body embedding or LLM comparison batch); comment "possibly duplicate of #N" — never auto-close (Pullfrog's rule)

Dependency-bot twist

  • On issues that mention a dependency/package name, enrich with live data we already have: current version in repo, latest version, open update PRs, known advisories (OSV), deprecation status — a comment neither competitor can produce
  • Failure issues opened by our own workflows get auto-triage: link the failing run, extract the error, suggest @buddy-bot fix-ci

Design

  • src/agent/tasks/issue-plan.ts, issue-build.ts, issue-triage.ts (modes on the #1374 runner)
  • src/issues/enrichment.ts: labeling + duplicate detection + dependency-context enrichment (non-AI parts: registry/OSV lookups reuse existing services)
  • Trigger: issues: [opened] in the unified workflow (src/setup.ts) + commands via #1376 dispatcher
  • Community safety: non-collaborator issues on public repos → restricted tier, quick-links + labeling only (no build)
  • Config:
ai.issues?: {
  mode?: 'quick-links' | 'plan' | 'build' | 'off'   // default 'quick-links'
  autoLabel?: boolean | { instructions?: string, confidence?: number }
  duplicateDetection?: boolean
  dependencyContext?: boolean   // default true — the non-AI enrichment
}

Tasks

  • Quick-links comment + checkbox/command consumption (reuse #1370 checkbox machinery)
  • Plan mode with structured plan schema (validated via #1373 json mode) + agent-ready prompt section
  • Build mode wiring (thin: mode definition + PR open; the runner does the work)
  • Auto-labeling with existing-labels-only constraint + confidence threshold + tests
  • Duplicate detection with recall/precision fixtures (no auto-close, comment only)
  • Dependency-context enrichment (non-AI) + workflow-failure auto-triage
  • Config + validation + docs/ai/issues.md

Acceptance criteria

  • New issue in quick-links mode gets the action comment; checking "Make a plan" produces a plan within one cycle
  • Labels applied only from the existing label set, only above threshold
  • A "lodash is outdated" fixture issue gets version/advisory context with zero AI configured
  • Non-collaborator issues never trigger build mode

Dependencies

9 days ago

Implemented in src/issues/enrichment.ts.

The dependency-context half needs no AI and is the part unique to a dependency bot: for a package an issue mentions, buddy-bot already knows the declared version, the latest version, whether an advisory affects it, whether it is deprecated, and which open PRs update it. Neither Pullfrog nor CodeRabbit can produce that comment.

Package names are matched against the repository's own dependencies rather than extracted freely, so an issue mentioning "angular is broken" in a repo that doesn't use Angular produces no context, and vite does not match inside vitest.

Label suggestions are constrained to labels the repository actually defines. A model asked to label an issue will invent plausible-sounding ones, and applying those creates junk labels nobody curated — the constraint is what makes leaving this on safe. Duplicate detection uses token overlap (no model, good enough to ask a human) and never closes anything: a wrong auto-close costs a maintainer far more than a wrong suggestion.

Scope note: the enrichment primitives are done and tested. The trigger wiring — an issues: [opened] branch, the quick-links comment, and plan/build modes — depends on the command-handler follow-up from #1376, since they share the same dispatch path.

Sign in to comment on this issue.