Host cleanup in ts-cloud works, but it only exists as a side effect of deploying. #194 is the narrow bug underneath this issue — the artifact cache is missing from buildHostCleanupScript() — and it should be merged on its own. This issue is the gap that remains once it is.
buildHostCleanupScript() (packages/ts-cloud/src/drivers/shared/deploy-script.ts:760) is invoked from exactly one place: compute-deploy.ts:325, at the tail of a successful deploy. Everything else about host disk retention follows from that one fact.
The gaps
1. Cleanup frequency is tied to deploy frequency, not to time or pressure
A box that stops deploying never cleans again, however full it gets. A box that deploys ten times a day cleans ten times a day, whether or not it needs to. Neither is what you want: retention should be a property of the host, not of how busy its CI happens to be. The rules themselves are already time-bounded (-mtime +7, --vacuum-time=14d), so they are safe to run on a schedule — nothing about them assumes a deploy just finished.
A failed deploy also skips cleanup entirely, which is backwards: a run that died partway through is the one most likely to have left something behind.
2. Nothing reacts to the disk actually filling
buildHostCleanupScript() echoes df -h / before and after, and that is the extent of its awareness. The windows are fixed constants regardless of whether the root filesystem is at 20% or 95%. There is no threshold below which it could skip work, and no escalation above which it prunes harder.
The box in #194 sat at 84% and nothing anywhere raised a hand. It was found by looking.
3. Retention windows are hardcoded
keepReleases is already per-site configurable (SiteDeployOptions.keepReleases, deploy-script.ts:250, defaulting to DEFAULT_KEEP_RELEASES = 4 in releases.ts:21) — that part is fine. Everything in buildHostCleanupScript() is not: the 60-minute staging bound, the 7-day Bun cache window, the 14d/512M journal vacuum, the 168h image prune are all literals in the returned strings, as is the artifact window #194 proposes to add.
140 sites on a 150 GB disk is not the same problem as three sites on the same disk, and there is currently no way to say so.
4. The path list is closed
buildHostCleanupScript() knows about the paths hardcoded into it. Anything else on the box is invisible to it, and there is no way for another component to register its own retention rule.
Concretely, on the same box as #194: /root/rpx-backups at 3.0 GB, plus cert-backup-vga-*, unit-backups-* and rpx-rollback-* accumulating in /root. Same class of artifact — timestamped, written by tooling, pruned by nobody — but written by a component that has no way to participate in the cleanup that ts-cloud already runs.
Sketch
Roughly in dependency order; each is independently useful.
- A systemd timer running the same script. The cleanup command already exists as a pure
string[]; installing it as a dailyts-cloud-cleanup.timeralongside the existing unit management decouples it from deploys without duplicating any rules. The deploy-time invocation can stay — it is cheap and it is useful right after a deploy. - Disk-pressure awareness. Read
dffirst. Under a floor, do the cheap rules only. Over a ceiling, escalate: shorten the artifact and cache windows, then reduce effectivekeepReleases. This turns cleanup from a fixed routine into something that responds to the condition it exists to prevent. - Configurable retention. Lift the windows out of the literals into a typed options object with the current values as defaults, so an operator can trade rollback depth and cache warmth against disk on a box where that trade is real.
- An extensible path registry. Let a component contribute
{ path, pattern, maxAge }rules so things like rpx's backups are pruned by the same mechanism, on the same schedule, instead of leaking into/root. - Reporting. The before/after
dfoutput already exists and goes nowhere anyone reads. Surfacing it — deploy output, driver state, an alert past a threshold — is what turns "found at 84% by looking" into something noticed months earlier.
Scope
#194 is deliberately not blocked on any of this: it is a two-line fix to an existing function and should land first. This issue is about the mechanism around that function, not the rules inside it.