ReviewOS

stacks/system-cleaner

Add pattern-based project cache scanning: 21 GB of build output is unreachable by fixed-path targets

#27
Closed glennmichael123 opened this 8 days ago ·
8 days ago

All 260 entries in packages/clean/src/categories.ts are fixed absolute paths. That covers global, per-tool caches well — but the largest reclaimable category on a developer machine is per-project build output scattered across the filesystem, and a static path list structurally cannot express it.

The code already knows this. categories.ts:176:

// Zig keeps a global cache next to the local `.zig-cache` each project owns.
// The XDG path is used by zig itself, the Library path by Homebrew builds.
t('zig-global-cache', 'Zig Global Cache', `${HOME}/.cache/zig`, ...)

The comment names the per-project caches, then only the global one gets a target — because there is no way to express the other.

Evidence

Measured on a working developer Mac. What the fixed-path targets cover:

TargetSize
~/Library/Caches864M
~/.bun/install/cache991M
~/Library/Developer/Xcode/DerivedData91M
~/.cache/zig (global)114M
Total~1.9 GB

What it cannot see:

PatternDirsSize
.zig-cache3213.1G
node_modules187.4G
zig-out50.5G
build20.3G
dist60.1G
Total~21.4 GB

The uncovered category is roughly 11× the covered one. The Zig numbers make the point sharpest: the global cache this tool cleans was 114 MB, while the 32 per-project caches it cannot see held 13.1 GB — a 115× difference on the same toolchain.

(node_modules was counted at -maxdepth 3, so the true figure is higher — nested workspace packages weren't included.)

Proposed: a second scanner type

Not more entries — a different kind of target. Alongside the fixed-path list, a pattern scanner with:

  • Configurable roots, defaulting to likely code locations (~/Documents, ~/Code, ~/Projects, ~/src, ~/dev) and user-editable. Never scan $HOME wholesale.
  • A depth limit, so a monorepo with thousands of nested matches doesn't take minutes to scan.
  • Named patterns with per-pattern risk levels — .zig-cache, zig-out, target, .next, .turbo, .parcel-cache, dist, build, __pycache__, .gradle.
  • An age threshold, so a project built this morning isn't offered up.
  • Results grouped by project, not a flat list of 32 paths. "home — 24 caches, 8.2 GB" is actionable; a wall of paths isn't.

Risk tiers matter here

These are not equally safe, and the UI should say so:

  • Safe — .zig-cache, zig-out, target, .next, .turbo, .parcel-cache, __pycache__. Pure build output, regenerated automatically, no network needed.
  • Careful — dist, build. Usually build output, but some projects commit them or ship from them. Check for a matching entry in .gitignore before offering.
  • Separate feature — node_modules. Deleting it requires a reinstall that needs network and can fail against a changed registry. Worth its own treatment rather than inclusion in a routine sweep; tracked separately.

Worth surfacing

Rebuild cost is the only real downside, and it should be stated. Clearing a 3.6 GB craft/packages/zig/.zig-cache means the next build is a cold compile measured in minutes. Showing "frees 3.6 GB · next build will be slower" lets the user make that trade knowingly.

Keeping the global cache (~/.cache/zig) while clearing project-local ones is the right default for exactly this reason — the global cache makes the subsequent rebuilds faster, and it's two orders of magnitude smaller.

8 days ago

Implemented in 5074829 (pushed straight to main).

packages/clean/src/project-caches.ts adds the second scanner type: 14 named patterns found by walking a small set of code roots — never $HOME — with a depth limit, an age floor, per-pattern risk, and results grouped by the outermost project. Exposed as /project-caches and /clean-project-caches, the latter validating requested paths against a fresh scan the way /clean-orphans does, since these paths sit inside the user's source tree.

Three things the issue didn't anticipate, all surfaced by running it against a real machine rather than a fixture:

Vendored dependencies. A dist inside pantry/pako is that package's shipped artifact — no build script in the host project regenerates it, and deleting it breaks the app. The .gitignore gate cannot catch this on its own, because it reads the project root's ignore file where dist is listed, so every vendored dist below inherits the pass. One repository contributed 291 of them. Vendor directories (pantry, vendor, bower_components, Pods, .venv, …) are now never descended into: 344 → 45 entries, and the scan got ~15s faster.

Double counting. A checkout under ~/Documents reachable through ~/Code was reported twice — same inode, two paths — promising 463 MB that deleting the first copy already freed. Entries are now deduplicated by resolved real path.

Ambiguous names. dist, build and target are build output only when the project ignores them; tracked, they may be the artifact it ships. All three are gated on a matching .gitignore entry and marked caution, so Select All leaves them alone.

tests/project-caches.test.ts adds 11 tests covering the pattern table, vendor exclusion, the gitignore gate, the age floor, symlink dedupe, depth limiting and the $HOME guard. Full suite 526 pass / 0 fail, typecheck clean, lint 0 errors — including size-cache-invalidation.test.ts, which correctly caught that the new delete route wasn't calling invalidateSizeCaches().

Measured on the development machine: 38 caches across 16 projects, 3.5 GB, ~20s. Below the 21 GB quoted above only because those .zig-cache directories had since been deleted by hand — the figure rebuilds with normal use.

node_modules remains out of scope here, tracked in #28.

Sign in to comment on this issue.