ReviewOS

stacks/uplink

ci: every job runs on ubuntu-latest, so nothing macOS-specific is tested

#38
Open glennmichael123 opened this 8 days ago ·
8 days ago

Part of #6.

Everything that can only work on a Mac is discovered broken at release time, on a Mac, by the one person who can cut a release. The bundle build, the loopback API, launchd, the Keychain and AppleScript all first execute when a maintainer runs uplink:release, so a regression in any of them surfaces after the tag exists rather than on the pull request that caused it.

What is there today

  • All three CI jobs pin the runner to Linux: lint at .github/workflows/ci.yml#L23, typecheck at .github/workflows/ci.yml#L58, test at .github/workflows/ci.yml#L94.
  • The uplink unit tests do run there, through bun run test at .github/workflows/ci.yml#L130, which is ./buddy test (package.json#L65) over the suite root set at bunfig.toml#L31. They cover the handle filter (tests/unit/uplink/messages.test.ts#L3), the chat.db reader (tests/unit/uplink/messages.test.ts#L5), the typedstream decoder (tests/unit/uplink/messages.test.ts#L6), the run queue (tests/unit/uplink/uplink.test.ts#L135) and the engine against a stub claude the test writes itself (tests/unit/uplink/engine.test.ts#L18).
  • Nothing anywhere exercises app/Uplink/desktop-agent.ts: not the single instance lock at app/Uplink/desktop-agent.ts#L52, not the LaunchAgent write at app/Uplink/desktop-agent.ts#L105, not the loopback server at app/Uplink/desktop-agent.ts#L277, not the status route at app/Uplink/desktop-agent.ts#L291.
  • Nothing exercises app/Uplink/service.ts, whose whole job is shelling out to codesign (app/Uplink/service.ts#L120) and launchctl (app/Uplink/service.ts#L136), nor runChecks at app/Uplink/doctor.ts#L84, nor either launcher (app/Desktop/launcher.ts#L46, app/Uplink/launcher.ts#L39).
  • app/Uplink/sender.ts is imported by the suite as a type only (tests/unit/uplink/fixtures.ts#L2), so AppleScriptSender at app/Uplink/sender.ts#L43 and its osascript call at app/Uplink/sender.ts#L46 never run.
  • The bundle build only ever runs by hand, from uplink:release: app/Commands/Uplink.ts#L208 runs buddy build:desktop, and app/Commands/Uplink.ts#L209 runs buddy build:dmg with the signing identity and notary profile.

Proposal

  1. Add a fourth job to .github/workflows/ci.yml on macos-latest, with the same checkout, pantry and bun install --frozen-lockfile steps as the existing test job.
  2. Run the uplink unit tests in that job, so the SQLite, typedstream and engine code is checked against Apple's SQLite and the macOS runner's Bun rather than only Linux.
  3. Run buddy build:desktop as an unsigned smoke check. It needs the Craft runtime declared at deps.yml#L9, so the pantry step has to install rather than pass install: 'false' the way the existing jobs do (.github/workflows/ci.yml#L35), or CRAFT_BIN has to be set the way app/Commands/Uplink.ts#L208 sets it. Assert the result: Contents/MacOS/Uplink exists and is Mach-O, with craft-runtime beside it, which is what craftRuntime() at app/Desktop/launcher.ts#L36 looks for.
  4. Boot the agent and assert the loopback API answers. Add tests/unit/uplink/desktop-agent.test.ts that calls startDesktopAgent (app/Uplink/desktop-agent.ts#L151), fetches /api/status (app/Uplink/desktop-agent.ts#L291), then calls agent.stop(). The server binds port 0 (app/Uplink/desktop-agent.ts#L279), so the port has to come from the returned agent rather than a fixed number.

Requirements

  • No signing secrets. The job has to run on a pull request from a fork with no certificate and no notarytool profile, so an unsigned build:desktop must be enough to fail on a broken bundle. Signing and notarization stay in build:dmg at app/Commands/Uplink.ts#L209 and stay out of CI.
  • Point HOME at a scratch directory. DATA_DIR comes from homedir() (app/Uplink/settings.ts#L16), so does LAUNCH_AGENT (app/Uplink/desktop-agent.ts#L40), and startDesktopAgent creates the data directory on boot (app/Uplink/desktop-agent.ts#L152). Run the boot step with HOME set to a temporary directory so the job cannot leave state in the runner's home or collide with a concurrent run.
  • Assert that the API answers, not that the checks pass. A runner has no Messages database, no Keychain entry and no claude binary, so /api/status will report failing checks. The test asserts a 200 with the expected keys and a clean shutdown through agent.stop().
  • Skip the new test off darwin. The Linux test job runs the whole suite, so the agent test has to be gated on process.platform === 'darwin' rather than break that job.
  • Say what the job still does not cover. setOpenAtLogin and retireOtherInstall return early unless the executable sits inside a bundle (IN_APP_BUNDLE at app/Uplink/desktop-agent.ts#L38, checked at app/Uplink/desktop-agent.ts#L88 and app/Uplink/desktop-agent.ts#L106), so booting from source leaves the LaunchAgent path untested. Either run the built bundle's own executable for that step, or record in the workflow that launchd, codesign and AppleScript stay manual.

Acceptance

  • .github/workflows/ci.yml has a macos-latest job that runs on pull requests without any repository secret.
  • Breaking the app bundle, for example by renaming the launcher entry or dropping a required Info.plist key, fails that job.
  • Breaking the loopback API, for example by changing the /api/status path or its response shape, fails that job.
  • The Linux jobs are unchanged, and the new agent test is skipped there rather than failing.

Sign in to comment on this issue.