also looking at this
fix(deploy): keep SQLite across deploys, and preflight an attached host's services
#174
16 files
+916
-12
| @@ -153,6 +153,36 @@ is often worth making, but it should be a decision rather than a surprise. | ||
| 153 | 153 | token actually reaches, separating the owner's boxes from the ones no one asked |
| 154 | 154 | for, so the radius can be shown before an attach is approved. |
| 155 | 155 | |
| 156 | ### It cannot install services on the owner's box | |
| 157 | ||
| 158 | Attach mode provisions nothing — installing engines on someone else's server is | |
| 159 | not a tenant's business. So `infrastructure.compute.managedServices` in an | |
| 160 | attached config is a statement about what the HOST already runs, not a request | |
| 161 | to install anything: | |
| 162 | ||
| 163 | ```typescript | |
| 164 | cloud: { provider: 'hetzner', attachTo: 'statushq' }, | |
| 165 | infrastructure: { | |
| 166 | appDatabase: { engine: 'postgres', name: 'loghq', username: 'loghq', password: '…' }, | |
| 167 | compute: { managedServices: { postgres: true } }, // statushq must ALREADY run postgres | |
| 168 | } | |
| 169 | ``` | |
| 170 | ||
| 171 | If the owner's box runs SQLite, there is no Postgres for the tenant's role and | |
| 172 | database to be created in, and nothing will ever install one. The deploy checks | |
| 173 | this before it ships anything and stops with the incompatibility named: | |
| 174 | ||
| 175 | ``` | |
| 176 | This project declares managedServices.postgres, but 'statushq' has no postgres | |
| 177 | on 203.0.113.10. Attach mode does not provision services on the owner's box, so | |
| 178 | nothing will install it. Point the app at an external service, or ask the owner | |
| 179 | of 'statushq' to add it to their own config and re-provision. | |
| 180 | ``` | |
| 181 | ||
| 182 | A service counts as present when its binary is on `PATH` or something is | |
| 183 | listening on its port. If the check itself cannot get an answer, the deploy | |
| 184 | proceeds — it is a preflight, not a gate. | |
| 185 | ||
| 156 | 186 | ## Two app models |
| 157 | 187 | |
| 158 | 188 | ts-cloud deploys apps two ways; pick per environment: |
| @@ -344,6 +374,54 @@ clean. If a site targets a server (`deploy: 'server'`, or `start` set) but no | ||
| 344 | 374 | actionable error instead of failing silently at runtime — set `deploy: 'bucket'` |
| 345 | 375 | or add a server. |
| 346 | 376 | |
| 377 | ### State that must survive a deploy | |
| 378 | ||
| 379 | Each deploy unpacks into a NEW `releases/<id>` directory and flips `current` at | |
| 380 | it; old releases are pruned. Anything the app writes and must keep therefore has | |
| 381 | to live in the site's `shared/` directory and be symlinked in, which is what | |
| 382 | `site.sharedPaths` declares. `.env` is always shared. | |
| 383 | ||
| 384 | ```typescript | |
| 385 | sites: { | |
| 386 | app: { | |
| 387 | start: 'bun run server.ts', | |
| 388 | sharedPaths: ['storage', 'public/uploads'], | |
| 389 | }, | |
| 390 | } | |
| 391 | ``` | |
| 392 | ||
| 393 | A **SQLite database is shared automatically**. The deploy already knows the | |
| 394 | connection and the file path from the environment it writes to the box, so when | |
| 395 | `DB_CONNECTION` is `sqlite` and `DB_DATABASE` names a path inside the release, | |
| 396 | that file is added to `sharedPaths` for you and the deploy log says so. Without | |
| 397 | it the database sits inside a release directory and the next deploy starts the | |
| 398 | app on an empty one — silently, with the data still in a release that is about | |
| 399 | to be pruned. | |
| 400 | ||
| 401 | Two cases it does not cover: | |
| 402 | ||
| 403 | - **`DB_DATABASE` unset.** An app can default its own path internally, which the | |
| 404 | deploy never sees. Guessing the filename would report the data as safe while | |
| 405 | sharing a path the app may not use, so the deploy warns instead — set | |
| 406 | `DB_DATABASE`, or list the file in `sharedPaths` yourself. | |
| 407 | - **An absolute path.** A database outside the release tree already survives; a | |
| 408 | deploy replaces the release, not the filesystem around it. | |
| 409 | ||
| 410 | Turning existing on-box state into shared state does not throw it away: the | |
| 411 | first deploy to share a path copies the live release's copy into `shared/` | |
| 412 | (SQLite's `-wal`/`-shm` sidecars included), and a site's first deploy seeds a | |
| 413 | still-empty shared file from the copy the artifact shipped. | |
| 414 | ||
| 415 | Several sites of one project can share ONE file — an app and its API on one | |
| 416 | SQLite database — with the object form, which names an absolute `target`: | |
| 417 | ||
| 418 | ```typescript | |
| 419 | sharedPaths: [{ path: 'database/app.sqlite', target: '/var/www/acme-app/shared/database/app.sqlite', seed: false }] | |
| 420 | ``` | |
| 421 | ||
| 422 | Each site installs under its own base, so a plain string would give each of them | |
| 423 | a database of its own. `seed: false` marks the sites that do not own the file. | |
| 424 | ||
| 347 | 425 | ### CDN / caching |
| 348 | 426 | |
| 349 | 427 | The `cache` hint applies to either origin: |
| @@ -1243,6 +1243,11 @@ export interface SiteConfig { | ||
| 1243 | 1243 | * A release is a fresh directory, so anything the app WRITES and must keep |
| 1244 | 1244 | * has to be listed here or the next deploy silently starts it from empty. |
| 1245 | 1245 | * |
| 1246 | * A SQLite database is the one exception, added for you: when the site's | |
| 1247 | * resolved env says `DB_CONNECTION=sqlite` and `DB_DATABASE` names a path | |
| 1248 | * inside the release, the deploy shares that file without being asked. An | |
| 1249 | * env that says SQLite but not WHERE is warned about rather than guessed at. | |
| 1250 | * | |
| 1246 | 1251 | * An entry may instead be a {@link SharedPathSpec} naming an absolute |
| 1247 | 1252 | * `target`, which is how SEVERAL sites of one project point at ONE file — |
| 1248 | 1253 | * an app and its API sharing a single SQLite database, say. Each site |
| @@ -5,6 +5,7 @@ import { join } from 'node:path' | ||
| 5 | 5 | import { resolveProjectStackName } from '@ts-cloud/core' |
| 6 | 6 | import { CloudFormationClient } from '../../aws/cloudformation' |
| 7 | 7 | import { normalizePublicIpv6 } from '../../deploy/server-dns' |
| 8 | import { summarizeRemoteFailures } from '../shared/remote-failure' | |
| 8 | 9 | import { EC2Client } from '../../aws/ec2' |
| 9 | 10 | import { S3Client } from '../../aws/s3' |
| 10 | 11 | import { SSMClient } from '../../aws/ssm' |
| @@ -463,7 +464,12 @@ export class AwsDriver implements CloudDriver { | ||
| 463 | 464 | success, |
| 464 | 465 | instanceCount: perInstance.length, |
| 465 | 466 | perInstance, |
| 466 | error: success ? undefined : 'One or more SSM command invocations failed', | |
| 467 | // Include each failing invocation's StandardErrorContent — callers report | |
| 468 | // `result.error` alone, so without this the reason is captured and then | |
| 469 | // dropped on the floor. | |
| 470 | error: success | |
| 471 | ? undefined | |
| 472 | : summarizeRemoteFailures(perInstance, 'One or more SSM command invocations failed'), | |
| 467 | 473 | } |
| 468 | 474 | } |
| 469 | 475 | } |
| @@ -22,6 +22,7 @@ import { generateUbuntuAppCloudInit, wrapCloudInitUserData } from './cloud-init' | ||
| 22 | 22 | import { resolveHetznerApiToken, resolveHetznerImage, resolveHetznerSettings } from './config' |
| 23 | 23 | import { buildHetznerFirewallRules } from './firewall-rules' |
| 24 | 24 | import { matchesTsCloudLabels, matchesTsCloudProject, resolveHetznerServerType, TS_CLOUD_LABEL_PREFIX, tsCloudLabels } from './instance-sizes' |
| 25 | import { summarizeRemoteFailures } from '../shared/remote-failure' | |
| 25 | 26 | import { readDriverState, writeDriverState } from './state' |
| 26 | 27 | |
| 27 | 28 | /** Output cap for SCP/SSH children — large enough for verbose tar extraction. */ |
| @@ -1277,7 +1278,11 @@ export class HetznerDriver implements CloudDriver { | ||
| 1277 | 1278 | success, |
| 1278 | 1279 | instanceCount: options.targets.length, |
| 1279 | 1280 | perInstance, |
| 1280 | error: success ? undefined : 'One or more SSH deploy commands failed', | |
| 1281 | // Carry the failing host's remote output into the error itself. Callers | |
| 1282 | // report `result.error` and nothing else, so a bare summary here is the | |
| 1283 | // difference between "the database ensure failed" and "the database | |
| 1284 | // ensure failed because there is no psql on this box". | |
| 1285 | error: success ? undefined : summarizeRemoteFailures(perInstance, 'One or more SSH deploy commands failed'), | |
| 1281 | 1286 | } |
| 1282 | 1287 | } |
| 1283 | 1288 | |
| @@ -912,9 +912,11 @@ describe('deployAllComputeSites attach-mode database ensure', () => { | ||
| 912 | 912 | const { ok, driver } = await deploy(attachConfig('canonical', 'stacks')) |
| 913 | 913 | expect(ok).toBe(true) |
| 914 | 914 | const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls |
| 915 | // One dashboard reconciliation, one ensure, and one site deploy call. | |
| 916 | expect(calls.length).toBe(3) | |
| 917 | const ensure = calls[1][0] | |
| 915 | // One dashboard reconciliation, one managed-services preflight, one ensure, | |
| 916 | // and one site deploy call. | |
| 917 | expect(calls.length).toBe(4) | |
| 918 | expect(calls[1][0].commands.join('\n')).toContain('ts_cloud_probe postgres') | |
| 919 | const ensure = calls[2][0] | |
| 918 | 920 | const sql = ensure.commands.join('\n') |
| 919 | 921 | // Same idempotent script the provisioning path runs at first boot. |
| 920 | 922 | expect(sql).toContain("IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'training')") |
| @@ -927,15 +929,15 @@ describe('deployAllComputeSites attach-mode database ensure', () => { | ||
| 927 | 929 | expect(ensure.comment).toBe('ts-cloud ensure database training/training') |
| 928 | 930 | // The ensure strictly precedes the site deploy so the app's first boot |
| 929 | 931 | // already finds its database. |
| 930 | expect(calls[2][0].commands.join('\n')).toContain('systemctl restart training-web@abc.service') | |
| 932 | expect(calls[3][0].commands.join('\n')).toContain('systemctl restart training-web@abc.service') | |
| 931 | 933 | }) |
| 932 | 934 | |
| 933 | 935 | it('honors the deprecated infrastructure.compute.database alias (bughq shape)', async () => { |
| 934 | 936 | const { ok, driver } = await deploy(attachConfig('legacy', 'stacks')) |
| 935 | 937 | expect(ok).toBe(true) |
| 936 | 938 | const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls |
| 937 | expect(calls.length).toBe(3) | |
| 938 | const sql = calls[1][0].commands.join('\n') | |
| 939 | expect(calls.length).toBe(4) | |
| 940 | const sql = calls[2][0].commands.join('\n') | |
| 939 | 941 | expect(sql).toContain('CREATE ROLE "training" LOGIN PASSWORD \'pw\'') |
| 940 | 942 | expect(sql).toContain('CREATE DATABASE "training" OWNER "training"') |
| 941 | 943 | }) |
| @@ -953,8 +955,10 @@ describe('deployAllComputeSites attach-mode database ensure', () => { | ||
| 953 | 955 | const { ok, driver } = await deploy(attachConfig('none', 'stacks')) |
| 954 | 956 | expect(ok).toBe(true) |
| 955 | 957 | const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls |
| 956 | expect(calls.length).toBe(2) | |
| 957 | expect(calls[0][0].commands.join('\n')).not.toContain('CREATE DATABASE') | |
| 958 | // Dashboard reconciliation, the managed-services preflight, and the site | |
| 959 | // deploy — no ensure, because there is no database to ensure. | |
| 960 | expect(calls.length).toBe(3) | |
| 961 | expect(calls.map((call: any[]) => call[0].commands.join('\n')).join('\n')).not.toContain('CREATE DATABASE') | |
| 958 | 962 | }) |
| 959 | 963 | |
| 960 | 964 | it('fails the deploy (shipping nothing) when the database ensure fails', async () => { |
| @@ -977,3 +981,122 @@ describe('deployAllComputeSites attach-mode database ensure', () => { | ||
| 977 | 981 | expect((driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls.length).toBe(1) |
| 978 | 982 | }) |
| 979 | 983 | }) |
| 984 | ||
| 985 | /** | |
| 986 | * Attach mode rides a box its OWNER provisioned and provisions nothing itself, | |
| 987 | * so `managedServices` here is a claim about the host rather than a request. | |
| 988 | * When the host does not honour it, every later step is built on an engine that | |
| 989 | * is not there — which is how a deploy shipped both releases and then died on | |
| 990 | * "Ensuring database 'loghq' failed" with nothing else to go on. | |
| 991 | */ | |
| 992 | describe('deployAllComputeSites attach-mode service preflight', () => { | |
| 993 | function attachedConfig(): CloudConfig { | |
| 994 | return { | |
| 995 | project: { name: 'Log HQ', slug: 'loghq', region: 'fsn1' }, | |
| 996 | environments: { production: { type: 'production' } }, | |
| 997 | cloud: { provider: 'hetzner', attachTo: 'uptime-status' }, | |
| 998 | sites: { web: { domain: 'loghq.example.com', port: 3000, root: '.output', start: 'bun run server.ts' } }, | |
| 999 | infrastructure: { | |
| 1000 | appDatabase: { engine: 'postgres', name: 'loghq', username: 'loghq', password: 'secret' }, | |
| 1001 | compute: { runtime: 'bun', managedServices: { postgres: true }, proxy: { engine: 'rpx' } }, | |
| 1002 | }, | |
| 1003 | } | |
| 1004 | } | |
| 1005 | ||
| 1006 | function run(driver: CloudDriver, config: CloudConfig, errors: string[]): Promise<boolean> { | |
| 1007 | const tempDir = mkdtempSync(join(tmpdir(), 'ts-cloud-attach-')) | |
| 1008 | const tarball = join(tempDir, 'release.tar.gz') | |
| 1009 | writeFileSync(tarball, 'fake tarball') | |
| 1010 | process.env.TS_CLOUD_UI_DISABLE = '1' | |
| 1011 | return deployAllComputeSites({ | |
| 1012 | config, | |
| 1013 | environment: 'production', | |
| 1014 | driver, | |
| 1015 | sha: 'abc', | |
| 1016 | runtime: 'bun', | |
| 1017 | tarballForSite: () => tarball, | |
| 1018 | logger: { | |
| 1019 | info: () => {}, | |
| 1020 | warn: () => {}, | |
| 1021 | error: (message: string) => errors.push(message), | |
| 1022 | step: () => {}, | |
| 1023 | success: () => {}, | |
| 1024 | }, | |
| 1025 | }).finally(() => { | |
| 1026 | delete process.env.TS_CLOUD_UI_DISABLE | |
| 1027 | rmSync(tempDir, { recursive: true, force: true }) | |
| 1028 | }) | |
| 1029 | } | |
| 1030 | ||
| 1031 | it('refuses before shipping anything when the host has no postgres', async () => { | |
| 1032 | const driver = createMockDriver({ | |
| 1033 | name: 'hetzner', | |
| 1034 | usesCloudFormation: false, | |
| 1035 | runRemoteDeploy: mock(async (options: { commands: string[] }) => ({ | |
| 1036 | success: true, | |
| 1037 | instanceCount: 1, | |
| 1038 | perInstance: [ | |
| 1039 | { | |
| 1040 | instanceId: 'i-abc123', | |
| 1041 | status: 'Success', | |
| 1042 | output: options.commands.join('\n').includes('ts_cloud_probe postgres') | |
| 1043 | ? 'ts-cloud-service:postgres:missing' | |
| 1044 | : '', | |
| 1045 | }, | |
| 1046 | ], | |
| 1047 | })), | |
| 1048 | }) | |
| 1049 | const errors: string[] = [] | |
| 1050 | expect(await run(driver, attachedConfig(), errors)).toBe(false) | |
| 1051 | expect(errors.join('\n')).toContain('managedServices.postgres') | |
| 1052 | expect(errors.join('\n')).toContain("'uptime-status'") | |
| 1053 | // The probe ran; the release upload never did. | |
| 1054 | expect(driver.uploadRelease).not.toHaveBeenCalled() | |
| 1055 | }) | |
| 1056 | ||
| 1057 | it('proceeds when the host provides what was declared', async () => { | |
| 1058 | const driver = createMockDriver({ | |
| 1059 | name: 'hetzner', | |
| 1060 | usesCloudFormation: false, | |
| 1061 | runRemoteDeploy: mock(async () => ({ | |
| 1062 | success: true, | |
| 1063 | instanceCount: 1, | |
| 1064 | perInstance: [{ instanceId: 'i-abc123', status: 'Success', output: 'ts-cloud-service:postgres:present' }], | |
| 1065 | })), | |
| 1066 | }) | |
| 1067 | const errors: string[] = [] | |
| 1068 | expect(await run(driver, attachedConfig(), errors)).toBe(true) | |
| 1069 | expect(errors).toEqual([]) | |
| 1070 | }) | |
| 1071 | ||
| 1072 | /** | |
| 1073 | * A preflight that cannot get an answer must not become a gate: an older box | |
| 1074 | * or a truncated capture would otherwise block deploys that are perfectly | |
| 1075 | * fine. | |
| 1076 | */ | |
| 1077 | it('does not block when the probe returns nothing', async () => { | |
| 1078 | const driver = createMockDriver({ | |
| 1079 | name: 'hetzner', | |
| 1080 | usesCloudFormation: false, | |
| 1081 | runRemoteDeploy: mock(async () => ({ | |
| 1082 | success: true, | |
| 1083 | instanceCount: 1, | |
| 1084 | perInstance: [{ instanceId: 'i-abc123', status: 'Success' }], | |
| 1085 | })), | |
| 1086 | }) | |
| 1087 | const errors: string[] = [] | |
| 1088 | expect(await run(driver, attachedConfig(), errors)).toBe(true) | |
| 1089 | }) | |
| 1090 | ||
| 1091 | it('probes nothing for a project that owns its box', async () => { | |
| 1092 | const config = attachedConfig() | |
| 1093 | delete config.cloud | |
| 1094 | const driver = createMockDriver({ name: 'hetzner', usesCloudFormation: false }) | |
| 1095 | const errors: string[] = [] | |
| 1096 | expect(await run(driver, config, errors)).toBe(true) | |
| 1097 | const commands = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls | |
| 1098 | .map((call: any[]) => call[0].commands.join('\n')) | |
| 1099 | .join('\n') | |
| 1100 | expect(commands).not.toContain('ts_cloud_probe') | |
| 1101 | }) | |
| 1102 | }) | |