ReviewOS

also looking at this

stacks/ts-cloud

fix(deploy): keep SQLite across deploys, and preflight an attached host's services

#174
Merged glennmichael123 wants to merge fix/sqlite-shared-path-and-attach-preflight into main
16 files +916 -12

Review threads live on the whole diff, not on one commit, so none are shown here - a thread's line means something in the branch's final form, and painting it into an intermediate step would put it on code it is not about.

docs/config.mdmodified+30-0
Changes to docs/config.md
@@ -153,6 +153,36 @@ is often worth making, but it should be a decision rather than a surprise.
153153token actually reaches, separating the owner's boxes from the ones no one asked
154154for, so the radius can be shown before an attach is approved.
155155
156### It cannot install services on the owner's box
157
158Attach mode provisions nothing installing engines on someone else's server is
159not a tenant's business. So `infrastructure.compute.managedServices` in an
160attached config is a statement about what the HOST already runs, not a request
161to install anything:
162
163```typescript
164cloud: { provider: 'hetzner', attachTo: 'statushq' },
165infrastructure: {
166 appDatabase: { engine: 'postgres', name: 'loghq', username: 'loghq', password: '…' },
167 compute: { managedServices: { postgres: true } }, // statushq must ALREADY run postgres
168}
169```
170
171If the owner's box runs SQLite, there is no Postgres for the tenant's role and
172database to be created in, and nothing will ever install one. The deploy checks
173this before it ships anything and stops with the incompatibility named:
174
175```
176This project declares managedServices.postgres, but 'statushq' has no postgres
177on 203.0.113.10. Attach mode does not provision services on the owner's box, so
178nothing will install it. Point the app at an external service, or ask the owner
179of 'statushq' to add it to their own config and re-provision.
180```
181
182A service counts as present when its binary is on `PATH` or something is
183listening on its port. If the check itself cannot get an answer, the deploy
184proceeds it is a preflight, not a gate.
185
156186## Two app models
157187
158188ts-cloud deploys apps two ways; pick per environment:
packages/ts-cloud/src/drivers/aws/driver.tsmodified+7-1
Changes to packages/ts-cloud/src/drivers/aws/driver.ts
@@ -5,6 +5,7 @@ import { join } from 'node:path'
55import { resolveProjectStackName } from '@ts-cloud/core'
66import { CloudFormationClient } from '../../aws/cloudformation'
77import { normalizePublicIpv6 } from '../../deploy/server-dns'
8import { summarizeRemoteFailures } from '../shared/remote-failure'
89import { EC2Client } from '../../aws/ec2'
910import { S3Client } from '../../aws/s3'
1011import { SSMClient } from '../../aws/ssm'
@@ -463,7 +464,12 @@ export class AwsDriver implements CloudDriver {
463464 success,
464465 instanceCount: perInstance.length,
465466 perInstance,
466 error: success ? undefined : 'One or more SSM command invocations failed',
467 // Include each failing invocation's StandardErrorContent — callers report
468 // `result.error` alone, so without this the reason is captured and then
469 // dropped on the floor.
470 error: success
471 ? undefined
472 : summarizeRemoteFailures(perInstance, 'One or more SSM command invocations failed'),
467473 }
468474 }
469475}
packages/ts-cloud/src/drivers/hetzner/driver.tsmodified+6-1
Changes to packages/ts-cloud/src/drivers/hetzner/driver.ts
@@ -22,6 +22,7 @@ import { generateUbuntuAppCloudInit, wrapCloudInitUserData } from './cloud-init'
2222import { resolveHetznerApiToken, resolveHetznerImage, resolveHetznerSettings } from './config'
2323import { buildHetznerFirewallRules } from './firewall-rules'
2424import { matchesTsCloudLabels, matchesTsCloudProject, resolveHetznerServerType, TS_CLOUD_LABEL_PREFIX, tsCloudLabels } from './instance-sizes'
25import { summarizeRemoteFailures } from '../shared/remote-failure'
2526import { readDriverState, writeDriverState } from './state'
2627
2728/** Output cap for SCP/SSH children — large enough for verbose tar extraction. */
@@ -1277,7 +1278,11 @@ export class HetznerDriver implements CloudDriver {
12771278 success,
12781279 instanceCount: options.targets.length,
12791280 perInstance,
1280 error: success ? undefined : 'One or more SSH deploy commands failed',
1281 // Carry the failing host's remote output into the error itself. Callers
1282 // report `result.error` and nothing else, so a bare summary here is the
1283 // difference between "the database ensure failed" and "the database
1284 // ensure failed because there is no psql on this box".
1285 error: success ? undefined : summarizeRemoteFailures(perInstance, 'One or more SSH deploy commands failed'),
12811286 }
12821287 }
12831288
packages/ts-cloud/src/drivers/shared/compute-deploy.tsmodified+64-0
Changes to packages/ts-cloud/src/drivers/shared/compute-deploy.ts
@@ -12,6 +12,7 @@ import { buildDatabaseSetupScript, buildManagedDbEnv } from './db-provision'
1212import { buildAwsArtifactFetch, buildHostCleanupScript, buildLocalArtifactFetch, buildSiteDeployScript, buildStaticSiteDeployScript, releaseTarballTmpPath, resolveExecStart } from './deploy-script'
1313import { buildFleetServicesEnv } from './fleet'
1414import { buildHealthCheckScript, buildLaravelDeployScript } from './laravel-deploy'
15import { buildManagedServicesProbeScript, declaredManagedServices, formatMissingManagedServicesError, parseMissingManagedServices } from './managed-services-probe'
1516import { buildNginxVhostScript, resolveNginxSnippet } from './nginx-vhost'
1617import { resolveNotifications, sendNotifications } from './notifications'
1718import { buildPhpFpmPoolScript, phpFpmPoolListen } from './php-fpm-pool'
@@ -493,6 +494,65 @@ async function reconcileManagementDashboardServices(
493494 return true
494495}
495496
497/**
498 * Attach mode preflight: does the owner's box actually provide the on-box
499 * services this project declares?
500 *
501 * Attach mode provisions nothing — that is the whole point of riding someone
502 * else's box — so `managedServices` here is a statement about what the HOST is
503 * expected to already run, not a request to install anything. When the host
504 * does not run it, every step built on top of it is doomed: the database ensure
505 * below talks to an engine that is not there, and the app ships with a
506 * connection string pointing at a closed port.
507 *
508 * Catching that here costs one SSH round-trip and turns an afternoon of
509 * guessing into one sentence. A probe that cannot run (no targets, no output)
510 * never blocks the deploy: this is a preflight, not a gate.
511 *
512 * Returns `false` only when the box positively reported a declared service
513 * missing.
514 */
515async function preflightAttachedHostServices(
516 driver: CloudDriver,
517 options: DeployAllSitesOptions,
518 logger: ComputeDeployLogger,
519): Promise<boolean> {
520 const { config, environment } = options
521 const ownerSlug = config.cloud?.attachTo
522 if (!ownerSlug) return true
523
524 const declared = declaredManagedServices(config.infrastructure?.compute?.managedServices)
525 if (declared.length === 0) return true
526
527 const slug = config.project.slug
528 const stackName = resolveProjectStackName(config, environment)
529 const targets = await driver.findComputeTargets({ slug, environment, role: 'app', stackName })
530 // No targets: the site deploy itself reports that error authoritatively.
531 if (targets.length === 0) return true
532
533 const result = await driver.runRemoteDeploy({
534 targets,
535 commands: buildManagedServicesProbeScript(declared),
536 comment: `ts-cloud preflight managed services ${slug}`,
537 tags: { Project: slug, Environment: environment, Role: 'app' },
538 })
539 if (!result.success) {
540 // The probe itself failing says nothing about the services. Deploy on and
541 // let the real work report its own error.
542 logger.warn(`Could not check '${ownerSlug}' for the declared services: ${result.error || 'unknown error'}`)
543 return true
544 }
545
546 const missing = new Set<string>()
547 for (const instance of result.perInstance) {
548 for (const name of parseMissingManagedServices(instance.output, declared)) missing.add(name)
549 }
550 if (missing.size === 0) return true
551
552 logger.error(formatMissingManagedServicesError([...missing], ownerSlug, targets[0]?.publicIp))
553 return false
554}
555
496556/**
497557 * Attach mode (`cloud.attachTo`): this project rides a box its OWNER provisioned,
498558 * so no cloud-init of ours ever ran the on-box database setup — the tenant role
@@ -653,6 +713,10 @@ export async function deployAllComputeSites(options: DeployAllSitesOptions): Pro
653713 // ran green and the box kept a hand-maintained fragment.
654714 if (deployable.length === 0) return reloadRpxGateway(options)
655715
716 // Attach mode (`cloud.attachTo`): before anything is built ON the owner's
717 // services, confirm the owner's box actually runs them.
718 if (!(await preflightAttachedHostServices(driver, options, logger))) return false
719
656720 // Attach mode (`cloud.attachTo`): the shared box was provisioned by its OWNER,
657721 // so no cloud-init of ours ever ran this project's on-box database setup — the
658722 // tenant role + database would not exist unless someone created them by hand.
packages/ts-cloud/src/drivers/shared/managed-services-probe.test.tsadded+72-0
Changes to packages/ts-cloud/src/drivers/shared/managed-services-probe.test.ts
@@ -0,0 +1,72 @@
1import { describe, expect, it } from 'bun:test'
2import {
3 buildManagedServicesProbeScript,
4 declaredManagedServices,
5 formatMissingManagedServicesError,
6 parseMissingManagedServices,
7} from './managed-services-probe'
8
9describe('declaredManagedServices', () => {
10 it('lists the enabled services, object form included', () => {
11 expect(declaredManagedServices({ postgres: true, redis: { version: '7' }, mysql: false }))
12 .toEqual(['postgres', 'redis'])
13 })
14
15 it('is empty for no config at all', () => {
16 expect(declaredManagedServices(undefined)).toEqual([])
17 expect(declaredManagedServices({})).toEqual([])
18 })
19})
20
21describe('buildManagedServicesProbeScript', () => {
22 it('probes each declared service by binary and by port', () => {
23 const out = buildManagedServicesProbeScript(['postgres', 'redis']).join('\n')
24 expect(out).toContain('ts_cloud_probe postgres 5432 postgres psql')
25 expect(out).toContain('ts_cloud_probe redis 6379 redis-server redis-cli')
26 })
27
28 /**
29 * A missing service is a finding to report, not a remote failure — the probe
30 * must come back with its answer rather than a non-zero exit.
31 */
32 it('always exits 0', () => {
33 expect(buildManagedServicesProbeScript(['postgres']).at(-1)).toBe('exit 0')
34 })
35
36 it('builds nothing when nothing is declared', () => {
37 expect(buildManagedServicesProbeScript([])).toEqual([])
38 })
39})
40
41describe('parseMissingManagedServices', () => {
42 it('reports only what the box said was missing', () => {
43 const output = 'ts-cloud-service:postgres:missing\nts-cloud-service:redis:present\n'
44 expect(parseMissingManagedServices(output, ['postgres', 'redis'])).toEqual(['postgres'])
45 })
46
47 /**
48 * Silence is not evidence of absence: an older box, or a truncated capture,
49 * must not invent a failure and block a deploy that is fine.
50 */
51 it('reports nothing when the probe said nothing', () => {
52 expect(parseMissingManagedServices('', ['postgres'])).toEqual([])
53 expect(parseMissingManagedServices(undefined, ['postgres'])).toEqual([])
54 expect(parseMissingManagedServices('some unrelated output', ['postgres'])).toEqual([])
55 })
56})
57
58describe('formatMissingManagedServicesError', () => {
59 it('names the setting, the owner, the host, and the ways out', () => {
60 const message = formatMissingManagedServicesError(['postgres'], 'uptime-status', '10.0.0.1')
61 expect(message).toContain('managedServices.postgres')
62 expect(message).toContain("'uptime-status'")
63 expect(message).toContain('10.0.0.1')
64 expect(message).toContain('Attach mode does not provision services')
65 })
66
67 it('reads correctly for several services', () => {
68 const message = formatMissingManagedServicesError(['postgres', 'redis'], 'owner', undefined)
69 expect(message).toContain('managedServices.postgres, managedServices.redis')
70 expect(message).toContain('none of postgres, redis')
71 })
72})
packages/ts-cloud/src/drivers/shared/managed-services-probe.tsadded+126-0
Changes to packages/ts-cloud/src/drivers/shared/managed-services-probe.ts
@@ -0,0 +1,126 @@
1/**
2 * Check that a box actually runs the on-box services a project declares.
3 *
4 * This exists for attach mode (`cloud.attachTo`). A project that attaches rides
5 * a box its OWNER provisioned, and attach mode deliberately provisions nothing:
6 * installing engines on someone else's server is not a tenant's business. But
7 * the tenant's config still carries `managedServices`, and ts-cloud used to act
8 * on it regardless — creating a role and database against an engine that was
9 * never there. The result was a deploy that shipped every release and then
10 * failed at the database step, with no statement anywhere that the two settings
11 * are structurally incompatible.
12 *
13 * A declared service is considered present when its binary is on PATH (pantry
14 * puts the engines there) OR something is listening on its port. Either alone
15 * is enough on purpose: a stopped service and an engine installed outside
16 * pantry are both "the owner has this", and a preflight that fails on those
17 * would block deploys it has no business blocking. The case worth catching is
18 * the unambiguous one — no binary, no listener, no engine.
19 */
20import type { ComputeServicesConfig } from '@ts-cloud/core'
21import { pantryEnvActivation } from './package-manager'
22
23/** How to recognize each on-box service the config can declare. */
24const SERVICE_PROBES: Record<string, { port: number; binaries: string[] }> = {
25 // `mysqld`/`mariadbd` are the servers; the clients are listed too because a
26 // box pointing at its own engine always has one.
27 mysql: { port: 3306, binaries: ['mysqld', 'mysql'] },
28 mariadb: { port: 3306, binaries: ['mariadbd', 'mariadb', 'mysqld'] },
29 postgres: { port: 5432, binaries: ['postgres', 'psql'] },
30 redis: { port: 6379, binaries: ['redis-server', 'redis-cli'] },
31 memcached: { port: 11211, binaries: ['memcached'] },
32 meilisearch: { port: 7700, binaries: ['meilisearch'] },
33 // Apps reach Vitess through vtgate, never the tablet's mysqld.
34 vitess: { port: 15306, binaries: ['vtgate'] },
35}
36
37/** Marker prefix the probe prints, one line per service. */
38const MARKER = 'ts-cloud-service:'
39
40function isEnabled(spec: boolean | { version?: string } | object | undefined): boolean {
41 return spec === true || (typeof spec === 'object' && spec != null)
42}
43
44/**
45 * The service names a config declares, in a stable order. Unknown keys are
46 * skipped rather than guessed at: a service this module cannot recognize is one
47 * it cannot honestly report as missing.
48 */
49export function declaredManagedServices(services: ComputeServicesConfig | undefined): string[] {
50 if (!services) return []
51 return Object.keys(SERVICE_PROBES).filter(name => isEnabled((services as Record<string, unknown>)[name] as never))
52}
53
54/**
55 * Shell that reports which of `names` the box provides, one `ts-cloud-service:`
56 * line each. Always exits 0 — a missing service is a finding to report, not a
57 * remote command failure to guess at.
58 */
59export function buildManagedServicesProbeScript(names: readonly string[]): string[] {
60 if (names.length === 0) return []
61 return [
62 // Engines installed by ts-cloud live in the pantry project, not on the
63 // default PATH.
64 pantryEnvActivation(),
65 'ts_cloud_port_open() {',
66 ' if command -v ss >/dev/null 2>&1; then ss -ltn 2>/dev/null | grep -qE "[:.]$1[[:space:]]" && return 0; fi',
67 ' if command -v netstat >/dev/null 2>&1; then netstat -ltn 2>/dev/null | grep -qE "[:.]$1[[:space:]]" && return 0; fi',
68 // Last resort on a box with neither tool: bash can open the socket itself.
69 ' (exec 3<>/dev/tcp/127.0.0.1/$1) 2>/dev/null && { exec 3<&- 3>&-; return 0; }',
70 ' return 1',
71 '}',
72 'ts_cloud_probe() {',
73 ' TS_CLOUD_SVC="$1"; TS_CLOUD_PORT="$2"; shift 2',
74 ' for TS_CLOUD_BIN in "$@"; do',
75 ` if command -v "$TS_CLOUD_BIN" >/dev/null 2>&1; then echo "${MARKER}$TS_CLOUD_SVC:present"; return 0; fi`,
76 ' done',
77 ` if ts_cloud_port_open "$TS_CLOUD_PORT"; then echo "${MARKER}$TS_CLOUD_SVC:present"; return 0; fi`,
78 ` echo "${MARKER}$TS_CLOUD_SVC:missing"`,
79 ' return 0',
80 '}',
81 ...names
82 .filter(name => SERVICE_PROBES[name])
83 .map(name => `ts_cloud_probe ${name} ${SERVICE_PROBES[name].port} ${SERVICE_PROBES[name].binaries.join(' ')}`),
84 'exit 0',
85 ]
86}
87
88/**
89 * The declared services the box reported as missing.
90 *
91 * A service the probe said nothing about is NOT reported missing: silence means
92 * the probe did not run (an older box, a truncated capture), and inventing a
93 * failure from missing evidence would block deploys that are perfectly fine.
94 */
95export function parseMissingManagedServices(output: string | undefined, declared: readonly string[]): string[] {
96 if (!output) return []
97 const status = new Map<string, string>()
98 for (const line of output.split('\n')) {
99 const marker = line.indexOf(MARKER)
100 if (marker === -1) continue
101 const [name, state] = line.slice(marker + MARKER.length).trim().split(':')
102 if (name) status.set(name, state)
103 }
104 return declared.filter(name => status.get(name) === 'missing')
105}
106
107/**
108 * The message an operator gets when the host does not provide what the tenant
109 * declared. Names the setting, the owner, the host, and the two ways out —
110 * because the one thing that is NOT an option is waiting for ts-cloud to
111 * install it.
112 */
113export function formatMissingManagedServicesError(
114 missing: readonly string[],
115 ownerSlug: string,
116 host: string | undefined,
117): string {
118 const where = host ? ` on ${host}` : ''
119 const list = missing.map(name => `managedServices.${name}`).join(', ')
120 const subject = missing.length === 1 ? `no ${missing[0]}` : `none of ${missing.join(', ')}`
121 return (
122 `This project declares ${list}, but '${ownerSlug}' has ${subject}${where}. `
123 + `Attach mode does not provision services on the owner's box, so nothing will install ${missing.length === 1 ? 'it' : 'them'}. `
124 + `Point the app at an external service, or ask the owner of '${ownerSlug}' to add ${missing.length === 1 ? 'it' : 'them'} to their own config and re-provision.`
125 )
126}
packages/ts-cloud/src/drivers/shared/remote-failure.test.tsadded+58-0
Changes to packages/ts-cloud/src/drivers/shared/remote-failure.test.ts
@@ -0,0 +1,58 @@
1import { describe, expect, it } from 'bun:test'
2import { summarizeRemoteFailures } from './remote-failure'
3
4describe('summarizeRemoteFailures', () => {
5 /**
6 * The failure this fixes: "One or more SSH deploy commands failed" was the
7 * ENTIRE error an operator got, while the line explaining it sat unused in
8 * perInstance.
9 */
10 it('appends the failing instance output to the summary', () => {
11 const message = summarizeRemoteFailures(
12 [{ instanceId: 'i-1', status: 'Failed', error: 'Remote SSH command failed (exit 127)\npsql: command not found' }],
13 'One or more SSH deploy commands failed',
14 )
15 expect(message).toContain('One or more SSH deploy commands failed')
16 expect(message).toContain('i-1: Failed')
17 expect(message).toContain('psql: command not found')
18 })
19
20 it('falls back to stdout when nothing was captured on stderr', () => {
21 const message = summarizeRemoteFailures(
22 [{ instanceId: 'i-1', status: 'Failed', output: 'ERROR: role "loghq" cannot be created' }],
23 'failed',
24 )
25 expect(message).toContain('role "loghq" cannot be created')
26 })
27
28 it('ignores the instances that succeeded', () => {
29 const message = summarizeRemoteFailures(
30 [
31 { instanceId: 'i-ok', status: 'Success', output: 'all good' },
32 { instanceId: 'i-bad', status: 'Failed', error: 'boom' },
33 ],
34 'failed',
35 )
36 expect(message).toContain('i-bad')
37 expect(message).not.toContain('i-ok')
38 })
39
40 it('counts the instances it does not quote', () => {
41 const message = summarizeRemoteFailures(
42 Array.from({ length: 5 }, (_, i) => ({ instanceId: `i-${i}`, status: 'Failed', error: 'boom' })),
43 'failed',
44 )
45 expect(message).toContain('(+2 more instances failed)')
46 })
47
48 it('keeps the tail of a long output, where the failure is named', () => {
49 const long = `${'noise\n'.repeat(2000)}FATAL: the actual reason`
50 const message = summarizeRemoteFailures([{ instanceId: 'i-1', status: 'Failed', error: long }], 'failed')
51 expect(message).toContain('FATAL: the actual reason')
52 expect(message.length).toBeLessThan(long.length)
53 })
54
55 it('returns the bare summary when nothing is marked failed', () => {
56 expect(summarizeRemoteFailures([], 'failed')).toBe('failed')
57 })
58})
packages/ts-cloud/src/drivers/shared/remote-failure.tsadded+54-0
Changes to packages/ts-cloud/src/drivers/shared/remote-failure.ts
@@ -0,0 +1,54 @@
1/**
2 * Turn per-instance remote-execution results into ONE operator-facing error.
3 *
4 * Both drivers used to collapse a failed remote run to a fixed sentence —
5 * "One or more SSH deploy commands failed" — while the thing that explains the
6 * failure (the remote command's stderr) sat right there in `perInstance`. The
7 * operator was left with a deploy that had failed for no stated reason, and no
8 * flag, not even `--verbose`, would print it: nothing had kept it.
9 *
10 * The remote output is already redacted where it is captured (the deploy script
11 * embeds a here-document with the full runtime environment; see
12 * `formatSshFailure`), so this only has to decide what is worth showing.
13 */
14import type { RemoteDeployInstanceResult } from '@ts-cloud/core'
15
16/** Longest remote output carried into the aggregate error, per instance. */
17const PER_INSTANCE_LIMIT = 4_000
18
19/** Instances beyond this are summarized as a count rather than quoted. */
20const MAX_QUOTED_INSTANCES = 3
21
22function clip(value: string): string {
23 // Keep the TAIL: a failing script's last lines are the ones that name the
24 // failure, while the head is setup noise.
25 return value.length > PER_INSTANCE_LIMIT ? `…\n${value.slice(-PER_INSTANCE_LIMIT)}` : value
26}
27
28/**
29 * Build the `error` for a {@link import('@ts-cloud/core').RemoteDeployResult}
30 * whose run did not fully succeed. `summary` is the one-line what-failed (e.g.
31 * "One or more SSH deploy commands failed"); the detail of each failing
32 * instance is appended beneath it.
33 */
34export function summarizeRemoteFailures(
35 perInstance: readonly RemoteDeployInstanceResult[],
36 summary: string,
37): string {
38 const failed = perInstance.filter(item => item.status !== 'Success')
39 if (failed.length === 0) return summary
40
41 const quoted = failed.slice(0, MAX_QUOTED_INSTANCES).map((item) => {
42 // Prefer the captured error (stderr + exit status); fall back to stdout,
43 // which is where a script that reports its own failure and exits non-zero
44 // without writing to stderr leaves the explanation.
45 const detail = (item.error || item.output || '').trim()
46 const head = `${item.instanceId}: ${item.status}`
47 return detail ? `${head}\n${clip(detail)}` : head
48 })
49
50 const remaining = failed.length - quoted.length
51 const more = remaining > 0 ? [`(+${remaining} more instance${remaining === 1 ? '' : 's'} failed)`] : []
52
53 return [summary, ...quoted, ...more].join('\n')
54}
packages/ts-cloud/test/drivers/compute-deploy.test.tsmodified+131-8
Changes to packages/ts-cloud/test/drivers/compute-deploy.test.ts
@@ -912,9 +912,11 @@ describe('deployAllComputeSites attach-mode database ensure', () => {
912912 const { ok, driver } = await deploy(attachConfig('canonical', 'stacks'))
913913 expect(ok).toBe(true)
914914 const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls
915 // One dashboard reconciliation, one ensure, and one site deploy call.
916 expect(calls.length).toBe(3)
917 const ensure = calls[1][0]
915 // One dashboard reconciliation, one managed-services preflight, one ensure,
916 // and one site deploy call.
917 expect(calls.length).toBe(4)
918 expect(calls[1][0].commands.join('\n')).toContain('ts_cloud_probe postgres')
919 const ensure = calls[2][0]
918920 const sql = ensure.commands.join('\n')
919921 // Same idempotent script the provisioning path runs at first boot.
920922 expect(sql).toContain("IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'training')")
@@ -927,15 +929,15 @@ describe('deployAllComputeSites attach-mode database ensure', () => {
927929 expect(ensure.comment).toBe('ts-cloud ensure database training/training')
928930 // The ensure strictly precedes the site deploy so the app's first boot
929931 // already finds its database.
930 expect(calls[2][0].commands.join('\n')).toContain('systemctl restart training-web@abc.service')
932 expect(calls[3][0].commands.join('\n')).toContain('systemctl restart training-web@abc.service')
931933 })
932934
933935 it('honors the deprecated infrastructure.compute.database alias (bughq shape)', async () => {
934936 const { ok, driver } = await deploy(attachConfig('legacy', 'stacks'))
935937 expect(ok).toBe(true)
936938 const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls
937 expect(calls.length).toBe(3)
938 const sql = calls[1][0].commands.join('\n')
939 expect(calls.length).toBe(4)
940 const sql = calls[2][0].commands.join('\n')
939941 expect(sql).toContain('CREATE ROLE "training" LOGIN PASSWORD \'pw\'')
940942 expect(sql).toContain('CREATE DATABASE "training" OWNER "training"')
941943 })
@@ -953,8 +955,10 @@ describe('deployAllComputeSites attach-mode database ensure', () => {
953955 const { ok, driver } = await deploy(attachConfig('none', 'stacks'))
954956 expect(ok).toBe(true)
955957 const calls = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls
956 expect(calls.length).toBe(2)
957 expect(calls[0][0].commands.join('\n')).not.toContain('CREATE DATABASE')
958 // Dashboard reconciliation, the managed-services preflight, and the site
959 // deploy — no ensure, because there is no database to ensure.
960 expect(calls.length).toBe(3)
961 expect(calls.map((call: any[]) => call[0].commands.join('\n')).join('\n')).not.toContain('CREATE DATABASE')
958962 })
959963
960964 it('fails the deploy (shipping nothing) when the database ensure fails', async () => {
@@ -977,3 +981,122 @@ describe('deployAllComputeSites attach-mode database ensure', () => {
977981 expect((driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls.length).toBe(1)
978982 })
979983})
984
985/**
986 * Attach mode rides a box its OWNER provisioned and provisions nothing itself,
987 * so `managedServices` here is a claim about the host rather than a request.
988 * When the host does not honour it, every later step is built on an engine that
989 * is not there — which is how a deploy shipped both releases and then died on
990 * "Ensuring database 'loghq' failed" with nothing else to go on.
991 */
992describe('deployAllComputeSites attach-mode service preflight', () => {
993 function attachedConfig(): CloudConfig {
994 return {
995 project: { name: 'Log HQ', slug: 'loghq', region: 'fsn1' },
996 environments: { production: { type: 'production' } },
997 cloud: { provider: 'hetzner', attachTo: 'uptime-status' },
998 sites: { web: { domain: 'loghq.example.com', port: 3000, root: '.output', start: 'bun run server.ts' } },
999 infrastructure: {
1000 appDatabase: { engine: 'postgres', name: 'loghq', username: 'loghq', password: 'secret' },
1001 compute: { runtime: 'bun', managedServices: { postgres: true }, proxy: { engine: 'rpx' } },
1002 },
1003 }
1004 }
1005
1006 function run(driver: CloudDriver, config: CloudConfig, errors: string[]): Promise<boolean> {
1007 const tempDir = mkdtempSync(join(tmpdir(), 'ts-cloud-attach-'))
1008 const tarball = join(tempDir, 'release.tar.gz')
1009 writeFileSync(tarball, 'fake tarball')
1010 process.env.TS_CLOUD_UI_DISABLE = '1'
1011 return deployAllComputeSites({
1012 config,
1013 environment: 'production',
1014 driver,
1015 sha: 'abc',
1016 runtime: 'bun',
1017 tarballForSite: () => tarball,
1018 logger: {
1019 info: () => {},
1020 warn: () => {},
1021 error: (message: string) => errors.push(message),
1022 step: () => {},
1023 success: () => {},
1024 },
1025 }).finally(() => {
1026 delete process.env.TS_CLOUD_UI_DISABLE
1027 rmSync(tempDir, { recursive: true, force: true })
1028 })
1029 }
1030
1031 it('refuses before shipping anything when the host has no postgres', async () => {
1032 const driver = createMockDriver({
1033 name: 'hetzner',
1034 usesCloudFormation: false,
1035 runRemoteDeploy: mock(async (options: { commands: string[] }) => ({
1036 success: true,
1037 instanceCount: 1,
1038 perInstance: [
1039 {
1040 instanceId: 'i-abc123',
1041 status: 'Success',
1042 output: options.commands.join('\n').includes('ts_cloud_probe postgres')
1043 ? 'ts-cloud-service:postgres:missing'
1044 : '',
1045 },
1046 ],
1047 })),
1048 })
1049 const errors: string[] = []
1050 expect(await run(driver, attachedConfig(), errors)).toBe(false)
1051 expect(errors.join('\n')).toContain('managedServices.postgres')
1052 expect(errors.join('\n')).toContain("'uptime-status'")
1053 // The probe ran; the release upload never did.
1054 expect(driver.uploadRelease).not.toHaveBeenCalled()
1055 })
1056
1057 it('proceeds when the host provides what was declared', async () => {
1058 const driver = createMockDriver({
1059 name: 'hetzner',
1060 usesCloudFormation: false,
1061 runRemoteDeploy: mock(async () => ({
1062 success: true,
1063 instanceCount: 1,
1064 perInstance: [{ instanceId: 'i-abc123', status: 'Success', output: 'ts-cloud-service:postgres:present' }],
1065 })),
1066 })
1067 const errors: string[] = []
1068 expect(await run(driver, attachedConfig(), errors)).toBe(true)
1069 expect(errors).toEqual([])
1070 })
1071
1072 /**
1073 * A preflight that cannot get an answer must not become a gate: an older box
1074 * or a truncated capture would otherwise block deploys that are perfectly
1075 * fine.
1076 */
1077 it('does not block when the probe returns nothing', async () => {
1078 const driver = createMockDriver({
1079 name: 'hetzner',
1080 usesCloudFormation: false,
1081 runRemoteDeploy: mock(async () => ({
1082 success: true,
1083 instanceCount: 1,
1084 perInstance: [{ instanceId: 'i-abc123', status: 'Success' }],
1085 })),
1086 })
1087 const errors: string[] = []
1088 expect(await run(driver, attachedConfig(), errors)).toBe(true)
1089 })
1090
1091 it('probes nothing for a project that owns its box', async () => {
1092 const config = attachedConfig()
1093 delete config.cloud
1094 const driver = createMockDriver({ name: 'hetzner', usesCloudFormation: false })
1095 const errors: string[] = []
1096 expect(await run(driver, config, errors)).toBe(true)
1097 const commands = (driver.runRemoteDeploy as ReturnType<typeof mock>).mock.calls
1098 .map((call: any[]) => call[0].commands.join('\n'))
1099 .join('\n')
1100 expect(commands).not.toContain('ts_cloud_probe')
1101 })
1102})