Roadmap relationship
Outcome
Schedule builds and deployable workloads across labeled capacity pools with explicit placement, resource reservations, draining, affinity constraints, health-aware rescheduling, and provider-native scaling where appropriate.
Current gap
Compute configuration and deployments primarily target one resolved box or a provider-specific service. There is no generic fleet capacity model, build-runner pool, placement policy, resource reservation, drain/move workflow, or multi-node view. Large builds and high-availability apps therefore compete with production workloads or require manual distribution.
User experience
- Define pools for application, build, worker, monitoring, and backup roles using servers/providers, labels, regions, architectures, GPU capability, and capacity limits.
- Placement preview explains eligible/ineligible nodes, constraints, estimated resource fit, failure-domain spread, and cost drivers.
- Fleet capacity dashboards show allocated/reserved/free CPU, memory, disk, build slots, pending work, hotspots, and unavailable nodes.
- Drain/move/rebalance workflows enumerate workloads, expected downtime, volume/network constraints, and progress.
- Remote builds show runner, cache hit, resource use, artifact digest, and teardown.
Scope
- Add pool, node membership, resource capacity/reservation, placement constraint/preference, workload replica, build runner, and scheduling-decision models.
- Support manual pinning plus automatic spread/bin-pack policies with anti-affinity, architecture, region/zone, role, labels/taints, and resource requirements.
- Implement remote isolated builders with concurrency, ephemeral workspace, layer/dependency cache, artifact upload, cleanup, and cancellation.
- Integrate provider-native ECS/ASG capabilities and server-box pools without forcing one scheduler implementation on all targets.
- Add drain/uncordon, move/redeploy, failure reconciliation, and optional safe auto-reschedule.
Technical direction
- Keep scheduling decisions deterministic, persisted, explainable, and replayable from inputs.
- Use leases/reservations so concurrent jobs cannot overcommit the same capacity.
- Separate stateless rescheduling from stateful-volume placement and require explicit storage strategy.
- Sign/verify remote build artifacts and use content-addressed cache keys that exclude secrets.
Safety and operability
- Builders receive least-privilege, short-lived credentials and cannot access production runtime secrets unless explicitly required.
- Untrusted builds run isolated with CPU/memory/disk/time/network limits and cleanup.
- Automatic rescheduling is disabled for stateful or non-idempotent workloads unless policy declares safety.
- Drain/delete cannot strand the last healthy replica or detach data implicitly.
Acceptance criteria
- Two pools can route production workloads and builds to different eligible nodes and explain each decision.
- Concurrent scheduling respects reservations and constraints without overcommit.
- A node drain safely moves/redeploys eligible stateless workloads and reports blocked stateful ones.
- Remote build produces a verified artifact usable by normal promotion without rebuilding.
Verification
- Deterministic placement, concurrency/reservation, affinity/spread, drain, node failure, and cache isolation tests.
- Security tests for build secrets, network policy, workspace escape, artifact signature, and cleanup.
- End-to-end remote build → artifact → multi-node deploy → node drain scenario.
Non-goals
- Reimplementing Kubernetes.
- Automatically moving stateful workloads without a storage/consistency plan.
- Scheduling based on opaque heuristics that cannot be explained.