Roadmap relationship
Outcome
Turn the documented MultiRegionManager into real, persisted orchestration for regional stacks, data replication, global traffic, health-driven failover, recovery, and safe teardown.
Current gap
Multi-region APIs and docs describe S3 replication, DynamoDB global tables, secret replication, Route53/CloudFront/WAF, regional deployment, and failover, but several core methods only log placeholder messages, deployment state is in memory, and no CLI/dashboard operational workflow exists.
User experience
- Configure regions, primary/secondary or active-active mode, weights, capacity/resource overrides, replication choices, health checks, failover thresholds, and budget implications.
- Plan view highlights global versus regional resources, unsupported services, data consistency model, replication direction, DNS/TTL/RTO effects, and estimated cost.
- Topology dashboard shows regional health, traffic, release/config versions, replication lag/state, failover readiness, and last drill.
- Failover is plan/confirm/execute with clear automatic/manual mode, traffic progression, abort/failback, and audit.
- Partial deployment/replication failure shows per-region recovery actions without collapsing all state to one error.
Scope
- Persist multi-region deployment/topology state and implement real regional stack create/update/delete via existing generator/CloudFormation services.
- Implement S3 cross-region replication prerequisites/rules, DynamoDB global table replicas, Secrets Manager replication, and explicit unsupported data paths.
- Provision/manage health checks, Route53 latency/weighted/failover records, CloudFront regional origins/behaviors, certificates, and WAF associations where selected.
- Add staged release rollout, traffic shifting, automatic health-triggered failover with cooldown, manual failover/failback, and disaster simulations.
- Expose plan/apply/status/events/failover/failback/destroy through CLI/API/dashboard.
Technical direction
- Model topology and operations as durable state machines with per-region checkpoints and idempotency.
- Validate account/region permissions, service availability, quotas, certificates, replication prerequisites, and naming before mutation.
- Use provider ETags/change tokens and observed-state reconciliation for DNS/CDN/global resources.
- Tag/correlate every global and regional resource and deployment release.
Safety and operability
- Do not claim failover readiness until data, health, DNS/traffic, capacity, and runbook checks pass.
- Automatic failover requires explicit policy, bounded attempts, split-brain protections, cooldown, and audited reason.
- Secret replication preserves access boundaries and never copies unsupported plaintext config.
- Destroy proceeds global traffic → replicas/data policy → regional compute with retention confirmations.
Acceptance criteria
- A two-region reference application deploys real regional stacks and configured global routing.
- Selected S3/DynamoDB/Secrets replication is observable and failures are actionable.
- A health drill shifts traffic according to policy, validates secondary health, and supports controlled failback.
- Restarting the control plane does not lose topology or in-flight failover state.
Verification
- Plan/preflight, per-region partial failure, idempotent resume, replication, ETag conflict, cooldown, split-brain, and destroy-order tests.
- Provider integration tests in isolated accounts/fixtures for CloudFormation, Route53, CloudFront, S3, DynamoDB, and Secrets Manager.
- Scheduled end-to-end failover/failback drill with measured RTO and recorded evidence.
Non-goals
- Promise of zero RPO/RTO for all data engines.
- Abstracting non-AWS providers into unsupported AWS global services.
- Automatic active-active writes for databases without a validated consistency model.