Roadmap relationship
- Parent: #135
- Master roadmap: #132
- Depends on: #139, #152
- Related existing work: #108, #109, #110, #111, #112, #114
Outcome
Deliver historical, correlated observability across server, serverless, container, and edge workloads, with clear freshness/retention/cost semantics and drill-down from request or alert to deployment and runtime context.
Current gap
Current dashboard metrics rely on live probes or CloudWatch snapshots, CPU trend data is not live, log polling can go stale silently, and traces/request analytics are separate or unavailable. There is no shared time range, query model, persistent correlation, or retention management.
User experience
- Use a shared time-range picker, timezone, auto-refresh/pause, comparison period, and visible data freshness across charts and tables.
- Provide service dashboards for golden signals: traffic, errors, latency percentiles, saturation, cold starts/restarts, queue depth, and resource usage.
- Offer structured log search by time, level, source, deployment, request/trace ID, and text with live tail, pause, download, and saved queries.
- Link request → trace → logs → deployment/release → runtime instance where identifiers exist.
- Show collection status, sampling, retention, estimated ingestion/storage cost, gaps, and unsupported signals.
Scope
- Define a normalized telemetry query API and adapters for CloudWatch metrics/logs/X-Ray, host metrics/journald, container stats/logs, load balancer/CDN requests, and application-emitted correlation IDs.
- Persist control-plane/deployment events and optional locally collected time series/log indexes with configurable retention.
- Add server, service, function, container, queue, database, deployment, and request views.
- Implement request analytics for status, method, host/path template, latency, bytes, region/source, cache result, and upstream without storing sensitive query/body data.
- Integrate cost/resource work tracked in the related issues into service and environment context.
Technical direction
- Use one time-series/log query contract with cursor pagination, aggregation windows, percentile semantics, and source provenance.
- Propagate trace/request/deployment/release IDs through proxy, runtime, and job boundaries when possible.
- Downsample/compact historical metrics and bound local log storage; allow external provider-only mode.
- Cache expensive provider queries with freshness metadata and avoid creating surprising billed API traffic.
Safety and operability
- Redact configured secret patterns, authorization headers, cookies, query values, and PII fields before persistence.
- Telemetry access follows resource permissions; exports are audited and bounded.
- Sampling and retention changes show operational/cost impact.
- Never present sample/demo data as live; empty, stale, and unavailable are distinct.
Acceptance criteria
- Operators can select a time window and correlate an error spike to affected requests/logs/traces and the release deployed before it.
- Charts label source, aggregation, timezone, freshness, and gaps correctly.
- Live-tail reconnect resumes without silent staleness and historical queries remain bounded.
- At least host/server and AWS serverless paths provide the same top-level golden-signal experience.
Verification
- Aggregation/percentile/timezone/DST, cursor, gap, downsampling, cache, and retention tests.
- PII/secret redaction and resource-authorization tests across queries/exports.
- End-to-end synthetic request with trace/log/deployment correlation.
Non-goals
- Building an unlimited-scale hosted observability vendor.
- Storing full request or response bodies by default.
- Inventing cost precision when provider allocation data is unavailable.