← Back to the chapter · Previous detail

Technical detail · Time-Series Query Optimization

Rollup refresh and invalidation rules

A rollup is not only a smaller table. It is a promise that a dashboard can read fewer rows without losing the evidence needed to explain the result. TimescaleDB continuous aggregates, InfluxDB downsampling tasks, and Prometheus recording rules all make that trade: compute a cheaper series or bucketed view on a schedule, then route repeated dashboards to that precomputed tier.

The cost shift is easy to review. A fleet with 10,000 devices sampled once per minute creates 432,000,000 raw rows in 30 days for one metric. A device-level one-hour rollup for the same window has 7,200,000 rows, a 60x reduction. If most dashboards show site averages for 50 sites, the site-hour rollup is only 36,000 rows. That is small enough for repeated dashboard refreshes, provided the query is deliberately routed to the rollup tier and the UI labels the granularity honestly.

The hidden contract is refresh scope. A continuous aggregate policy might refresh buckets from three days ago up to five minutes ago, leaving the newest five minutes to raw real-time reads. Late readings from yesterday are corrected during the next refresh. Late readings from last month may require an explicit backfill job. Prometheus recording rules and InfluxDB tasks have the same ownership problem: they must name the source range, destination series or bucket, late-arrival behavior, and rerun policy.

Rule
What it proves
Failure if missing
Source query
Which raw rows, labels, quality states, and bucket size create the rollup.
The summary cannot be audited against the raw telemetry path.
Refresh window
Which recent and historical buckets are recomputed on the schedule.
Late arrivals silently diverge from the dashboard result.
Invalidation trigger
Which buckets are marked dirty when backfilled or corrected data arrives.
The system looks fast because it is reading stale summaries.
Raw fallback
Which incident or audit path can recover the highest-resolution window.
Operators cannot explain a short spike hidden by the aggregate.

← Back to the chapter · Previous detail