← Back to the chapter · Previous detail

Technical detail · Time-Series Platform Selection

Storage engine units

9.8 Storage Engine Units

Under the product names, mature time-series systems often share one storage idea: buffer recent writes, seal immutable time-partitioned units, compact and compress in the background, and prune by time.

InfluxDB 1.x/2.x storage used the TSM and TSI lineage: writes landed in a write-ahead log and cache, then flushed to immutable time-series files with an index that mapped tags to series. Newer InfluxDB engines use a different storage stack, so a platform review should always record the exact product version and storage engine being approved. TimescaleDB splits a hypertable into chunks partitioned by time and optionally by a space dimension; each chunk is a PostgreSQL child table behind a routing layer. Prometheus appends to a write-ahead log and a hot head block, then seals immutable on-disk blocks with chunk files and a label index that compaction can merge into larger ranges.

Use storage units to reason about cost. In TimescaleDB, a one-day chunk interval over 180 days creates roughly 180 time chunks before any space partitioning. A query for the last six hours should prune almost all of those chunks; a retention policy can drop old chunks without scanning every row; and compression after seven days can rewrite colder chunks into a better layout for scans. In Prometheus, the head block covers recent samples and then turns into block windows, so 15 days of local retention starts as many block ranges before compaction merges them. A query engine can skip blocks outside the time range and then use the label index to find matching series.

The durable idea is that time-series platforms trade row-by-row mutation for append, partition, compact, and prune. Once students can identify the partition unit, index path, and retention drop unit, platform documentation becomes much easier to compare.

← Back to the chapter · Previous detail