Data Storage · Study deck

Time-Series Database Fundamentals

Take one temperature reading and follow it through a system.

Data Dora is your guide for this deck.

timeseries

Data Dora, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Define a timestamp contract that separates observed time, receive time, and ingest time.
Describe an append write path. Explain how fixed segments, chunks, indexes, and later merging support time-series storage.
Explain how column layout and three compression methods reduce search work: storing changes between values, replacing repeated text with codes, and counting repeated values.
Review whether a time-series design has enough evidence for release.

Major section

In 60 Seconds

Each reading has a timestamp, which records a time.

Telemetry means measurements or status reports sent from a device: A successful database keeps those facts intact while data enters, is stored, is searched, is summarised, is aged, and is restored after a failure.
The fundamentals are design ideas, not product names.
The design states which clock supplied each time.

Major section

The Shape of Time-Series Data

A time-series record is not just a table row with a time.

A data contract states the fields, units, time rules, and quality checks that the system must preserve.
The storage engine sees a repeated pattern.
New readings arrive often, while changes to old readings are rare.

Major section

Timestamp Contracts

Timestamp bugs are storage bugs.

A chart that mixes device local time, gateway time, and platform ingest time without labels can hide late data, create false gaps, and make incident replay unreliable.
The primary time column for storage depends on the use case.
Device observed time is often best for physical-world analysis.

Major section

The Append Write Path

Depending on the engine, buffered readings are flushed into immutable blocks or written into time chunks that remain updateable.

Later merge and rollup jobs change the data layout away from the reply path.
Some use relational partitions or hypertables.
Some use log-structured storage.
Some use columnar files and metadata.

Example append-oriented write path from durable append to immutable blocks, compaction, and rollup.

Major section

Log-Structured Write Mechanics

A normal B-tree index can store timestamps, but high-rate append telemetry stresses the in-place update model.

A B-tree keeps keys sorted in pages, so a stream of inserts must update the main index and every secondary index while the write path is still waiting.
The tradeoff is read amplification.

Log-structured time-series write path: append to a write-ahead log, batch readings in an in-memory buffer, flush immutable files or blocks, and later compact smaller units while updating summary metadata.

Major section

Query Lifecycle

The best time-series query is the query that reads only the necessary time range, entity subset, metric subset, and columns.

That requires the schema, chunking, indexes, and rollups to match the actual questions people ask.
The returned value is trustworthy only when the result also carries freshness, coverage, and quality evidence for the question asked.

Query performance depends on pruning by time first, then using dimensions, summaries, and rollups to avoid scanning raw history unnecessarily.

Major section

Cardinality and Dimensions

Cardinality is the number of distinct series or indexed dimension combinations.

A stable tag such as site=plant-a can be useful.
High-cardinality information is not automatically bad.
The danger is putting it in the access path when it is not used for broad filtering.

Major section

Worked Review: Vibration Telemetry

Without it, a benchmark, product label, or compression claim cannot prove the system will answer the user's questions.

The platform needs live dashboards, maintenance investigation, and a monthly reliability report.
The storage review should not start by asking which database is fastest.
It should define evidence for the workload.

Major section

Common Pitfalls

If a record only says time, reviewers cannot tell whether it came from the sensor, gateway, broker, or storage system.

Retention is risky when raw data disappears before aggregate refresh, sample counts, min/max values, and late arrivals have been checked.
Compression cannot fix mixed units, unclear metrics, unbounded labels, irregular timestamps, or a query that scans the wrong data.

Deck summary

Key takeaways

Each reading has a timestamp, which records a time.

A time-series record is not just a table row with a time.
Timestamp bugs are storage bugs.
Depending on the engine, buffered readings are flushed into immutable blocks or written into time chunks that remain updateable.
A normal B-tree index can store timestamps, but high-rate append telemetry stresses the in-place update model.

Retrieval practice

Recall check 1 of 6

Data Dora says: answer from memory, then check your reasoning.

Q1Which observation most strongly indicates a time-series storage workload rather than a normal transactional workload?

AThe database stores at least one timestamp column.

BEvery record must be updated several times before it is complete.

CQueries usually retrieve one customer account by primary key.

DRecords are mostly appended and most reads filter by time range.

Show answer

Answer: D Append-heavy writes and time-window reads are the core workload signals that drive time-series physical design.

Retrieval practice

Recall check 2 of 6

Data Dora says: answer from memory, then check your reasoning.

Q2Why do high-ingest time-series stores commonly use log-structured write paths instead of relying only on in-place B-tree updates?

AThey remove the need to record data durably before acknowledging a write.

BThey make every range query read exactly one file without metadata.

CThey turn hot writes into append and buffer work, then merge immutable files later.

DThey prove timestamps cannot be indexed by B-trees at all.

Show answer

Answer: C Log-structured storage uses append, memory buffering, immutable flushed units, and compaction to fit append-heavy telemetry while preserving range-query metadata.

Retrieval practice

Recall check 3 of 6

Data Dora says: answer from memory, then check your reasoning.

Q3Telemetry has near-regular timestamps, repeated site labels, long runs of one quality state, and time-range queries. Which storage plan fits these patterns?

AColumns use time deltas, site codes, quality runs and time/min-max summaries.

BDictionary-code unique timestamps and omit time summaries.

CRun-length-code changing timestamps and scan each column for time queries.

DKeep averages while discarding status labels and extrema.

Show answer

Answer: A Each technique matches its data pattern, while time and min/max summaries help skip irrelevant columns or blocks.

Retrieval practice

Recall check 4 of 6

Data Dora says: answer from memory, then check your reasoning.

Q4Complete the timestamp admission check before a reading is written to time-series storage.

Aif observed > now_utc + seconds(max_future_skew_s):

Bif observed < now_utc + seconds(max_future_skew_s):

Cif received == observed:

Dif record["device_id"] == "":

Show answer

Answer: A A timestamp contract should reject impossible future observations and mark late arrivals without losing the original observed time.

Retrieval practice

Recall check 5 of 6

Data Dora says: answer from memory, then check your reasoning.

Q5A team says their time-series design is ready because the database can insert readings quickly in a demo. What evidence is still missing?

AThe selected database product's ingest benchmark and provider service agreement.

BDemo insert speed as acceptance evidence for writes and later queries.

CQuery, lifecycle, and recovery evidence, not just fast inserts.

DA larger disk estimate based on expected daily write volume.

Show answer

Answer: C Ingest speed alone does not prove the time-series storage design will remain correct, queryable, and recoverable.

Retrieval practice

Recall check 6 of 6

Data Dora says: answer from memory, then check your reasoning.

Q6A sensor reading reaches storage late. Which time fields should its record keep?

AObserved, received and ingested times, separately labelled.

BOne unlabeled local clock value for every stage.

COnly dashboard refresh time as the event time.

DIngest time replacing the original observed time.

Show answer

Answer: A Separate event, gateway-receive, and storage-ingest times expose clock error and pipeline delay without moving the physical event.

Print reference

Answers 1 of 2

Answer key.

D · Append-heavy writes and time-window reads are the core workload signals that drive time-series physical design.
C · Log-structured storage uses append, memory buffering, immutable flushed units, and compaction to fit append-heavy telemetry while preserving range-query metadata.
A · Each technique matches its data pattern, while time and min/max summaries help skip irrelevant columns or blocks.
A · A timestamp contract should reject impossible future observations and mark late arrivals without losing the original observed time.

Print reference

Answers 2 of 2

Answer key.

C · Ingest speed alone does not prove the time-series storage design will remain correct, queryable, and recoverable.
A · Separate event, gateway-receive, and storage-ingest times expose clock error and pipeline delay without moving the physical event.