Data Storage · Study deck

Time-Series Database Fundamentals

Take one temperature reading and follow it through a system.

Data Dora is your guide for this deck.

timeseries

Data Dora, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Define a timestamp contract that separates observed time, receive time, and ingest time.
Describe an append write path. Explain how fixed segments, chunks, indexes, and later merging support time-series storage.
Explain how column layout and three compression methods reduce search work: storing changes between values, replacing repeated text with codes, and counting repeated values.
Review whether a time-series design has enough evidence for release.

Major section

In 60 Seconds

Each reading has a timestamp, which records a time.

Telemetry means measurements or status reports sent from a device.: A successful database keeps those facts intact while data enters, is stored, is searched, is summarised, is aged, and is restored after a failure.
The fundamentals are design ideas, not product names.
The design states which clock supplied each time.

Major section

The Shape of Time-Series Data

A time-series record is not just a table row with a time.

A data contract states the fields, units, time rules, and quality checks that the system must preserve.
The storage engine sees a repeated pattern.
New readings arrive often, while changes to old readings are rare.

Major section

Timestamp Contracts

Timestamp bugs are storage bugs.

A chart that mixes device local time, gateway time, and platform ingest time without labels can hide late data, create false gaps, and make incident replay unreliable.
The primary time column for storage depends on the use case.
Device observed time is often best for physical-world analysis.

Major section

The Append Write Path

The common pattern is to validate, append for durability, buffer or sort, write immutable segments or chunks, and maintain small summaries that help later queries skip irrelevant data.

This separation explains how the store can accept a sustained stream without making every insert pay the full cost of future layout.
Some use relational partitions or hypertables.

An append-oriented write path protects durability first, then organizes readings into immutable time chunks that can be scanned, compacted, rolled up, or expired.

Major section

Log-Structured Write Mechanics

A normal B-tree index can store timestamps, but high-rate append telemetry stresses the in-place update model.

A B-tree keeps keys sorted in pages, so a stream of inserts must update the main index and every secondary index while the write path is still waiting.
The tradeoff is read amplification.

LSM tree write path showing writes buffered in a memtable, flushed to sorted files, and compacted by merging levels.

Major section

Query Lifecycle

The best time-series query is the query that reads only the necessary time range, entity subset, metric subset, and columns.

That requires the schema, chunking, indexes, and rollups to match the actual questions people ask.
The returned value is trustworthy only when the result also carries freshness, coverage, and quality evidence for the question asked.

Query performance depends on pruning by time first, then using dimensions, summaries, and rollups to avoid scanning raw history unnecessarily.

Major section

Cardinality and Dimensions

Cardinality is the number of distinct series or indexed dimension combinations.

A stable tag such as site=plant-a can be useful.
High-cardinality information is not automatically bad.
The danger is putting it in the access path when it is not used for broad filtering.

Major section

Worked Review: Vibration Telemetry

Without it, a benchmark, product label, or compression claim cannot prove the system will answer the user's questions.

The platform needs live dashboards, maintenance investigation, and a monthly reliability report.
The storage review should not start by asking which database is fastest.
It should define evidence for the workload.

Major section

Common Pitfalls

If a record only says time, reviewers cannot tell whether it came from the sensor, gateway, broker, or storage system.

Retention is risky when raw data disappears before aggregate refresh, sample counts, min/max values, and late arrivals have been checked.
Compression cannot fix mixed units, unclear metrics, unbounded labels, irregular timestamps, or a query that scans the wrong data.

Deck summary

Key takeaways

Each reading has a timestamp, which records a time.

A time-series record is not just a table row with a time.
Timestamp bugs are storage bugs.
The common pattern is to validate, append for durability, buffer or sort, write immutable segments or chunks, and maintain small summaries that help later queries skip irrelevant data.
A normal B-tree index can store timestamps, but high-rate append telemetry stresses the in-place update model.

Retrieval practice

Recall check 1 of 4

Data Dora says: answer from memory, then check your reasoning.

Q1Which observation most strongly indicates a time-series storage workload rather than a normal transactional workload?

AThe database stores at least one timestamp column.

BEvery record must be updated several times before it is complete.

CRecords are mostly appended and most reads filter by time range.

DQueries usually retrieve one customer account by primary key.

Show answer

Answer: C Append-heavy writes and time-window reads are the core workload signals that drive time-series physical design.

Retrieval practice

Recall check 2 of 4

Data Dora says: answer from memory, then check your reasoning.

Q2Why do high-ingest time-series stores commonly use log-structured write paths instead of relying only on in-place B-tree updates?

AThey turn hot writes into append and buffer work, then merge immutable files later.

BThey remove the need to record data durably before acknowledging a write.

CThey make every range query read exactly one file without metadata.

DThey prove timestamps cannot be indexed by B-trees at all.

Show answer

Answer: A Log-structured storage uses append, memory buffering, immutable flushed units, and compaction to fit append-heavy telemetry while preserving range-query metadata.

Retrieval practice

Recall check 3 of 4

Data Dora says: answer from memory, then check your reasoning.

Q3Place each storage responsibility where it lives so you can trace a reading from acceptance to defensible query evidence.

AIngest Contract

BAppend Durability

CTime Chunks

DQuery and Lifecycle Evidence

Show answer

Answer: A Place acceptance, durable organization, and release evidence correctly so you can explain which storage promise failed.

Q4Complete the timestamp admission check before a reading is written to time-series storage.

Aif observed > now_utc + seconds(max_future_skew_s):

Bif observed < now_utc + seconds(max_future_skew_s):

Cif received == observed:

Dif record["device_id"] == "":

Show answer

Answer: A A timestamp contract should reject impossible future observations and mark late arrivals without losing the original observed time.

Retrieval practice

Recall check 4 of 4

Data Dora says: answer from memory, then check your reasoning.

Q5A team says their time-series design is ready because the database can insert readings quickly in a demo. What evidence is still missing?

AQuery, lifecycle, and recovery evidence, not just fast inserts.

BOnly the product logo and cloud provider region.

CNothing else; insert speed is the only important time-series requirement.

DOnly a larger disk size estimate.

Show answer

Answer: A Ingest speed alone does not prove the time-series storage design will remain correct, queryable, and recoverable.

Print reference

Answers 1 of 2

Answer key.

C · Append-heavy writes and time-window reads are the core workload signals that drive time-series physical design.
A · Log-structured storage uses append, memory buffering, immutable flushed units, and compaction to fit append-heavy telemetry while preserving range-query metadata.
A · Place acceptance, durable organization, and release evidence correctly so you can explain which storage promise failed.
A · A timestamp contract should reject impossible future observations and mark late arrivals without losing the original observed time.

Print reference

Answers 2 of 2

Answer key.

A · Ingest speed alone does not prove the time-series storage design will remain correct, queryable, and recoverable.