Data Storage · Study deck

Storage Design Case Studies

Picture a cold-room sensor that reports four degrees during the day, then reconnects at night with six hours of saved readings.

Data Dora is your guide for this deck.

Data Dora, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

translate IoT requirements into storage roles, query paths, and lifecycle windows;
design a fleet tracking storage stack that separates registry, telemetry, current state, and archive data;
design a smart city data lake with separate tracks for telemetry, video, metadata, and compliance exports;
identify where exact cost and performance numbers must be replaced by measured release evidence;

Major section

Start With the Review Room

The manager needs to know which values are current, which arrived late, and which can support an audit.

A product name cannot answer those questions.
Telemetry means readings and status sent by a remote device.
Firmware means the software stored on that device.
The live view should not treat old data as new.

Major section

Start With the Review Room (continued)

The history should keep the original time and identity.

The restore should reproduce the reviewed result.
This runway does not choose a database or prove its cost.
A team should be able to point at the workload, explain the storage roles, show which queries matter, and prove what happens when data ages, moves, or must be restored.
The example is useful only if it leaves that review trail visible.

Major section

Capacity Arithmetic Before Product Choice

The term that often breaks estimates is active series count.

A low-cardinality dimension such as site may be safe when each device belongs to exactly one site and users filter by site.
The arithmetic therefore needs a separate retention window and overhead assumption for each tier.
Capacity review also separates disk from memory.

Retention arithmetic becomes practical when each data class has a temperature tier: hot data supports frequent reads, warm data supports periodic investigation, cold data supports rare restore, and archive data supports compliance evidence.

Major section

Capacity Arithmetic Before Product Choice (continued)

An unbounded per-message field such as request_id, raw GPS coordinate, session id, or free-text note should not become a tag or series key.

Hot capacity must serve dashboards and investigations; warm and cold tiers trade speed for cost; archive is credible only with checksums and a tested restore.
Its ingest rate is 19,200 / 15 = 1,280 points per second.
Over a day, that is 1,280 x 86,400 = 110,592,000 points.

Major section

Capacity Arithmetic Before Product Choice (continued)

For example, a building fleet with 2,400 devices, 8 metrics, and one sample every 15 seconds has 2,400 x 8 = 19,200 active metric streams.

At 12 compressed bytes per point, the value stream is about 1.33 GB per day, so 30 hot days is about 40 GB before replicas, indexes, write-ahead logs, object manifests, and backups.
Disk is mostly a function of points and bytes per point, which depends on codec, sampling regularity, value variance, and retention tier.
Memory is driven heavily by active series metadata.

Major section

Worked Example 1: Fleet Tracking

InfluxDB can still be appropriate when spatial joins are not central.

Users need a live map, route history, geofence alerts, driver and vehicle reports, and audit exports.
The design must support growth without forcing a database migration every time a new query appears.
MongoDB can be appropriate when event documents are the primary query surface.

Major section

Worked Example 1: Fleet Tracking (continued)

The deciding factor is the query contract, not a generic product ranking.

The important question is not only "how many rows?" It is "which reads must be local, fresh, and cheap?".
Fleet tracking is a poor fit for a single "one database does everything" decision.
The schema keeps event time, insert time, tenant, vehicle, location, and quality evidence together.

Major section

Worked Example 2: Smart City Data Lake

A city platform collects environmental telemetry, traffic counts, streetlight status, water quality readings, video clips, asset metadata, and incident records.

City operators need dashboards.
Analysts need historical queries and model training datasets.
Compliance teams need retention evidence and restore procedures.
The first design mistake is treating every stream as the same kind of data.

A city data platform routes sensors and connectivity into stream processing, time-series or lake storage, analytics, and application services.

Major section

Worked Example 2: Smart City Data Lake (continued)

Otherwise every investigation becomes a bucket crawl.

The second is optimizing the small stream while ignoring the large one.
The sequence explains why the telemetry, media, registry, and analytics tracks below need separate query, lifecycle, and ownership contracts rather than one undifferentiated lake.
The object path is not the only index.

Major section

Summary

Worked examples should train design judgment.

For fleet tracking, the key is separating registry, telemetry, spatial queries, latest state, rollups, and archive exports.
For smart city storage, the key is splitting telemetry, media, metadata, analytics, and compliance tracks.
For migration safety, the key is a stable event envelope and durable ingestion log.

Deck summary

Key takeaways

The manager needs to know which values are current, which arrived late, and which can support an audit.

The history should keep the original time and identity.
The term that often breaks estimates is active series count.
An unbounded per-message field such as request_id, raw GPS coordinate, session id, or free-text note should not become a tag or series key.
For example, a building fleet with 2,400 devices, 8 metrics, and one sample every 15 seconds has 2,400 x 8 = 19,200 active metric streams.

Retrieval practice

Recall check 1 of 4

Data Dora says: answer from memory, then check your reasoning.

Q1Two storage workloads ingest the same number of points per second, but one has 100 times more active series. Why is the high-cardinality workload usually more expensive to run?

AThe index and query engine must track far more active series metadata.

BMore series always means every individual point uses more bytes on disk.

CHigh cardinality removes the need for rollups and retention windows.

DThe sampling interval no longer affects ingest rate.

Show answer

Answer: A Disk sizing follows points and bytes per point, while memory and planning costs often follow active series count.

Retrieval practice

Recall check 2 of 4

Data Dora says: answer from memory, then check your reasoning.

Q2In the fleet tracking example, why is a latest-state table or cache useful even when raw telemetry is stored in a time-series table?

AIt replaces the need to store historical telemetry

BIt removes the need for geospatial indexes

CIt guarantees that vehicles never send late data

DIt gives live maps a bounded read path

Show answer

Answer: D The map should read one current row per vehicle and expose freshness lag, rather than scanning raw history for every refresh.

Retrieval practice

Recall check 3 of 4

Data Dora says: answer from memory, then check your reasoning.

Q3In the smart city example, which design choice most directly prevents old sensor readings from becoming impossible to interpret after hardware upgrades?

AStoring every video clip in the same object bucket

BKeeping a schema version and unit contract with the sensor data

CUsing the lowest-cost archive tier for all data

DRemoving district and sensor type from dashboard queries

Show answer

Answer: B Schema version, unit contract, and calibration metadata let analysts interpret historical readings even after sensor models change.

Q4Complete the event-envelope validator used before routing data to multiple storage sinks:

Amissing = sorted(REQUIRED_FIELDS - event.keys())

Bmissing = sorted(event.keys() - REQUIRED_FIELDS)

Cmissing = []

Dmissing = list(event.values())

Show answer

Answer: A A migration-safe pipeline needs a stable event envelope.

Retrieval practice

Recall check 4 of 4

Data Dora says: answer from memory, then check your reasoning.

Q5Place each storage role where it lives so you can route writes and queries to the right freshness, identity, and retention contract.

ARegistry Store

BColor Palette

CPacket Checksum

DButton Theme

Show answer

Answer: A The three regions separate durable identity, operational history and current views, and governed archives so you can choose a store from the workload rather than a product label.

Print reference

Answers 1 of 2

Answer key.

A · Disk sizing follows points and bytes per point, while memory and planning costs often follow active series count.
D · The map should read one current row per vehicle and expose freshness lag, rather than scanning raw history for every refresh.
B · Schema version, unit contract, and calibration metadata let analysts interpret historical readings even after sensor models change.
A · A migration-safe pipeline needs a stable event envelope.

Print reference

Answers 2 of 2

Answer key.

A · The three regions separate durable identity, operational history and current views, and governed archives so you can choose a store from the workload rather than a product label.