Data Storage · Study deck

Storage Design Case Studies

Picture a cold-room sensor that reports four degrees during the day, then reconnects at night with six hours of saved readings.

Data Dora is your guide for this deck.

Data Dora, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

translate IoT requirements into storage roles, query paths, and lifecycle windows;
design a fleet tracking storage stack that separates registry, telemetry, current state, and archive data;
design a smart city data lake with separate tracks for telemetry, video, metadata, and compliance exports;
identify where exact cost and performance numbers must be replaced by measured release evidence;

Major section

Start With the Review Room

Firmware means the software stored on that device.

The manager needs to know which values are current, which arrived late, and which can support an audit.
A product name cannot answer those questions.
The live view should not treat old data as new.
The history should keep the original time and identity.

Major section

Start With the Review Room (continued)

Telemetry here is the time-stamped reading and device status that the storage path must preserve, including the original device identity and observation time when delayed batches arrive after newer values.

The restore should reproduce the reviewed result.
This runway does not choose a database or prove its cost.
The example is useful only if it leaves that review trail visible.

Major section

How to Use These Examples

For 2,400 devices with eight metrics each, the design has 19,200 streams.

At one sample every 15 seconds, 19,200 / 15 s = 1,280 points/s; over 86,400 seconds that is about 110.6 million points/day.

A four-step storage decision path (quantify, split roles, sketch schema, prove release) with a worked capacity calculation, run before any database product is named.

Major section

Capacity Arithmetic Before Product Choice

The term that often breaks estimates is active series count.

A low-cardinality dimension such as site may be safe when each device belongs to exactly one site and users filter by site.
The arithmetic therefore needs a separate retention window and overhead assumption for each tier.
Capacity review also separates disk from memory.

Retention arithmetic becomes practical when each data class has a temperature tier: hot data supports frequent reads, warm data supports periodic investigation, cold data supports rare restore, and archive data supports compliance evidence.

Major section

Capacity Arithmetic Before Product Choice (continued)

An unbounded per-message field such as request_id, raw GPS coordinate, session id, or free-text note should not become a tag or series key.

Hot capacity must serve dashboards and investigations; warm and cold tiers trade speed for cost; archive is credible only with checksums and a tested restore.
Its ingest rate is 19,200 / 15 = 1,280 points per second.
Over a day, that is 1,280 x 86,400 = 110,592,000 points.

Major section

Capacity Arithmetic Before Product Choice (continued)

For example, a building fleet with 2,400 devices, 8 metrics, and one sample every 15 seconds has 2,400 x 8 = 19,200 active metric streams.

At 12 compressed bytes per point, the value stream is about 1.33 GB per day, so 30 hot days is about 40 GB before replicas, indexes, write-ahead logs, object manifests, and backups.
Disk is mostly a function of points and bytes per point, which depends on codec, sampling regularity, value variance, and retention tier.
Memory is driven heavily by active series metadata.

Major section

Worked Example 1: Fleet Tracking

InfluxDB can still be appropriate when spatial joins are not central.

Users need a live map, route history, geofence alerts, driver and vehicle reports, and audit exports.
The design must support growth without forcing a database migration every time a new query appears.
MongoDB can be appropriate when event documents are the primary query surface.

Major section

Worked Example 1: Fleet Tracking (continued)

The deciding factor is the query contract, not a generic product ranking.

The important question is not only "how many rows?" It is "which reads must be local, fresh, and cheap?".
Fleet tracking is a poor fit for a single "one database does everything" decision.
The schema keeps event time, insert time, tenant, vehicle, location, and quality evidence together.

Major section

Worked Example 2: Smart City Data Lake

A city platform collects environmental telemetry, traffic counts, streetlight status, water quality readings, video clips, asset metadata, and incident records.

City operators need dashboards.
Analysts need historical queries and model training datasets.
Compliance teams need retention evidence and restore procedures.
The first design mistake is treating every stream as the same kind of data.

A city data platform routes sensors and connectivity into stream processing, time-series or lake storage, analytics, and application services.

Major section

Worked Example 2: Smart City Data Lake (continued)

The second is optimizing the small stream while ignoring the large one.

Parking, traffic, lights, and waste can arrive through LoRaWAN, cellular, fibre, Ethernet, or city Wi-Fi.
Sensor data, media, and device records have different search needs, life spans, access rules, and restore plans.
The object path is not the only index.

Major section

Worked Example 2: Smart City Data Lake (continued)

Otherwise every investigation becomes a bucket crawl.

One city platform can join them without forcing all of them into the same storage contract.
A searchable metadata table should hold camera id, district, incident id, retention class, privacy status, and review status.
The data lake needs to know what a value meant when it was recorded.

Major section

Summary

Worked examples should train design judgment.

For fleet tracking, the key is separating registry, telemetry, spatial queries, latest state, rollups, and archive exports.
For smart city storage, the key is splitting telemetry, media, metadata, analytics, and compliance tracks.
For migration safety, the key is a stable event envelope and durable ingestion log.

Deck summary

Key takeaways

Firmware means the software stored on that device.

Telemetry here is the time-stamped reading and device status that the storage path must preserve, including the original device identity and observation time when delayed batches arrive after newer values.
For 2,400 devices with eight metrics each, the design has 19,200 streams.
The term that often breaks estimates is active series count.
An unbounded per-message field such as request_id, raw GPS coordinate, session id, or free-text note should not become a tag or series key.

Retrieval practice

Recall check 1 of 5

Data Dora says: answer from memory, then check your reasoning.

Q1Two storage workloads ingest the same number of points per second, but one has 100 times more active series. Why is the high-cardinality workload usually more expensive to run?

AMore series always means every individual point uses more bytes on disk.

BThe index and query engine must track far more active series metadata.

CHigh cardinality removes the need for rollups and retention windows.

DThe sampling interval no longer affects ingest rate.

Show answer

Answer: B Disk sizing follows points and bytes per point, while memory and planning costs often follow active series count.

Retrieval practice

Recall check 2 of 5

Data Dora says: answer from memory, then check your reasoning.

Q2In the fleet tracking example, why is a latest-state table or cache useful even when raw telemetry is stored in a time-series table?

AIt gives live maps a bounded read path

BIt replaces the need to store historical telemetry

CIt removes the need for geospatial indexes

DIt guarantees that vehicles never send late data

Show answer

Answer: A The map should read one current row per vehicle and expose freshness lag, rather than scanning raw history for every refresh.

Retrieval practice

Recall check 3 of 5

Data Dora says: answer from memory, then check your reasoning.

Q3In the smart city example, which design choice most directly prevents old sensor readings from becoming impossible to interpret after hardware upgrades?

AKeeping a schema version and unit contract with the sensor data

BStoring every video clip in the same object bucket

CUsing the lowest-cost archive tier for all data

DRemoving district and sensor type from dashboard queries

Show answer

Answer: A Schema version, unit contract, and calibration metadata let analysts interpret historical readings even after sensor models change.

Q4What serves the current fleet state without rescanning raw telemetry history?

ARegistry Store

BTelemetry Store

CLatest State

DArchive Export

Show answer

Answer: C A current-view path answers fleet and map queries without rescanning raw history.

Retrieval practice

Recall check 4 of 5

Data Dora says: answer from memory, then check your reasoning.

Q5A storage estimate uses a sample price and a small demo query. What evidence is needed before release?

ACopy the sample estimate into the release decision.

BUse vendor benchmark speed as the query-plan evidence.

CMeasure target payloads, growth, query plans, cost and restore time.

DTest only the ingest path and assume queries and restore will work.

Show answer

Answer: C Release claims need measurements for the actual workload and environment.

Retrieval practice

Recall check 5 of 5

Data Dora says: answer from memory, then check your reasoning.

Q6A team may change its telemetry database later. Which design best avoids a device firmware change?

AMake devices write directly to a vendor-only database API.

BStable events and an ingest log let stores change downstream.

CRemove event versions so the new store sees fewer fields.

DDelete the old ingest log as soon as a new store starts.

Show answer

Answer: B A stable ingest contract lets storage sinks change behind the device boundary.

Print reference

Answers 1 of 2

Answer key.

B · Disk sizing follows points and bytes per point, while memory and planning costs often follow active series count.
A · The map should read one current row per vehicle and expose freshness lag, rather than scanning raw history for every refresh.
A · Schema version, unit contract, and calibration metadata let analysts interpret historical readings even after sensor models change.
C · A current-view path answers fleet and map queries without rescanning raw history.

Print reference

Answers 2 of 2

Answer key.

C · Release claims need measurements for the actual workload and environment.
B · A stable ingest contract lets storage sinks change behind the device boundary.