System Design Patterns and Data Flow
By the end of this chapter, you will be able to:
Architecture is the “big picture” of how an IoT system fits together. Just like a building has a floor plan showing how rooms connect, a sensor system architecture shows how data flows from sensors, through gateways and networks, up to the cloud where it gets analyzed and turned into useful insights. Understanding this layered structure helps you plan projects that work reliably at any scale.
In This Series:
Architecture Topics:
This layered view categorizes sensors by criticality level - from safety-critical to convenience - helping you understand different reliability and accuracy requirements.
This decision tree helps you quickly narrow down connectivity options based on key deployment criteria. The path from indoor battery-powered leads to BLE for ultra-low power, while outdoor real-time applications typically require cellular connectivity.
Key Architecture Decisions:
Temperature sensors dominate IoT deployments because they are: - Universal: Needed across all domains (cities, homes, industry, agriculture, health) - Low-cost: Starting at $1-2 for basic sensors - Easy to integrate: Simple analog or digital interfaces - Well-understood: Mature technology with predictable behavior
Lifecycle Phases Explained:
Sensor1 ---+
|
Sensor2 ---+--- Gateway --- Cloud
|
Sensor3 ---+Use when: Small deployments (<20 sensors), all sensors within gateway range, Wi-Fi/BLE connectivity Example: Smart home, single office floor, small greenhouse
Sensor1 --- Sensor2
| X |
Sensor3 --- Sensor4 --- Gateway --- CloudUse when: Large area coverage, redundancy required, Zigbee/Thread connectivity Example: Industrial floor, smart city block, large farm
Sensors --- Edge1 ---+
|
Sensors --- Edge2 ---+--- Cloud
|
Sensors --- Edge3 ---+Use when: Large-scale deployments (100+ sensors), edge processing needed, LoRaWAN/cellular Example: City-wide deployment, multi-site enterprise, large agricultural operation
Critical Sensors --- Direct Cloud (Cellular)
|
Bulk Sensors --- Edge ---+ (LoRaWAN)
|
Local Sensors --- Hub ---+ (Wi-Fi)Use when: Mixed requirements (real-time + batch), different criticality levels, cost optimization Example: Hospital (critical patient monitors + environmental sensors), factory (safety + monitoring)
| Sensor Type | Indoor Coverage | Outdoor Coverage | Notes |
|---|---|---|---|
| Temperature | 50-100 m2 | 500-1000 m2 | Depends on HVAC zones |
| Motion (PIR) | 5-12 m cone | Not recommended | Line of sight required |
| Gas (CO2, VOC) | 50 m2 | 100-500 m2 | Diffusion-limited |
| Particulate (PM2.5) | 100 m2 | 1-2 km2 | Wind disperses particles |
| Noise | 20-50 m radius | 100-200 m radius | Inverse square law |
| Ultrasonic | 2-4 m direct | 4-8 m direct | Point measurement |
Scenario: Monitor air quality in 10,000 m2 office building
Formula: Data Rate = Sensors × Sample Rate × Bytes per Sample × Overhead
Example: 100 temperature sensors, 1 sample/minute, 4 bytes/sample, 2× overhead = 100 × (1/60) × 4 × 2 = 13.3 bytes/second = 0.1 kbps
Network Bandwidth vs. Cloud Cost Relationship
For 1,000 sensors sampling every 5 minutes: - Raw data rate: \(1000 \times \frac{1}{300\text{s}} \times 10\text{ bytes} \times 1.5\text{ overhead} = 50\text{ bytes/s}\) - Daily volume: \(50 \times 86400 = 4.32\text{ MB/day} \approx 130\text{ MB/month}\)
Cloud storage cost (AWS IoT Core at \(0.08/million messages):\)\(\text{Cost} = \frac{1000 \times 12 \times 30}{1,000,000} \times 0.08 = \$0.029/\text{month}\)$
But data transfer (egress) costs $0.09/GB, so: \(0.13\text{ GB} \times 0.09 = \$0.012/\text{month}\)
Key insight: For low-frequency IoT sensors, messaging costs dominate bandwidth concerns—network capacity is rarely the bottleneck; pricing models are. This math justifies edge aggregation strategies that reduce message counts rather than raw bytes.
| Application | Sensors | Sample Rate | Data per Sensor | Total Bandwidth |
|---|---|---|---|---|
| Smart Parking | 1000 | 0.01 Hz | 10 bytes | 0.1 kbps |
| Air Quality | 100 | 0.1 Hz | 50 bytes | 0.5 kbps |
| Industrial Vibration | 20 | 1000 Hz | 4 bytes | 640 kbps |
| Video Surveillance | 10 | 30 fps | 100 KB/frame | 240 Mbps |
Key Insight: Video dominates bandwidth. For sensor-only deployments, bandwidth is rarely the bottleneck; power and cost are.
Sensor application architecture is about matching your design choices (topology, connectivity, edge vs. cloud processing) to your specific constraints (latency, data volume, power, cost, reliability). Not all data needs to reach the cloud – edge processing can reduce bandwidth by 100x. Start with the deployment lifecycle: define requirements, select sensors, design the architecture, prototype at 5-10% scale, optimize, then deploy fully.
Sammy the Sensor had a BIG project – helping design a sensor network for a whole city!
“There are so many sensors to organize,” Sammy said, looking at a map. “Parking sensors, air quality monitors, traffic counters, streetlights…”
Max the Microcontroller pulled out his planning board. “Let’s think about this like organizing a school. Not every student goes directly to the principal, right?”
“Right!” said Lila the LED. “Students talk to their teacher first. Teachers report to the department head. Department heads report to the principal. That’s a HIERARCHY!”
Max drew it out: “Exactly! Sensors talk to nearby gateways. Gateways send summaries to the cloud. The cloud shows everything on a dashboard. This way, the network doesn’t get overwhelmed.”
Bella the Battery raised a concern: “But what if a gateway breaks? Won’t all those sensors lose connection?”
“Good thinking, Bella!” Max replied. “For really important sensors – like gas leak detectors – we give them their own direct phone line to the cloud using cellular. Less important ones, like temperature sensors, can wait and send data in batches. That’s called a HYBRID design – mixing different connection types based on how important each sensor is.”
“So it’s like having a regular mailbox for birthday cards but a special emergency phone for 911?” Sammy asked.
“Perfect analogy! In IoT architecture, we always match the connection type to how critical the data is. Fast and reliable for emergencies, slow and cheap for routine check-ins!”
Scenario: DHL Express is retrofitting a distribution warehouse in Rotterdam, Netherlands. The warehouse handles 45,000 packages/day and must monitor temperature zones, forklift traffic, and air quality for worker safety compliance (EU Directive 89/654/EEC).
Given:
Step 1 – Calculate sensor counts per zone:
| Zone | Area | Sensor Type | Coverage/Sensor | Sensors Needed | Unit Cost | Zone Cost |
|---|---|---|---|---|---|---|
| Ambient | 12,000 m2 | Temperature (DHT22) | 100 m2 | 120 | EUR 8 | EUR 960 |
| Chilled | 5,000 m2 | Temperature (DS18B20, waterproof) | 50 m2 | 100 | EUR 12 | EUR 1,200 |
| Frozen | 3,000 m2 | Temperature (PT100 industrial) | 50 m2 | 60 | EUR 45 | EUR 2,700 |
| All zones | 20,000 m2 | CO2 (SCD30) | 200 m2 | 100 | EUR 55 | EUR 5,500 |
| Loading docks (x4) | 800 m2 | Door sensors + PIR | Per dock | 8 | EUR 25 | EUR 200 |
| Forklift paths | – | UWB anchors (Decawave DWM1001) | 400 m2 radius | 50 | EUR 85 | EUR 4,250 |
| Total sensors | 438 | EUR 14,810 |
Step 2 – Select architecture pattern (Hierarchical):
Step 3 – Calculate bandwidth and cloud costs:
Step 4 – Deployment timeline (hybrid lifecycle):
Result: 438 sensors across 5 edge gateways providing zone-level temperature compliance, CO2 monitoring, and forklift tracking for EUR 15,000 hardware + EUR 1,020/year cloud. Edge aggregation reduced cloud data by 98% (25 MB/day raw to 0.5 MB/day aggregated). Temperature excursions in the frozen zone are detected within 30 seconds and trigger SMS alerts to warehouse managers.
Key Insight: The biggest cost driver is not the sensors (EUR 14,810) but the installation labor (estimated EUR 40,000 for cable runs, mounting, and commissioning). When budgeting, allocate 2-3x the hardware cost for professional installation. Hierarchical architecture with edge gateways reduces ongoing cloud costs to under EUR 100/month.
This chapter covered sensor application architecture patterns:
A single MQTT broker and a single-node InfluxDB instance handle 100 sensors easily, but adding 1,000 sensors may overwhelm them. Plan for 10x growth at design time by choosing broker and database solutions that support clustering and horizontal scaling, even if you start with single-node deployments.
Storing 1-Hz raw readings from 500 sensors indefinitely generates 15 TB/year. Design data retention tiers upfront: keep raw data for 7-30 days, 1-minute aggregates for 1 year, hourly averages permanently. Retroactively re-architecturing data retention after the database is full is significantly more difficult than designing it upfront.
HTTP REST requires a new TCP connection for each request, adding 50-200 ms latency per reading. For sensors reporting every second or faster, this overhead is unacceptable and the bandwidth cost is significant. Use MQTT or WebSocket for streaming sensor data, and reserve REST APIs for configuration and historical data queries.
Network connectivity between the gateway and cloud is not guaranteed. Without local buffering at the gateway (RabbitMQ, SQLite, file-based queue), sensor readings are lost during connectivity outages. Design gateways to buffer at least 24-72 hours of sensor data locally and replay it to the cloud when connectivity is restored.
| If you want to… | Read this |
|---|---|
| Learn which sensors to choose for specific application domains | Sensor Apps: Selection Wizard |
| Practice building sensor applications in hands-on labs | Sensor Application Labs |
| Explore edge computing architecture for sensor applications | Edge Computing |
| Learn about IoT reference models and standardized architectures | IoT Reference Models |