21  Pipeline Transmission

In 60 Seconds

Transmission is the stage where all of the careful work in sensing, calibration, and formatting finally becomes expensive. The radio burst is usually short, but it dominates battery drain, adds most of the end-to-end delay, and turns a 2-byte sensor reading into a much larger over-the-air packet. This chapter shows how to choose the right wireless link, where latency really accumulates, and which optimizations produce meaningful battery and bandwidth savings.

21.1 Learning Objectives

By the end of this chapter, you will be able to:

• Compare Wireless Technologies: Evaluate Wi-Fi, BLE, LoRa, and NB-IoT for range, power, and data rate trade-offs in IoT deployments
• Trace the Complete Journey: Follow sensor data from physical measurement through each pipeline stage to a cloud dashboard
• Calculate Energy Budgets: Quantify power consumption and battery life for different wireless protocols and transmission intervals
• Diagnose Latency Bottlenecks: Identify where time is spent across pipeline stages and determine which optimizations yield the largest improvements
• Design Optimized Pipelines: Select appropriate formats, protocols, and processing locations based on latency, power, and bandwidth constraints

Related Chapters

This Series:

• Sensor Pipeline Index - Series overview
• Pipeline Overview and Signal Acquisition - Stages 1-3
• Processing and Formatting - Stages 4-6

Networking:

• Networking Fundamentals - Network basics
• IoT Protocols Overview - Protocol selection

Architecture:

• Edge Computing - Data processing at the edge
• Energy Management - Power optimization

21.2 Prerequisites

Before diving into this chapter, you should be familiar with:

• Processing and Formatting: Stages 4-6 of the pipeline
• Packet Structure and Framing: How data is packaged for transmission

For Beginners: The Final Mile

Your sensor data is now processed, formatted, and wrapped in protocol headers. The final stage is transmission - actually sending it through the air to a gateway or directly to the cloud.

This is where the pipeline meets the physical world: - Radio waves carry your data through walls, across fields, or up to satellites - Power consumption spikes dramatically during transmission - Latency accumulates as packets travel through networks

Understanding transmission completes your knowledge of the full pipeline, enabling you to optimize where it matters most.

MVU: Transmission Dominates Everything

Core Concept: Network transmission typically consumes 60-70% of total pipeline latency and 80-90% of energy in battery-powered IoT devices. This single stage determines battery life more than all other stages combined.

Why It Matters: A common mistake is optimizing local processing speed when the real bottleneck is network transmission. Spending weeks improving ADC conversion by 10μs is pointless when transmission takes 41ms. Focus optimization efforts proportionally to where time and energy are actually spent.

Key Takeaway: Always design your pipeline for the bottleneck - which is almost always network transmission. The IEEE defines efficient IoT systems as those achieving >95% data reduction before transmission through edge processing.

---

21.3 Stage 7: Network Transmission

⏱️ ~9 min | ⭐⭐ Intermediate | 📋 P02.C04.U08

Key Concepts

• Radio bursts dominate energy: A device can sleep for minutes, but one high-current transmission still consumes most of the energy budget.
• Payload size multiplies downstream cost: Every extra byte picks up framing, routing, and retransmission overhead as it moves through the stack.
• Latency is mostly off-device: Sampling and local encoding are usually microseconds to milliseconds; network hops and cloud handling are the larger delays.
• Protocol choice must match the job: Wi-Fi, BLE, LoRa, and NB-IoT make different trade-offs in range, airtime, power draw, and infrastructure requirements.
• Edge processing is the highest-leverage optimization: Suppressing unimportant transmissions often saves more than any per-packet compression trick.
• Over-the-air behavior matters more than spec-sheet throughput: Duty cycle, retries, gateway availability, and session setup determine real deployment performance.

How It Works: Wireless Transmission

The big picture: Radio transmission physically moves data through the air using electromagnetic waves, consuming 60-90% of total system energy.

Step-by-step breakdown:

1. Modulation: Digital bits 101010… encode as radio frequency changes - Real example: LoRa uses chirp spread spectrum, Wi-Fi uses OFDM
2. Amplification: Power amplifier boosts signal from milliwatts to transmit power - Real example: Wi-Fi transmits at 200mW (23 dBm), LoRa at 25mW (14 dBm)
3. Propagation: Radio waves travel through air at speed of light (300,000 km/s) - Real example: 100-meter Wi-Fi range takes 0.33 microseconds, but protocol overhead adds 5-50ms

Why this matters: Transmission dominates energy budget. A sensor sleeping 99% of time at 50µA but transmitting 1% at 200mA averages 2050µA — transmission contributes 98% of total power despite being only 1% of time.

21.3.1 Wireless Transmission

Physical transmission over wireless medium:

• Wi-Fi: 2.4 GHz radio, CSMA/CA protocol
• LoRa: Sub-GHz, chirp spread spectrum modulation
• BLE: 2.4 GHz, frequency hopping

Power consumption varies dramatically:

• Wi-Fi: 200-500 mW transmit, drains 1000 mAh battery in 8 hours
• BLE: 10-20 mW transmit, battery lasts weeks
• LoRa: 20-40 mW transmit, battery lasts years (due to duty cycle)

Putting Numbers to It

Radio transmission energy for different protocols:

Battery-powered sensor transmits 32-byte packet once per minute for 1 year:

\[\text{Annual transmissions} = 365 \times 24 \times 60 = 525{,}600 \text{ packets}\]

Wi-Fi (200 mW TX, 5ms airtime): \[E_{\text{Wi-Fi}} = 200 \text{ mW} \times 5 \text{ ms} \times 525{,}600 = 525.6 \text{ Wh/year}\] \[\text{CR2032 capacity} \approx 0.7 \text{ Wh} \rightarrow \text{battery lasts } 0.5 \text{ days}\]

BLE (10 mW TX, 2ms airtime): \[E_{\text{BLE}} = 10 \text{ mW} \times 2 \text{ ms} \times 525{,}600 = 10.5 \text{ Wh/year}\] \[\text{Battery lasts } \approx 24 \text{ days}\]

LoRa (30 mW TX, 80ms airtime at SF7): \[E_{\text{LoRa}} = 30 \text{ mW} \times 80 \text{ ms} \times 525{,}600 = 1.26 \text{ Wh/year}\] \[\text{Battery lasts } \approx 6.6 \text{ months}\]

Conclusion: LoRa’s longer airtime is offset by infrequent transmissions. For 1 reading/minute, LoRa provides 8× better battery life than BLE despite higher per-transmission energy cost.

Try It: Battery Life Calculator

Experiment with different wireless protocols and transmission rates to see their impact on battery life:

viewof protocol = Inputs.select(["Wi-Fi", "BLE", "LoRa"], {label: "Wireless Protocol", value: "LoRa"})
viewof tx_power = Inputs.range([10, 500], {value: 30, step: 10, label: "TX Power (mW)"})
viewof airtime = Inputs.range([1, 200], {value: 80, step: 1, label: "Airtime per Packet (ms)"})
viewof interval_mins = Inputs.range([1, 1440], {value: 1, step: 1, label: "Transmission Interval (minutes)"})
viewof battery_capacity = Inputs.range([0.1, 3.0], {value: 0.7, step: 0.1, label: "Battery Capacity (Wh)"})

battery_calc = {
  const transmissions_per_year = 365 * 24 * (60 / interval_mins);
  const energy_per_year_wh = (tx_power / 1000) * (airtime / 1000 / 3600) * transmissions_per_year;
  const battery_life_years = battery_capacity / energy_per_year_wh;
  const battery_life_days = battery_life_years * 365;
  const battery_life_months = battery_life_years * 12;

  return {
    transmissions_per_year: Math.round(transmissions_per_year),
    energy_per_year_wh: energy_per_year_wh.toFixed(3),
    battery_life_years: battery_life_years.toFixed(2),
    battery_life_days: Math.round(battery_life_days),
    battery_life_months: battery_life_months.toFixed(1)
  };
}

html`<div style="background: var(--bs-light, #f8f9fa); padding: 1.5rem; border-radius: 8px; border-left: 4px solid #16A085; margin-top: 1rem;">
<h4 style="margin-top: 0; color: #2C3E50;">Results for ${protocol}</h4>
<table style="width: 100%; border-collapse: collapse;">
<tr style="border-bottom: 1px solid #ddd;">
  <td style="padding: 0.5rem;"><strong>Annual Transmissions:</strong></td>
  <td style="padding: 0.5rem; text-align: right;">${battery_calc.transmissions_per_year.toLocaleString()}</td>
</tr>
<tr style="border-bottom: 1px solid #ddd;">
  <td style="padding: 0.5rem;"><strong>Energy per Year:</strong></td>
  <td style="padding: 0.5rem; text-align: right;">${battery_calc.energy_per_year_wh} Wh</td>
</tr>
<tr style="border-bottom: 1px solid #ddd; background: #e8f5e9;">
  <td style="padding: 0.5rem;"><strong>Battery Life:</strong></td>
  <td style="padding: 0.5rem; text-align: right; font-size: 1.1em; font-weight: bold; color: #16A085;">
    ${battery_calc.battery_life_days < 365 ?
      `${battery_calc.battery_life_days} days (${battery_calc.battery_life_months} months)` :
      `${battery_calc.battery_life_years} years`}
  </td>
</tr>
</table>
<p style="margin-top: 1rem; font-size: 0.9em; color: #7F8C8D;">
<strong>Insight:</strong> ${
  interval_mins <= 5 ?
    "Frequent transmissions dramatically reduce battery life. Consider edge processing to reduce transmission frequency." :
  battery_calc.battery_life_days < 30 ?
    "Short battery life. Consider using a lower-power protocol like BLE or LoRa, or increasing transmission interval." :
  battery_calc.battery_life_days >= 365 * 2 ?
    "Excellent battery life! This configuration supports multi-year deployments." :
    "Moderate battery life. Suitable for applications with annual battery replacement."
}
</p>
</div>`

Figure 21.1

21.3.2 Wireless Technology Comparison

• Wi-Fi: Roughly 50 m indoors, 11-600 Mbps, and the highest transmit current. Best when power is available and you need rich payloads, video, or existing LAN infrastructure.
• BLE: 10-100 m, 1-2 Mbps, and low radio energy. Best for wearables, phone-linked sensors, beacons, and frequent short updates.
• LoRa: 2-15 km, 0.3-50 kbps, and very low average power when duty-cycled. Best for sparse telemetry from remote assets where multi-year batteries matter more than latency.
• NB-IoT: Up to wide-area cellular coverage, about 250 kbps, and moderate power with operator dependency. Best for outdoor deployments where gateway ownership is impractical but carrier coverage exists.

---

21.4 The Complete Journey: Sensor to Cloud and Back

⏱️ ~12 min | ⭐⭐⭐ Advanced | 📋 P02.C04.U09

21.4.1 Following a Single Temperature Reading

Let’s trace one measurement from a smart home temperature sensor all the way to appearing on your smartphone dashboard and back.

Starting Point: Temperature sensor reads 23.5°C Ending Point: Value appears on dashboard 500 milliseconds later Question: What happened in between?

Figure 21.2

21.4.2 Step-by-Step Breakdown with Timing

1. Sensor ADC conversion: 10 microseconds to convert the analog reading into a raw 2-byte value.
2. MCU calibration and formatting: about 100 microseconds to calibrate the reading and encode it as a 32-byte JSON payload such as {"temp":23.5,"unit":"C"}.
3. Radio transmission: about 5 milliseconds to place the payload into a Wi-Fi frame with MAC overhead and CRC, growing the over-the-air packet to about 127 bytes.
4. Gateway or access point handling: about 20 milliseconds to bridge the device traffic into the site network and prepare it for MQTT or HTTP forwarding.
5. Internet transport: 50-200 milliseconds of variable WAN latency, where the message grows again to about 250 bytes after IP, TCP, and application headers.
6. Cloud processing: about 50 milliseconds to parse, validate, store, and route the update.
7. Dashboard update: about 100 milliseconds for query, push, and client rendering.

Total journey: roughly 500 milliseconds, with the original 2-byte reading expanding to about 250 bytes by the time it reaches the cloud service.

21.4.3 Where Time is Spent

Understanding latency distribution helps you optimize the right part of the pipeline:

Figure 21.3

Key Insight: Network Optimization Has Biggest Impact

• Network transmission: 60-70% of latency (325ms)
• Cloud processing: 20-25% (100ms)
• Local processing: 5-10% (25ms)

Optimization Strategy: Reduce network round-trips first! Moving from cloud to edge processing can cut latency from 500ms → 50ms (10× improvement).

21.4.4 Where Bytes are Added (Encapsulation Overhead)

Watch how a simple 2-byte temperature reading grows to 250 bytes:

Figure 21.4

• Original sensor value: 2 bytes for the raw temperature reading.
• After JSON encoding: 32 bytes, a 16x jump caused by field names, punctuation, and timestamp metadata.
• After Wi-Fi framing: 127 bytes once MAC addresses, frame control fields, and CRC are attached for the local radio hop.
• After TCP/IP and cloud delivery headers: about 250 bytes after IP, TCP, MQTT, and service-side delivery metadata are included.

Key Insight: Protocol Efficiency Matters for Constrained Devices

• JSON overhead: 2 bytes → 32 bytes (16× increase)
• Protocol headers: 32 bytes → 250 bytes (8× increase)
• Total overhead: 2 bytes → 250 bytes (125× increase)

Real Impact on Battery-Powered Sensor:

• Transmitting 2 bytes @ 20mW for 1ms = 20 µJ
• Transmitting 250 bytes @ 20mW for 125ms = 2,500 µJ (125× more energy!)

---

21.5 Optimization Comparison

Let’s compare three pipeline designs for the same temperature sensor:

• Cloud-First: JSON over Wi-Fi and TCP/IP. About 500 ms end-to-end, about 15 mJ per reading, and roughly 6 months on a 2000 mAh battery at one reading per minute. Best when development simplicity matters more than battery life.
• Edge-Optimized: CBOR over BLE and MQTT-SN. About 50 ms end-to-end, about 1.2 mJ per reading, and about 6 years of battery life. This is often the best balance for responsive battery-powered systems.
• Ultra-Efficient: compact binary over LoRa and CoAP. Around 1-2 seconds of latency, about 0.8 mJ per reading, and up to 12 years of battery life. Best for remote sensing where long life matters more than immediacy.

Key Tradeoffs:

1. Cloud-First: Fast development, easy debugging, but short battery life
2. Edge-Optimized: Best balance - good latency, 10× battery improvement
3. Ultra-Efficient: Maximum battery life, but higher latency (acceptable for slow-changing temperature)

Design Decision Framework

When optimizing your sensor-to-cloud pipeline, ask:

1. What’s the latency budget?
   • Real-time control (< 100ms)? → Edge processing + fast protocol
   • Monitoring (1-60s)? → Cloud is fine
   • Historical logging (> 1 min)? → Optimize for battery, not speed
2. What’s the data rate?
   • High frequency (> 1 Hz)? → Edge processing to reduce cloud traffic
   • Low frequency (< 0.1 Hz)? → Cloud direct is acceptable
3. What’s the power budget?
   • Wall-powered? → Use Wi-Fi, JSON, cloud - simplicity wins
   • Battery (< 1 year)? → Needs optimization
   • Battery (> 5 years)? → Requires BLE/LoRa + binary + edge processing
4. What’s the bandwidth cost?
   • Cellular IoT (paid per MB)? → Aggressive compression essential
   • Wi-Fi/Ethernet (fixed cost)? → Bandwidth is free

Example Decisions:

• Smart thermostat (wall-powered, 1 reading/min, 100ms latency OK)
  • → Wi-Fi + JSON + Cloud = Simple, maintainable
• Soil moisture sensor (battery, 1 reading/hour, 1-hour latency OK)
  • → LoRa + Binary + Edge gateway = 10-year battery
• Industrial vibration monitor (battery, 1000 readings/sec, 10ms latency required)
  • → Edge FFT processing + alert-only transmission = Real-time + efficient

21.5.1 The Return Journey: Cloud to Dashboard

Don’t forget the data also needs to travel back to the user!

Dashboard Update Flow (additional 200-300ms):

1. Database query: Cloud fetches latest reading (10-20ms)
2. WebSocket push: Server pushes to connected clients (50-100ms)
3. Client rendering: Browser/app updates chart (50-100ms)
4. Display refresh: Screen shows new value (16ms @ 60fps)

Total round-trip (sensor → cloud → dashboard): ~700-800ms

Optimization: Use WebSocket for live updates instead of HTTP polling (reduces latency from 5-30s to < 1s)

---

21.6 Knowledge Check Scenarios

Scenario 1: Smart Agriculture Deployment

You’re deploying 500 soil moisture sensors across a vineyard. Each sensor measures moisture every 15 minutes and transmits via LoRaWAN.

Current design:

• Sensor outputs 12-bit ADC value (2 bytes)
• JSON payload: {"sensor_id": "V001", "moisture": 2847, "battery": 3.72, "timestamp": 1704067200}
• Payload size: 72 bytes after LoRaWAN headers

Question: The sensors are draining batteries in 8 months instead of the target 2 years. What pipeline optimizations would you recommend?

21.6.1 Recommended answer

Recommended optimizations (potential 3-4x battery life improvement):

1. Switch from JSON to Binary Format (Stage 5)

• Current: 72 bytes JSON
• Optimized: 8 bytes binary
  • Sensor ID: 2 bytes (uint16)
  • Moisture: 2 bytes (uint16)
  • Battery: 1 byte (0-255 mapped to 2.5-4.2V)
  • Timestamp: 3 bytes (delta from gateway time)
• Savings: 89% payload reduction

2. Implement Change-Based Transmission (Stage 4)

• Soil moisture changes slowly (hours, not minutes)
• Only transmit when moisture changes by >2%
• Reduces transmissions from 96/day to ~5-10/day
• Savings: 90% transmission reduction

3. Optimize Sampling (Stage 3)

• 12-bit ADC is overkill for soil moisture (±5% accuracy)
• Use 10-bit ADC or average 4 samples
• Reduce ADC power by 25%

Combined Impact:

• Original: 72 bytes × 96 TX/day × 20mW = significant power
• Optimized: 8 bytes × 8 TX/day × 20mW = ~90% power reduction
• New battery life: 8 months × 3-4 = 24-32 months (exceeds target!)

Key insight: The biggest wins come from reducing transmission frequency (edge processing) and payload size (binary encoding), not from optimizing individual stages.

Scenario 2: Real-Time Industrial Monitoring

A factory uses vibration sensors on 20 critical motors. The maintenance team needs alerts within 500ms of detecting abnormal vibration patterns.

Current design:

• Accelerometer samples at 10 kHz
• Raw data streamed to cloud via Wi-Fi
• Cloud performs FFT analysis
• Alert pushed back to factory floor

Problem: End-to-end latency is 2-3 seconds, missing the 500ms requirement.

Question: Where in the pipeline is latency being added, and how would you fix it?

21.6.2 Recommended answer

Latency breakdown analysis:

• Sampling (Stage 3): about 10 ms for 1024 samples at 10 kHz.
• Processing (Stage 4): about 5 ms for packaging and handoff.
• Formatting (Stage 5): about 2 ms for JSON encoding.
• Network transport (Stages 6-7): 50-200 ms across Wi-Fi and internet hops.
• Cloud processing: 1000-2000 ms, which is the real bottleneck because FFT analysis and queueing happen after upload.
• Return path: another 500-1000 ms for alert routing back to the floor.
• Total: roughly 2-3 seconds, dominated by off-device analysis.

Solution: Move FFT to Edge (Stage 4)

Redesigned pipeline:

1. Sensor → 10 kHz sampling (unchanged)
2. MCU → Perform FFT locally (100ms for 1024-point FFT on ESP32)
3. MCU → Analyze frequency spectrum for known defect signatures
4. MCU → Only transmit if anomaly detected (not raw data)
5. Alert → Direct MQTT publish to local display

New latency breakdown:

• Sampling: 100 ms to collect 1024 samples.
• Local FFT: 100 ms on the ESP32.
• Local analysis: 50 ms for pattern matching.
• Local alert publish: 50 ms through a nearby broker or controller.
• Display update: 100 ms to render on the screen.
• Total: about 400 ms, which meets the 500 ms target.

Additional benefits:

• Bandwidth reduction: 10 kHz × 2 bytes × 20 sensors = 400 KB/s → alert-only = 1 KB/day
• Cloud cost reduction: No real-time streaming charges
• Reliability: Works even if internet connection fails

Key insight: For real-time requirements, move processing as close to the sensor as possible. Network latency is often the bottleneck that can’t be optimized—you must reduce the number of network hops.

Scenario 3: Protocol Overhead Analysis

You’re comparing two designs for a wearable health monitor that sends heart rate every second via BLE to a smartphone app.

Design A (Simple):

• Payload: JSON {"hr": 72} = 10 bytes
• BLE characteristic write

Design B (Optimized):

• Payload: Single byte (heart rate 30-285 BPM mapped to 0-255)
• BLE notification

Question: Calculate the total bytes transmitted per second for each design, including BLE protocol overhead.

21.6.3 Recommended answer

Design A - JSON over BLE Write:

• Payload: 10 bytes for {"hr": 72}.
• ATT write request: 3 bytes.
• L2CAP header: 4 bytes.
• Link-layer data PDU: 2 bytes.
• Access address: 4 bytes.
• CRC: 3 bytes.
• Total per packet: 26 bytes.

Plus acknowledgment packet: ~10 bytes

Total Design A: ~36 bytes/second

---

Design B - Binary over BLE Notification:

• Payload: 1 byte for the heart-rate value.
• ATT notification: 3 bytes.
• L2CAP header: 4 bytes.
• Link-layer data PDU: 2 bytes.
• Access address: 4 bytes.
• CRC: 3 bytes.
• Total per packet: 17 bytes.

No acknowledgment needed (notifications are unconfirmed)

Total Design B: ~17 bytes/second

---

Comparison:

• Bytes per second: 36 for Design A versus 17 for Design B, a 53% reduction.
• Daily data volume: about 3.1 MB versus 1.5 MB, saving about 1.6 MB per day.
• Radio airtime: about 3 ms versus about 1.5 ms, roughly halving airtime.
• Battery impact: lower average radio time in Design B yields roughly 20% longer battery life.

Key insight: Protocol overhead often exceeds payload size in IoT! A 10-byte JSON payload requires 36 bytes total (3.6x overhead), while a 1-byte binary payload requires 17 bytes (17x overhead ratio but less absolute overhead). For frequently transmitted small payloads, binary format and notifications provide significant efficiency gains.

21.6.4 Try It: Protocol Overhead Calculator

Explore how BLE protocol overhead affects total transmission size:

viewof payload_size = Inputs.range([1, 200], {value: 10, step: 1, label: "Payload Size (bytes)"})
viewof use_notification = Inputs.toggle({label: "Use Notification (vs Write)", value: true})

ble_overhead = {
  const att_header = 3;  // ATT opcode + handle
  const l2cap_header = 4;  // L2CAP length + channel ID
  const ll_pdu = 2;  // Link Layer PDU header
  const access_addr = 4;  // BLE preamble
  const crc = 3;  // Error checking

  const overhead = att_header + l2cap_header + ll_pdu + access_addr + crc;
  const total_bytes = payload_size + overhead;
  const ack_bytes = use_notification ? 0 : 10;  // Write requires ACK
  const total_with_ack = total_bytes + ack_bytes;
  const overhead_percent = ((overhead + ack_bytes) / total_with_ack * 100).toFixed(1);

  return {
    overhead,
    total_bytes,
    ack_bytes,
    total_with_ack,
    overhead_percent
  };
}

html`<div style="background: var(--bs-light, #f8f9fa); padding: 1.5rem; border-radius: 8px; border-left: 4px solid #3498DB; margin-top: 1rem;">
<h4 style="margin-top: 0; color: #2C3E50;">BLE ${use_notification ? "Notification" : "Write"} Analysis</h4>
<table style="width: 100%; border-collapse: collapse;">
<tr style="border-bottom: 1px solid #ddd;">
  <td style="padding: 0.5rem;"><strong>Payload:</strong></td>
  <td style="padding: 0.5rem; text-align: right;">${payload_size} bytes</td>
</tr>
<tr style="border-bottom: 1px solid #ddd;">
  <td style="padding: 0.5rem;"><strong>Protocol Overhead:</strong></td>
  <td style="padding: 0.5rem; text-align: right;">${ble_overhead.overhead} bytes</td>
</tr>
${use_notification ? "" : `<tr style="border-bottom: 1px solid #ddd;">
  <td style="padding: 0.5rem;"><strong>ACK Overhead:</strong></td>
  <td style="padding: 0.5rem; text-align: right;">${ble_overhead.ack_bytes} bytes</td>
</tr>`}
<tr style="border-bottom: 1px solid #ddd; background: #fff3cd;">
  <td style="padding: 0.5rem;"><strong>Total Transmitted:</strong></td>
  <td style="padding: 0.5rem; text-align: right; font-weight: bold;">${ble_overhead.total_with_ack} bytes</td>
</tr>
<tr style="background: #e3f2fd;">
  <td style="padding: 0.5rem;"><strong>Overhead Percentage:</strong></td>
  <td style="padding: 0.5rem; text-align: right; font-size: 1.1em; font-weight: bold; color: #3498DB;">
    ${ble_overhead.overhead_percent}%
  </td>
</tr>
</table>
<p style="margin-top: 1rem; font-size: 0.9em; color: #7F8C8D;">
<strong>Insight:</strong> ${
  ble_overhead.overhead_percent > 80 ?
    `With ${ble_overhead.overhead_percent}% overhead, most bytes are headers! For tiny payloads, consider batching multiple readings into one packet.` :
  ble_overhead.overhead_percent > 50 ?
    `${ble_overhead.overhead_percent}% overhead is typical for small IoT payloads. Binary encoding and notifications help minimize waste.` :
    `At ${ble_overhead.overhead_percent}% overhead, your payload size is well-optimized for BLE transmission.`
}
${!use_notification ? " Note: BLE Writes require ACK packets, doubling latency and airtime. Use Notifications for one-way streaming data." : ""}
</p>
</div>`

---

21.7 Additional Knowledge Check

Knowledge Check: Pipeline Optimization Quick Check

Concept: Understanding pipeline bottlenecks and optimization strategies.

21.7.1 Worked Example: Syngenta Precision Agriculture Pipeline Latency Budget

Scenario: Syngenta operates a precision irrigation trial on a 500-hectare vineyard in Mendoza, Argentina. Soil moisture sensors trigger irrigation valves in real-time. The pipeline must deliver sensor readings to the valve controller within 10 seconds to prevent water waste from over-irrigation.

Given:

• 2,000 Watermark 200SS soil moisture sensors (200 per hectare zone, 10 zones)
• Measurement interval: every 2 minutes
• Sensor output: 0-200 centibars (cb), 12-bit ADC
• Communication: LoRaWAN Class C (continuous receive), SF7, 125 kHz
• Gateway: Kerlink Wirnet iFemtoCell-evolution, 1 per zone
• Network server to irrigation controller: 4G LTE backhaul
• Latency budget: 10 seconds end-to-end

Step 1: Map pipeline stages to latency

Pipeline Stage	Operation	Time	% of Budget
1. Physical sensing	Watermark sensor excitation + stabilization	200 ms	2%
2. Signal conditioning	Anti-aliasing RC filter, settling time	50 ms	0.5%
3. ADC conversion	12-bit SAR ADC at 10 ksps	0.1 ms	~0%
4. Processing	Calibration + 5-sample median filter	5 ms	0.05%
5. Formatting	CBOR encode (device_id + moisture_cb + zone)	1 ms	0.01%
6. Packet assembly	LoRaWAN MAC framing (13-byte header)	1 ms	0.01%
7a. LoRa transmission	11-byte payload + 13-byte header at SF7	46 ms airtime	0.5%
7b. Gateway processing	Demodulation + packet forwarding	15 ms	0.15%
7c. Network server	LoRaWAN session handling + MQTT publish	120 ms	1.2%
7d. 4G backhaul	LTE round-trip to irrigation controller	80 ms	0.8%
7e. Controller processing	Decision logic + valve actuation command	50 ms	0.5%
Total		568 ms	5.7%

Step 2: Identify bottleneck and margin

Analysis	Value
Total pipeline latency	568 ms
Latency budget	10,000 ms
Margin	9,432 ms (94.3% headroom)
Dominant stage	Stage 7c: Network server (120 ms, 21% of actual latency)
Dominant category	Transmission (stages 7a-7e): 311 ms, 55% of total

Step 3: Evaluate design alternatives

What if…?	Impact on Latency	Impact on Battery
Use SF12 instead of SF7	7a increases from 46 ms to 1,810 ms (total: 2,332 ms, still within budget)	Battery life drops from 5 years to 8 months
Use Wi-Fi instead of LoRaWAN	7a drops to 5 ms, but need powered AP per zone	No battery benefit (Wi-Fi draws 200 mA vs 40 mA for LoRa)
Add edge processing to skip cloud	Remove 7c+7d, total drops to 368 ms	No change
Reduce from SF7 to SF6	7a drops to 25 ms (total: 547 ms)	15% battery improvement

Result: The LoRaWAN pipeline at SF7 delivers end-to-end latency of 568 ms, well within the 10-second budget. The dominant cost is not speed but power: at SF7 with 2-minute intervals, the CR2477 coin cell battery lasts 5.1 years. This means the real optimization target is battery life, not latency, since 94% of the latency budget is unused.

Key Insight: Pipeline latency budgeting often reveals that the bottleneck is not where you expect. In this vineyard deployment, the physical sensing stage (200 ms sensor stabilization) takes longer than the entire LoRa radio transmission (46 ms). And the biggest surprise: with 94% latency headroom, the correct optimization focus shifts from latency to energy – extending battery life from 5 to 8 years by batching 5 readings per transmission rather than reducing per-packet latency.

Concept Relationships: Complete Pipeline

• Network transmission -> energy budget: the radio burst consumes 80-90% of total energy in many battery-powered deployments.
• Latency -> network hops: each gateway, cellular attach, broker, or cloud handoff adds delay that local optimization cannot remove.
• Edge processing -> transmission volume: local filtering or summarization can cut traffic by 90-99% for slowly changing signals.
• Protocol overhead -> payload size: for tiny readings, headers often outweigh the application data itself.

Cross-module connection: Energy and Power Management covers duty-cycling and low-power optimization strategies.

Common Pitfalls

1. Picking the Radio Before Defining the Real Requirement

Teams often choose Wi-Fi, BLE, or LoRa based on familiarity instead of the actual update interval, range, and latency budget. Start with the requirement first: how far, how often, how fast, and on what battery target. The radio should be the consequence of that analysis, not the starting assumption.

2. Sending Human-Friendly Payloads Over Expensive Links

JSON is attractive during development, but on low-rate or metered links it can multiply airtime, bandwidth cost, and retry exposure. If the payload is tiny and frequent, every field name matters. Use compact encodings or summarized events when the link is the bottleneck.

3. Treating Connectivity Loss as an Exception

In field deployments, interference, missed gateways, weak coverage, and carrier delays are normal operating conditions. If the design assumes perfect delivery, the system will either drain the battery with retries or silently lose data. Build around buffering, retry policy, and graceful degradation from day one.

🏷️ Label the Diagram

Code Challenge

21.8 Summary

The complete sensor-to-network pipeline transforms physical phenomena into transmitted packets:

1. Physical Measurement (Stage 1): Sensor converts phenomenon to electrical signal
2. Signal Conditioning (Stage 2): Amplify, filter, and offset analog signals
3. ADC Conversion (Stage 3): Transform continuous analog to discrete digital
4. Digital Processing (Stage 4): Calibrate, filter, and apply edge intelligence
5. Data Formatting (Stage 5): Encode as JSON, CBOR, or binary
6. Packet Assembly (Stage 6): Wrap in protocol headers
7. Network Transmission (Stage 7): Send over wireless medium

Key Takeaway

Network transmission (Stage 7) dominates both latency (60-70% of total pipeline time) and energy consumption (80-90% of total pipeline energy), making it the single most important stage to optimize. Edge processing before transmission provides the biggest savings: sending a 1-byte “alert” instead of raw data can reduce energy by 100x. Design your pipeline around its primary bottleneck – latency-critical systems minimize hops, power-critical systems minimize transmissions, bandwidth-critical systems maximize compression.

21.9 See Also

• LoRaWAN Overview — Long-range, low-power wireless protocol
• BLE Basics — Bluetooth Low Energy for short-range IoT
• Networking Fundamentals — TCP/IP stack and protocol overhead

For Kids: Meet the Sensor Squad!

Bella the Battery was very worried about Stage 7 – the final stage!

“Transmission is the energy monster!” Bella explained. “When Sammy sends data through the radio, it uses 80-90% of ALL my energy. Everything else – sensing, processing, formatting – is just 10-20%!”

Sammy the Sensor felt guilty. “Is there anything I can do to help?”

Max the Microcontroller had a plan: “Instead of sending every single reading, I’ll be smart about it! I’ll process the data locally and only send an alert when something changes. Instead of 1000 messages, maybe just 10!”

Lila the LED cheered: “That’s edge computing! Max does the thinking locally, and Sammy only has to transmit the important stuff!”

Bella smiled and felt her charge level stabilize. “If Max processes 100 readings and only sends 1 summary, I can last 100 times longer! The best transmission is the one you don’t have to make!”

The lesson: The radio is the biggest energy drain in IoT. Smart processing before transmission is the key to long battery life!

---

21.10 What’s Next

Now that you can trace, calculate, and optimize the complete sensor-to-network pipeline, explore these related topics:

• Signal Processing Essentials: deeper coverage of ADC conversion, sampling theory, and digital filtering.
• Data Formats for IoT: detailed comparison of JSON, CBOR, Protocol Buffers, and custom binary encodings.
• Edge Computing: when and how to process data at the edge to reduce transmission volume.
• LoRaWAN Overview: long-range, low-power networking for remote sensor deployments.
• BLE Fundamentals: BLE architecture for short-range wearables and beacon systems.
• Energy Management: duty-cycling and context-aware low-power optimization strategies.

Continue to Signal Processing Essentials –>