14 CoAP Observe: Efficiency and Lifecycle
More
Depth: Practitioner
Progress saves as you read. Mark sections complete as you go; your progress appears in the sidebar.
14.1 Start With the Decision
This section: Start With the Decision — A network address translator (NAT) may let an idle UDP mapping expire before the next change reaches an observer, a client waiting for sensor changes.
A network address translator (NAT) may let an idle UDP mapping expire before the next change reaches an observer, a client waiting for sensor changes. A keep-alive is a small message sent to keep that path open. Weigh its cost against making a new connection.
14.2 Route Overview
This section: Route Overview — This is part 2 of 2.
This is part 2 of 2. Review CoAP Observe: Subscriptions and Updates for the preceding evidence.
14.3 Learning Objectives
This section: Learning Objectives
- Calculate a NAT keep-alive interval and its traffic.
- Compare Observe traffic, battery wakes, and client restarts.
14.4 Chapter Roadmap
This section: Chapter Roadmap
- Interactive Calculator: NAT Keep-Alive Interval
- Pitfall: Mismanaging Observe Tokens Across Client Restarts
- Bandwidth Savings Calculation
- Interactive Calculator: Battery Life Comparison
- Interactive: CoAP Observe Pattern Animation
- Worked Example: Industrial Vibration Monitoring with Observe
- Decision Framework: When to Use CoAP Observe
- Common Mistake: Forgetting to Handle Observer Cleanup After Client Crashes
- Checkpoint: Failure Modes
- Common Pitfalls
- 1. Using Confirmable Messages for Every CoAP Request
- 2. Ignoring CoAP Proxy Caching Semantics
- 3. Forgetting DTLS Session Management
- Label the Diagram
- Order the Steps
- Practice Exercises
- Hands-On Practice
- Checkpoint: Evidence Before Selection
- Concept Relationships
- See Also
- Match the Concepts
- What’s Next
- Summary
14.5 Pitfall: Mismanaging Observe Tokens Across Client Restarts
The Mistake: Clients generate new random tokens after reboot without deregistering previous observations, causing “ghost subscriptions” where the server continues sending notifications to tokens the client no longer recognizes.
Why It Happens: The Observe pattern uses tokens to match notifications to subscriptions. When a client reboots, it loses its token-to-subscription mapping but the server still has the observer registered.
The Fix: Implement proper token lifecycle management:
- Re-register after reboot: Rebuild observation state with fresh tokens; persist state only if its endpoint context remains valid
- Bind each request safely: Use a fresh observation Token after reboot. For secure CoAP, follow RFC 9175 Section 4.2 to prevent a response being bound to the wrong request; for example, use a per-connection sequence that is not reused until rekeying
- Handle orphaned notifications gracefully: When receiving notification with unknown token, send RST
- Server-side lease: If needed, configure a separate observer lease; Max-Age only controls representation freshness
14.6 Bandwidth Savings Calculation
This section: Bandwidth Savings Calculation
Example: Temperature sensor with 100 observers
Polling approach (GET every 10 seconds):
- Request: 14 bytes (
header + token + Uri-Path) - Response: 15 bytes (4-byte header + 2-byte Token + 2-byte Content-Format + 1-byte payload marker + 6-byte JSON string
"22.5") - Total:
29 bytes x 100 clients x 6/min x 60 min = 1,044,000 bytes/hour
Observe approach (notify on change, avg 6 changes/hour):
- CoAP header: 4 bytes
- Token: 2 bytes
- Observe option: 3 bytes
- Content-Format: 2 bytes
- Payload marker: 1 byte
- Payload: 6 bytes (JSON string
"22.5", including quotes) - Total:
18 bytes x 100 clients x 6 changes = 10.8 KB/hour
Savings: about 99% bandwidth reduction (1,044,000 vs 10,800 bytes/hour)
Scenario: A manufacturing plant monitors vibration levels on 100 motors using ESP32 sensors with accelerometers. Maintenance dashboard needs real-time updates when vibration exceeds thresholds (normal: <2.5 mm/s, warning: 2.5-4.5 mm/s, critical: >4.5 mm/s).
Comparing polling vs CoAP Observe:
Option A: Polling (GET every 5 seconds):
Client polls: GET coap://motor42.local/vibration every 5 seconds
Request: 16 bytes (CoAP header + token + URI)
Response: 20 bytes (CoAP header + token + payload "2.3")
Per sensor traffic (24 hours):
17,280 requests × 16 bytes = 276,480 bytes
17,280 responses × 20 bytes = 345,600 bytes
Total: 622,080 bytes/day per sensor
Fleet traffic: 622,080 × 100 = 62.2 MB/day
Energy per sensor: 17,280 × 3.0 mJ (CON request-response) = 51.8 J/day
Radio-event energy-only quotient (nominal 18650, 3.7 V, 3,000 mAh): 40 kJ ÷ 51.8 J/day ≈ 772 days
Option B: CoAP Observe (notify on change, max 1/minute):
Client registers: GET /vibration, Observe: 0 (once at startup)
Registration: 18 bytes (request) + 22 bytes (response with Observe: 1)
Server notifies only when vibration crosses thresholds:
Typical motor: 3 threshold crossings/day (normal ↔ warning ↔ critical)
Notifications: 3 × 20 bytes = 60 bytes/day
Plus max-rate limit: If vibrating continuously, max 1,440 notifications/day
Per sensor traffic (stable operation, 3 changes/day):
Registration: 40 bytes (one-time)
Notifications: 60 bytes/day
Total: 100 bytes on registration day; 60 bytes on later days
Fleet traffic after startup: 60 × 100 sensors = 6,000 bytes/day (vs 62.2 MB with polling)
Energy: 3 notifications × 1.5 mJ = 4.5 mJ/day
Radio-event energy-only quotient: 40,000 J ÷ 0.0045 J/day ≈ 8.9 million days (not a physical battery-life prediction)
Bandwidth savings after startup: (62,208,000 - 6,000) / 62,208,000 ≈ 99.99%
Transmit-event energy reduction: 51.8 J/day ÷ 0.0045 J/day ≈ 11,500×; actual battery-life gain also depends on sleep, sensing, listening, and battery ageing
Implementation with rate limiting:
class RateLimitedVibrationResource:
def __init__(self):
self.min_notify_interval = 60 # seconds (max 1/minute)
self.threshold_change = 0.5 # mm/s (notify if change ≥ 0.5)
self.critical_alarm_value = 4.5 # mm/s; alarm crossings bypass routine cap
self.last_notify_time = {}
self.last_notified_value = {}
async def notify_observers(self, new_value):
now = time.time()
for observer in self.observers:
last_time = self.last_notify_time.get(observer.token, 0)
last_value = self.last_notified_value.get(observer.token, None)
# Routine changes need both a value delta and the 60 s interval.
significant_change = (
last_value is not None and
abs(new_value - last_value) >= self.threshold_change
)
interval_passed = (now - last_time) >= self.min_notify_interval
urgent_alarm = (
new_value >= self.critical_alarm_value and
(last_value is None or last_value < self.critical_alarm_value)
)
if last_value is None or urgent_alarm or (significant_change and interval_passed):
await self.send_notification(observer, new_value)
self.last_notify_time[observer.token] = now
self.last_notified_value[observer.token] = new_value
Decision: CoAP Observe
Reasoning:
- About 99.99% bandwidth reduction after startup (critical for cellular backhaul)
- Transmit-event energy falls from 51.8 J/day to 4.5 mJ/day under these illustrative assumptions; total battery life also depends on sleep and sensing current
- Critical alarm crossings bypass the routine 60-second cap, so alerts have no polling delay
- Server-side rate limiting prevents notification floods
This section:
Use this flowchart to decide if Observe is appropriate for your application:
| Requirement | Use Observe | Use Polling | Use Neither (MQTT) |
|---|---|---|---|
| Update frequency | Event-driven, infrequent changes | Periodic, consistent intervals | Continuous streaming |
| Number of observers | 1-10 per resource | 1-3 per resource | 10+ observers (broker scales better) |
| Change rate | <10% of polling rate | >50% of polling rate | Constant changes |
| Battery constraints | Critical (coin cell, multi-year) | Moderate (rechargeable) | Mains-powered |
| Network reliability | Stable (LAN, Wi-Fi) | Lossy (cellular) | Stable with fallback |
| Observer lifecycle | Long-lived (hours to days) | Short-lived (seconds to minutes) | Permanent subscriptions |
Decision tree:
-
Does the resource change less often than you would poll it? → No: Use polling (Observe overhead not justified) → Yes: Continue
-
Do you need notifications from more than 10 resources per client? → Yes: Consider MQTT (broker-based pub-sub scales better) → No: Continue
-
Are clients behind NAT/firewall without port forwarding? → Yes: Problem - server can’t push to client. Options:
- CoAP over TCP (RFC 8323) with connection keep-alives where supported
- Long polling instead of Observe
- MQTT instead → No: Continue
-
Is battery life critical (>1 year target on coin cell)? → Yes: Use CoAP Observe (eliminate polling overhead) → No: Polling is acceptable, but Observe still beneficial
Example decisions:
| Application | Observe? | Reasoning |
|---|---|---|
| Temperature sensor (changes every 30 min) | Yes | Polling every 5 min wastes 5× bandwidth |
| Stock price ticker (changes every second) | No | Polling every second = always changing, no savings |
| Door sensor (changes 10×/day) | Yes | Massive savings (99% fewer messages) |
| Accelerometer (100 Hz continuous) | No | Use MQTT or streaming protocol |
| Smart meter (reads every 15 min) | Depends | If value changes every time, polling OK. If often unchanged, Observe better |
This section:
This section:
The Error: Server registers Observe subscriptions but never removes them when clients crash or restart, leading to “ghost observers” that consume memory and network bandwidth sending notifications to unreachable clients.
Why It Happens: CoAP Observe uses tokens to match notifications to requests. When a client crashes and restarts, it may generate a new token. The server can continue sending notifications to the old token; if the restarted client receives an unrecognized notification, it can reject it with RST.
Illustrative worst case without cleanup: An industrial monitoring system with 500 sensors and 20 dashboard clients observing each sensor, assuming every refresh creates a distinct observation and each stale one persists:
Without observer cleanup:
Each browser refresh creates new Observe subscription (new token)
Each old subscription remains active (server doesn't know browser closed)
After 1 week (browsers refresh ~50 times each):
Per sensor: 20 clients × 50 refreshes = 1,000 stale observations
Per sensor: 20 active observations; 1,020 entries total
Fleet: 500,000 stale + 10,000 active = 510,000 entries
Ratio per sensor: 1,000 / 20 = 50:1 stale to active
Notification overhead:
500 sensors × 10 changes/hour × 1,020 observations per sensor = 5.1M notifications/hour
Wasted: 500 sensors × 10 changes/hour × 1,000 stale observations = 5M/hour (98% waste)
CPU and network load depend on message size, transport, and how quickly RSTs are processed; they cannot be inferred from these counts.
The Fix:
1. Optional server-side observer lease (application policy, not Max-Age):
# Max-Age describes client-side representation freshness, not a server-side lease.
observer_lease_seconds = 3600 # This application chooses a one-hour lease.
# Application policy: observer must re-register before its lease expires.
def cleanup_expired_observers(self):
now = time.time()
for resource_uri, observers in self.observers.items():
self.observers[resource_uri] = [
o for o in observers
if now - o.registered_at < o.timeout
]
# Run cleanup periodically (for example, once a minute); defining it alone does nothing.
schedule_periodic(self.cleanup_expired_observers, interval_seconds=60)
2. RST detection (immediate cleanup):
def on_rst_received(self, client_addr, message_id):
"""Remove observer when client sends RST to notification"""
for resource_uri, observers in self.observers.items():
self.observers[resource_uri] = [
o for o in observers
if not (o.client_addr == client_addr and o.last_mid == message_id)
]
logging.info(f"Removed observer {client_addr} after RST")
3. CON notification failures (retry limit): This is transport pseudocode: send_datagram_once sends exactly one datagram and does not perform library-managed retransmission.
async def send_notification(self, resource_uri, observer, value):
"""Send CON notification, remove observer after 4 failed retries"""
msg = Message(code=CONTENT, token=observer.token, payload=value)
msg.mtype = CON # Confirmable - requires ACK
msg.mid = allocate_message_id() # Same MID on every retransmission.
timeout = random.uniform(2.0, 3.0) # RFC 7252 initial ACK timeout.
for attempt in range(5): # original transmission plus four retransmissions
try:
await send_datagram_once(observer.client_addr, msg)
await wait_for_ack(msg.mid, timeout=timeout)
return # Success
except TimeoutError:
logging.warning(f"Notification attempt {attempt+1} failed for {observer}")
timeout *= 2
# No ACK after original send and four retransmissions: remove observer
self.remove_observer(resource_uri, observer)
logging.info(f"Removed unresponsive observer {observer.client_addr}")
Results after implementing cleanup:
With the stated one-hour application lease, stale entries survive at most one hour unless
RST or failed CON delivery removes them earlier. Measure CPU and traffic in the deployment;
these mechanisms alone do not establish percentage savings.
Prevention checklist:
- Set Max-Age to state representation freshness (CoAP default: 60 seconds)
- If needed, define a separate server-side observer lease as application policy
- Implement RST detection and immediate observer removal
- Use CON notifications (not NON) so you detect unreachable clients
- Use at most four retransmissions after the original CON send (RFC 7252 default)
- Log observer registration/removal for debugging
- Monitor observer count per resource (alert if >expected)
This section:
This section:
Checkpoint: Failure Modes
- You now know unknown tokens after a restart should trigger RST cleanup rather than silent drops.
- You now know UDP NAT mapping lifetimes vary, so quiet Observe streams may need client-originated keep-alives or re-registration.
- You now know RST and failed CON delivery remove ghost observers; an optional server lease is separate from Max-Age freshness.
This section:
Common Pitfalls
This section: Common Pitfalls
CON messages require an ACK exchange. With RFC 7252 defaults, four retransmissions occur after randomized waits totaling 30–45 seconds; the final acknowledgement wait can extend failure detection to 62–93 seconds, regardless of any fixed packet-loss percentage. Use NON for periodic telemetry where data freshness matters more than guaranteed delivery; reserve CON for actuation commands.
CoAP proxies cache GET responses based on Max-Age option — a sensor returning temperature with Max-Age=60 will serve cached values for 60 seconds even if the physical reading changes. Set Max-Age to match your data freshness requirement, not the default 60 seconds.
A fresh DTLS handshake can dominate a short CoAP exchange; its round trips and latency depend on DTLS version, authentication, loss, and link delay. Reuse an established session where appropriate, or use the resumption mechanism supported by the deployed DTLS version (RFC 9147 for DTLS 1.3).
14.8 Practice Exercises
This section: Practice Exercises
Exercise 1: Observer Registry Design Design an observer registry that supports:
- Maximum 50 observers per resource
- Automatic cleanup of stale observers (> 24 hours)
- Rate limiting: max 1 notification per second per observer
Exercise 2: Bandwidth Calculation A smart building has 200 temperature sensors, each observed by 3 clients (dashboard, HVAC controller, alarm system). Sensors report every time temperature changes by 0.5°C. Calculate:
Read these points as one connected sequence: start with Estimated notifications per hour (assuming 2 significant changes per sensor per hour); then Bandwidth usage with 20-byte notification payloads; and finish with Savings compared to polling every 30 seconds.
- Estimated notifications per hour (assuming 2 significant changes per sensor per hour)
- Bandwidth usage with 20-byte notification payloads
- Savings compared to polling every 30 seconds
Exercise 3: NAT Keep-Alive Strategy Design a client-originated keep-alive strategy for a CoAP deployment where:
- Measured NAT mapping timeout is 45 seconds
- 1,000 client mappings need refreshing
- Aggregate outbound bandwidth is limited to 10 Kbps for keep-alives
What keep-alive interval would you choose and why?
This section:
Checkpoint: Evidence Before Selection
- You now know the chapter’s 100-observer example can show 99% bandwidth reduction when changes are infrequent.
- You now know the vibration example’s 772-day polling figure is a transmit-only charge quotient, and its Observe quotient is much larger under the same assumptions.
- You now know MQTT becomes the better comparison when subscriber counts or continuous streams outgrow direct server push.
This section:
14.9 Concept Relationships
This section: Concept Relationships — Start with message types to see how updates arrive.
How CoAP Observe connects to broader IoT patterns and protocols:
Observe builds on:
Start with message types to see how updates arrive. Then look at the state a server keeps over UDP.
- CoAP Message Types - Uses CON/NON for notifications with sequence number tracking
- UDP Transport - Stateless protocol requiring application-layer state management
Similar patterns in other protocols:
- MQTT Subscriptions - Topic-based pub/sub vs URI-based observe
- WebSocket Server Push - Full-duplex vs observe’s asymmetric push
- Server-Sent Events - HTTP push mechanism for comparison
Observe enables:
- Event-Driven IoT - React to changes instead of polling
- Real-Time Monitoring - Dashboard updates without refresh
- Smart Home Automation - Light sensors triggering actuators instantly
Implementation challenges:
- NAT Traversal - UDP mapping timeouts requiring keepalives
- Sequence Number Management - Detecting out-of-order delivery
- State Synchronization - Client-server observation lifecycle
Performance considerations:
- Energy Optimization - compare CON and NON event charge with the full device duty cycle
- Bandwidth Management - Push vs poll bandwidth comparison
- Scalability Patterns - Server memory per observer (illustrative 50-100 bytes; measure the chosen stack)
14.10 See Also
This section: See Also — Review CoAP message basics before the GET registration method that starts an Observe subscription.
CoAP Core Topics:
Review CoAP message basics before the GET registration method that starts an Observe subscription.
- CoAP Fundamentals - Message types and reliability mechanisms
- CoAP Methods - GET with Observe: 0 option for registration
- CoAP Security - Securing observe notifications with DTLS
Implementation Guides:
Try the Observe code first. Then study how the server tracks subscribers.
- CoAP Observe Implementation - Python and ESP32 code examples
- Observer Registry Design - Server-side state management
- Rate Limiting Strategies - Preventing notification floods
Protocol Comparisons:
- MQTT vs CoAP Observe - Topic subscriptions vs resource observation
- Long Polling vs Observe - HTTP alternative to push notifications
- gRPC Streaming - Modern RPC with bidirectional streaming
Real-World Applications:
Start with the vibration sensor example. Then compare the farm example.
- Industrial IoT Monitoring - Vibration sensors with conditional observe
- Smart Agriculture - Soil moisture notifications on threshold changes
- Building Automation - Temperature updates to thermostats
Specifications & RFCs:
Read the Observe rules first. Then review the base CoAP exchange.
- RFC 7641 - Observe Extension - Official specification
- RFC 7252 Section 5.10 - CoAP base protocol observe hooks
- CoRE WG Documents - Latest IETF working group updates
Debugging & Tools:
Use the Wireshark reference to inspect notifications, then the client tool to send an Observe request.
- Wireshark CoAP Dissector Reference - Analyzing observe sequence numbers
- coap-client CLI Tool - Testing observe from command line
- Californium Proxy - HTTP-CoAP observe bridging
14.11 What’s Next
This section: What’s Next
Now that you understand CoAP server push and the Observe extension, these chapters build directly on your knowledge:
-
CoAP Observe Registration and Freshness Contracts Focus: Token binding, Max-Age freshness, CON liveness checks, sequence ordering, and stale-notification handling Why read it: Turn Observe from a bandwidth-saving pattern into an auditable notification contract that recovers from loss, reordering, NAT expiry, and client restarts.
-
CoAP Advanced Features Focus: Block-wise transfer and large payload handling Why read it: Extend your Observe knowledge to handle firmware-update notifications that exceed a single UDP packet.
-
CoAP API Design Focus: RESTful URI patterns and resource modeling Why read it: Design well-structured observable resources that follow CoAP best practices for naming and content formats.
-
CoAP Security Applications Focus: Securing CoAP with DTLS and OSCORE Why read it: Protect observer registration and notification streams from eavesdropping and injection attacks.
-
CoAP Implementation Labs Focus: Python and ESP32 hands-on examples Why read it: Apply the registry and rate-limiting patterns from this chapter in working code on real hardware.
-
CoAP Message Types Focus: CON vs NON reliability and ACK/RST mechanics Why read it: Understand the reliability layer underpinning Observe notifications and how CON retransmission affects observer cleanup.
-
MQTT Broker and Topics Focus: Topic-based publish-subscribe with a broker Why read it: Compare Observe’s direct server-push model against MQTT’s broker-mediated fan-out for high-subscriber scenarios.
14.12 Summary
This section: Summary — CoAP Observe turns a resource into a subscription-style stream while keeping RESTful resource semantics.
CoAP Observe turns a resource into a subscription-style stream while keeping RESTful resource semantics. Clients register interest, servers send ordered notifications, and both sides must handle sequence numbers, cancellation, and loss.
14.13 Continue Your Route
This section: Continue Your Route — This is the last part of this chapter.
This is the last part of this chapter. Return to CoAP Observe: Subscriptions and Updates or continue from the coap module index.
Find help
3 reviewed answers.
Where should I start in a module?
Open the module page from the sidebar to see its chapters in order. Use the module map to return to a topic you need.
Modules and pathsReviewed
What can I try when a project does not work?
Check the symptom, then work through the diagnostic steps. Keep the error message and the smallest example that reproduces it.
When you are stuckReviewed
How do interactive examples work?
Use the interactive guide to check how examples and simulations work before changing your project.
Interactive examples guideReviewed
