Efficiency and Lifecycle: CoAP Observe Reference
More
Depth: Practitioner
Progress saves as you read. Mark sections complete as you go; your progress appears in the sidebar.
The original supporting material follows intact, with knowledge checks moved to the standalone check page.
Start With the Decision
A temperature resource has 100 observers: clients registered to receive updated readings. It changes six times an hour. Would sending those updates cost less than having every client request a reading every ten seconds? First count messages, then add the traffic needed to keep clients reachable.
A network address translator (NAT) keeps a mapping that lets replies reach a client using UDP, a datagram transport. That mapping can expire while the temperature is unchanged. A keep-alive is client-originated traffic sent before expiry. With the calculator’s assumed 45-second lifetime and 0.75 fraction, send every 33.75 seconds: about 106.7 refreshes/hour. At 20 bytes each, 100 separate mappings cost about 213,333 client outbound bytes/hour. These are model inputs, not measured network costs.
Your decision must answer two questions: does Observe still save counted traffic after refresh and recovery, and what happens when the last reading becomes stale? Max-Age is the response option that bounds how long that reading can be treated as fresh; it does not expire the server’s observer entry.
Route Overview
This is part 2 of 2. Review CoAP Observe: Subscriptions and Updates for the preceding evidence.
Learning Objectives
- Calculate a NAT keep-alive interval and its traffic.
- Compare Observe traffic, battery wakes, and client restarts.
Chapter Roadmap
- Interactive Calculator: NAT Keep-Alive Interval
- Pitfall: Mismanaging Observe Tokens Across Client Restarts
- Bandwidth Savings Calculation
- Interactive Calculator: Battery Life Comparison
- Interactive: CoAP Observe Pattern Animation
- Worked Example: Industrial Vibration Monitoring with Observe
- Decision Framework: When to Use CoAP Observe
- Common Mistake: Forgetting to Handle Observer Cleanup After Client Crashes
- Checkpoint: Failure Modes
- Common Pitfalls
- 1. Using Confirmable Messages for Every CoAP Request
- 2. Ignoring CoAP Proxy Caching Semantics
- 3. Forgetting DTLS Session Management
- Label the Diagram
- Order the Steps
- Practice Exercises
- Hands-On Practice
- Checkpoint: Evidence Before Selection
- Concept Relationships
- See Also
- Match the Concepts
- What’s Next
- Summary
The Mistake: Clients generate new random tokens after reboot without deregistering previous observations, causing “ghost subscriptions” where the server continues sending notifications to tokens the client no longer recognizes.
Why It Happens: The Observe pattern uses tokens to match notifications to subscriptions. When a client reboots, it loses its token-to-subscription mapping but the server still has the observer registered.
The Fix: Implement proper token lifecycle management:
- Re-register after reboot: Rebuild observation state with fresh tokens; persist state only if its endpoint context remains valid
- Bind each request safely: Use a fresh observation Token after reboot. For secure CoAP, follow RFC 9175 Section 4.2 to prevent a response being bound to the wrong request; for example, use a per-connection sequence that is not reused until rekeying
- Handle orphaned notifications gracefully: When receiving an unrecognized CON or NON notification, reject it with RST in the appropriate endpoint context
- Server-side lease: If needed, configure a separate observer lease; Max-Age only controls representation freshness
Bandwidth Savings Calculation
Example: Temperature sensor with 100 observers
Continue the opening decision. The sizes below are assumed CoAP message sizes, not packet captures. They omit UDP/IP, security, link framing, acknowledgements, retransmissions, registration and NAT refresh traffic. A Token is the request identifier echoed in responses; an option carries protocol metadata. NON means nonconfirmable, with no acknowledgement; CON means confirmable, requiring an acknowledgement (ACK).
Polling approach (GET every 10 seconds):
- Request: 14 bytes (
header + token + Uri-Path) - Response: 15 bytes (4-byte header + 2-byte Token + 2-byte Content-Format + 1-byte payload marker + 6-byte JSON string
"22.5") - Total:
29 bytes x 100 clients x 6/min x 60 min = 1,044,000 bytes/hour
Observe approach (notify on change, avg 6 changes/hour):
- CoAP header: 4 bytes
- Token: 2 bytes
- Observe option: 3 bytes
- Content-Format: 2 bytes
- Payload marker: 1 byte
- Payload: 6 bytes (JSON string
"22.5", including quotes) - Total:
18 bytes x 100 clients x 6 changes = 10.8 KB/hour
Savings: about 99% of the counted CoAP bytes (1,044,000 vs 10,800 bytes/hour).
Now include the opening example’s refresh budget: 10,800 + 213,333 ≈ 224,133 bytes/hour before registration and recovery. That remains below 1,044,000 in this simplified comparison. It is not a total wire-cost comparison: apply the same network and security accounting to both alternatives and avoid counting refreshes already supplied by normal outbound traffic. Frequent changes or repeated registrations can change the decision.
Scenario: Transfer the temperature decision to 100 motor sensors. A dashboard needs vibration updates. For this teaching scenario only, use category boundaries at 2.5 and 4.5 mm/s; these are not validated machine safety limits. The preserved code treats 4.5 mm/s as an urgent upper-threshold crossing.
Predict first: if only three notifications are sent per motor per day, which costs dominate after startup? Then check whether the code actually enforces that rate.
Comparing polling vs CoAP Observe:
Option A: Polling (GET every 5 seconds):
Client polls: GET coap://motor42.local/vibration every 5 seconds
Request: 16 bytes (CoAP header + token + URI)
Response: 20 bytes (CoAP header + token + payload "2.3")
Per sensor traffic (24 hours):
17,280 requests × 16 bytes = 276,480 bytes
17,280 responses × 20 bytes = 345,600 bytes
Total: 622,080 bytes/day per sensor
Fleet traffic: 622,080 × 100 = 62.2 MB/day
Energy per sensor: 17,280 × 3.0 mJ (CON request-response) = 51.8 J/day
Radio-event energy-only quotient (nominal 18650, 3.7 V, 3,000 mAh): 40 kJ ÷ 51.8 J/day ≈ 772 days
Option B: CoAP Observe (notify on change, routine path at most once/minute):
The preserved calculation block’s “max 1,440 notifications/day” label applies only to the routine path; it is not an overall maximum. Its “only when vibration crosses thresholds” and “3 threshold crossings/day” labels simplify the code: the totals require an assumed measured send rate of three notifications/day, including urgent sends.
Client registers: GET /vibration, Observe: 0 (once at startup)
Registration: 18 bytes (request) + 22 bytes (response with Observe: 1)
Server notifies only when vibration crosses thresholds:
Typical motor: 3 threshold crossings/day (normal ↔ warning ↔ critical)
Notifications: 3 × 20 bytes = 60 bytes/day
Plus max-rate limit: If vibrating continuously, max 1,440 notifications/day
Per sensor traffic (stable operation, 3 changes/day):
Registration: 40 bytes (one-time)
Notifications: 60 bytes/day
Total: 100 bytes on registration day; 60 bytes on later days
Fleet traffic after startup: 60 × 100 sensors = 6,000 bytes/day (vs 62.2 MB with polling)
Energy: 3 notifications × 1.5 mJ = 4.5 mJ/day
Radio-event energy-only quotient: 40,000 J ÷ 0.0045 J/day ≈ 8.9 million days (not a physical battery-life prediction)
Bandwidth savings after startup: (62,208,000 - 6,000) / 62,208,000 ≈ 99.99%
Transmit-event energy reduction: 51.8 J/day ÷ 0.0045 J/day ≈ 11,500×; actual battery-life gain also depends on sleep, sensing, listening, and battery ageing
Implementation with rate limiting: The calculation assumes a measured send rate of three notifications/day, including urgent sends. The code below is illustrative and is not a complete CoAP resource. It sends on a value difference of at least 0.5 mm/s after the routine interval, rather than on every category crossing. Urgent upper-threshold crossings bypass that interval. The preserved code comment “max 1/minute” describes the routine path only; the code provides no overall flood bound. For example, after a notified value of 4.5 mm/s, a routine send of 4.0 mm/s after 60 seconds permits an immediate urgent send of 4.5 mm/s. Repeating that cycle can exceed 1,440 sends/day. Measure actual sends before using the traffic estimate.
class RateLimitedVibrationResource:
def __init__(self):
self.min_notify_interval = 60 # seconds (max 1/minute)
self.threshold_change = 0.5 # mm/s (notify if change ≥ 0.5)
self.critical_alarm_value = 4.5 # mm/s; alarm crossings bypass routine cap
self.last_notify_time = {}
self.last_notified_value = {}
async def notify_observers(self, new_value):
now = time.time()
for observer in self.observers:
last_time = self.last_notify_time.get(observer.token, 0)
last_value = self.last_notified_value.get(observer.token, None)
# Routine changes need both a value delta and the 60 s interval.
significant_change = (
last_value is not None and
abs(new_value - last_value) >= self.threshold_change
)
interval_passed = (now - last_time) >= self.min_notify_interval
urgent_alarm = (
new_value >= self.critical_alarm_value and
(last_value is None or last_value < self.critical_alarm_value)
)
if last_value is None or urgent_alarm or (significant_change and interval_passed):
await self.send_notification(observer, new_value)
self.last_notify_time[observer.token] = now
self.last_notified_value[observer.token] = new_value
Decision: CoAP Observe
Reasoning:
- About 99.99% reduction in counted CoAP bytes after startup if the measured rate is three notifications/motor/day, including urgent sends; include omitted traffic before budgeting cellular backhaul
- Transmit-event energy falls from 51.8 J/day to 4.5 mJ/day under these illustrative assumptions; total battery life also depends on sleep and sensing current
- Urgent upper-threshold crossings bypass the routine cap. This removes waiting for the next poll, but sensing, processing and delivery still add delay
- The routine path is limited to one send/minute per observer; urgent sends bypass it, so this code provides no overall flood bound
Use the temperature calculation as a decision method, then test it against the motor example.
| Evidence to collect | Decision it supports |
|---|---|
| Poll interval, actual notification rate and encoded message sizes | Compare recurring traffic with the same accounting on both sides. |
| Client mappings, measured expiry and outbound traffic | Add only the refresh traffic that normal exchanges do not already provide. |
| Registration and restart counts | Include recovery cost and verify old observer entries are removed. |
| Radio wakes, listening, sensing and sleep current | Test whether fewer messages reduce full-device energy. |
| Observer count, memory and delivery delay under load | Decide whether direct server fan-out meets capacity needs. Compare broker-mediated MQTT if it does not. |
Decision tree:
- Count polling exchanges and expected notifications over the same interval. If the notification stream is frequent, do not assume Observe saves traffic.
- Add registration, refresh, acknowledgements and retries using the chosen transport and security settings. Compare totals before selecting a design.
- Specify a stale-reading rule and verify reachability after quiet periods and client restarts. A low byte count does not establish a fresh dashboard.
- Measure the full device power profile and server capacity. No fixed subscriber count, change-rate percentage or battery target selects the protocol by itself.
Example decisions:
| Application | Provisional decision and test |
|---|---|
| Temperature resource from this page | Observe reduces counted traffic under the stated assumptions; verify refresh and recovery costs. |
| Motor dashboard | Sparse notifications favor Observe in the model; record actual sends because urgent crossings bypass the cap. |
| Continuous accelerometer stream | Compare sustained throughput and delivery needs; Observe’s change-only savings cannot be assumed. |
| Quiet door sensor behind NAT | Include path refresh traffic and test a long quiet period before claiming a saving. |
The Error: Server registers Observe subscriptions but never removes them when clients crash or restart, leading to “ghost observers” that consume memory and network bandwidth sending notifications to unreachable clients.
Why It Happens: CoAP Observe uses tokens to match notifications to requests. When a client crashes and restarts, it may generate a new token. The server can continue sending notifications to the old token; if the restarted client receives an unrecognized notification, it can reject it with RST.
Illustrative worst case without cleanup: An industrial monitoring system with 500 sensors and 20 dashboard clients observing each sensor, assuming every refresh creates a distinct observation and each stale one persists:
Without observer cleanup:
Each browser refresh creates new Observe subscription (new token)
Each old subscription remains active (server doesn't know browser closed)
After 1 week (browsers refresh ~50 times each):
Per sensor: 20 clients × 50 refreshes = 1,000 stale observations
Per sensor: 20 active observations; 1,020 entries total
Fleet: 500,000 stale + 10,000 active = 510,000 entries
Ratio per sensor: 1,000 / 20 = 50:1 stale to active
Notification overhead:
500 sensors × 10 changes/hour × 1,020 observations per sensor = 5.1M notifications/hour
Wasted: 500 sensors × 10 changes/hour × 1,000 stale observations = 5M/hour (98% waste)
CPU and network load depend on message size, transport, and how quickly RSTs are processed; they cannot be inferred from these counts.
The Fix:
1. Optional server-side observer lease (application policy, not Max-Age):
# Max-Age describes client-side representation freshness, not a server-side lease.
observer_lease_seconds = 3600 # This application chooses a one-hour lease.
# Application policy: observer must re-register before its lease expires.
def cleanup_expired_observers(self):
now = time.time()
for resource_uri, observers in self.observers.items():
self.observers[resource_uri] = [
o for o in observers
if now - o.registered_at < o.timeout
]
# Run cleanup periodically (for example, once a minute); defining it alone does nothing.
schedule_periodic(self.cleanup_expired_observers, interval_seconds=60)
2. RST detection (immediate cleanup): The preserved sketch matches address and message ID. A production stack must validate the complete endpoint and security context before accepting a Reset; this sketch is not a complete implementation.
def on_rst_received(self, client_addr, message_id):
"""Remove observer when client sends RST to notification"""
for resource_uri, observers in self.observers.items():
self.observers[resource_uri] = [
o for o in observers
if not (o.client_addr == client_addr and o.last_mid == message_id)
]
logging.info(f"Removed observer {client_addr} after RST")
3. CON notification failures (retry limit): This is transport pseudocode: send_datagram_once sends exactly one datagram and does not perform library-managed retransmission.
async def send_notification(self, resource_uri, observer, value):
"""Send CON notification, remove observer after 4 failed retries"""
msg = Message(code=CONTENT, token=observer.token, payload=value)
msg.mtype = CON # Confirmable - requires ACK
msg.mid = allocate_message_id() # Same MID on every retransmission.
timeout = random.uniform(2.0, 3.0) # RFC 7252 initial ACK timeout.
for attempt in range(5): # original transmission plus four retransmissions
try:
await send_datagram_once(observer.client_addr, msg)
await wait_for_ack(msg.mid, timeout=timeout)
return # Success
except TimeoutError:
logging.warning(f"Notification attempt {attempt+1} failed for {observer}")
timeout *= 2
# No ACK after original send and four retransmissions: remove observer
self.remove_observer(resource_uri, observer)
logging.info(f"Removed unresponsive observer {observer.client_addr}")
Results after implementing cleanup: The preserved estimate below omits sweep delay: with a once-per-minute sweep, an entry can remain until the next sweep after lease expiry, and scheduler delays can extend that. Interpret the one-hour bound as the lease duration, not an exact removal deadline.
With the stated one-hour application lease, stale entries survive at most one hour unless
RST or failed CON delivery removes them earlier. Measure CPU and traffic in the deployment;
these mechanisms alone do not establish percentage savings.
Prevention checklist:
- Set Max-Age to state representation freshness (CoAP default: 60 seconds)
- If needed, define a separate server-side observer lease as application policy
- Implement RST detection and immediate observer removal
- Include CON liveness checks when using NON notifications; NON alone provides no ACK failure signal
- Use at most four retransmissions after the original CON send (RFC 7252 default)
- Log observer registration/removal for debugging
- Monitor observer count per resource (alert if >expected)
Checkpoint: Failure Modes
- You now know unknown tokens after a restart should trigger RST cleanup rather than silent drops.
- You now know UDP NAT mapping lifetimes vary, so quiet Observe streams may need client-originated keep-alives or re-registration.
- You now know RST and failed CON delivery remove ghost observers; an optional server lease is separate from Max-Age freshness.
Common Pitfalls
CON messages require an ACK exchange. With RFC 7252 defaults, four retransmissions occur after randomized waits totaling 30–45 seconds; the final acknowledgement wait can extend failure detection to 62–93 seconds, regardless of any fixed packet-loss percentage. Choose CON when acknowledgement evidence is needed. NON can reduce routine traffic, but an Observe stream using NON still needs CON liveness checks. Neither message type proves that an actuator performed an action.
CoAP proxies cache GET responses based on Max-Age option — a temperature response with Max-Age=60 permits a cached representation to be reused while fresh, even if the physical reading has changed. It does not require the proxy to serve that value for the full interval. Set Max-Age to match your data freshness requirement, not the default 60 seconds.
A fresh DTLS handshake can dominate a short CoAP exchange; its round trips and latency depend on DTLS version, authentication, loss, and link delay. Reuse an established session where appropriate, or use the resumption mechanism supported by the deployed DTLS version (RFC 9147 for DTLS 1.3).
Practice Exercises
Exercise 1: Observer Registry Design
- Sketch a registry with a chosen limit of 50 observers/resource, a separate 24-hour application lease, and a routine cap of one notification/second/observer.
- Trace a client restart and an old-token notification in separate cases: the rejecting RST reaches the server and is validated against the notification, endpoint and security context; the RST is lost; or CON delivery fails after the retry limit. Treat RST as a Reset rejecting a CON or NON notification, not as a new subscription. For the lost-RST case, run a sweep before the old entry’s 24-hour lease deadline, then the first successfully executed sweep after that deadline, without renewing the old lease. Record each removal and its cause.
- Check the expected result: the new observation receives updates when delivery succeeds. Remove the old entry on validated rejection, failed CON delivery, or the first successfully executed sweep after lease expiry. With no validated rejection or failed CON delivery, survival before expiry is expected; restart plus one sweep alone does not establish a fault. Retention after an applicable removal condition indicates a cleanup fault; a missed scheduled sweep indicates a scheduling fault. Max-Age expiry alone should mark a reading stale, not remove the entry.
Exercise 2: Bandwidth Calculation
A building has 200 sensors and three clients per sensor. Assume two significant changes/sensor/hour, 20 payload bytes/notification, and polling every 30 seconds.
- Calculate notification deliveries/hour, then payload bytes/hour.
- Calculate polling response deliveries and payload bytes over the same hour. Assume the same 20-byte payload per response.
- Explain what is missing before calling the difference a wire-bandwidth saving.
Expected result and reason: Observe delivers 200 × 3 × 2 = 1,200 notifications and 24,000 payload bytes/hour. Polling delivers 200 × 3 × 120 = 72,000 responses and 1,440,000 payload bytes/hour. The payload reduction is about 98.3%. Requests, headers, security, refreshes and recovery are omitted. Counting only 400 notifications forgets fan-out to three clients.
Exercise 3: NAT Keep-Alive Strategy
The measured mapping lifetime is 45 seconds. There are 1,000 separate client mappings and a 10 Kbps aggregate outbound refresh budget, meaning 10,000 bits/second in this exercise.
- Apply the calculator’s 0.75 fraction to choose an illustrative interval.
- Calculate the average refresh budget using its assumed 20 outbound bytes/refresh.
- State which measurement could overturn your decision.
Expected result and reason: The interval is 33.75 seconds. Average counted traffic is 1,000 × 20 × 8 / 33.75 ≈ 4,741 bits/second, below the budget. The largest average size allowed at this interval is about 42.2 bytes/refresh. Measure actual outbound packet size and mapping lifetime; omitted headers or bursts may break the budget. This interval is a policy to test, not a universal NAT guarantee.
Checkpoint: Evidence Before Selection
- You now know the chapter’s 100-observer example can show 99% bandwidth reduction when changes are infrequent.
- You now know the vibration example’s 772-day polling figure is a transmit-only charge quotient, and its Observe quotient is much larger under the same assumptions.
- Compare MQTT when measurements show direct server fan-out misses capacity or delivery targets; subscriber count alone is not a protocol cutoff.
Concept Relationships
How CoAP Observe connects to broader IoT patterns and protocols:
Observe builds on:
Start with message types to see how updates arrive. Then look at the state a server keeps over UDP.
- CoAP Message Types - Uses CON/NON for notifications with sequence number tracking
- UDP Transport - Stateless protocol requiring application-layer state management
Similar patterns in other protocols:
- MQTT Subscriptions - Topic-based pub/sub vs URI-based observe
- WebSocket Server Push - Full-duplex vs observe’s asymmetric push
- Server-Sent Events - HTTP push mechanism for comparison
Observe enables:
- Event-Driven IoT - React to changes instead of polling
- Real-Time Monitoring - Dashboard updates without refresh
- Smart Home Automation - Light sensor updates informing actuators, subject to delivery delay and application checks
Implementation challenges:
- NAT Traversal - UDP mapping timeouts requiring keepalives
- Sequence Number Management - Detecting out-of-order delivery
- State Synchronization - Client-server observation lifecycle
Performance considerations:
- Energy Optimization - compare CON and NON event charge with the full device duty cycle
- Bandwidth Management - Push vs poll bandwidth comparison
- Scalability Patterns - Measure server memory per observer in the chosen stack

Maths · guided foundations
Where the 89% Saving and the 772-Day Battery Come From
Check the quantities and assumptions.
See Also
CoAP Core Topics:
Review CoAP message basics before the GET registration method that starts an Observe subscription.
Implementation Guides:
Try the Observe code first. Then study how the server tracks subscribers.
Protocol Comparisons:
- MQTT vs CoAP Observe - Topic subscriptions vs resource observation
- Long Polling vs Observe - HTTP alternative to push notifications
- gRPC Streaming - Modern RPC with bidirectional streaming
Real-World Applications:
Start with the vibration sensor example. Then compare the farm example.
Specifications & RFCs:
Read the Observe rules first. Then review the base CoAP exchange.
Debugging & Tools:
Use the Wireshark reference to inspect notifications, then the client tool to send an Observe request.
A measured NAT mapping lasts forty seconds. The chosen safety fraction is three quarters. What keep-alive interval follows the chapter’s rule?
Thirty seconds, three quarters of the observed 40-second mapping.
Correct. Forty seconds multiplied by three quarters gives a thirty-second interval, before mapping expiry.
A measured NAT mapping lasts 80 seconds. The chosen keep-alive safety fraction is three quarters. What interval follows?
60 seconds
Correct. Take three quarters of the measured lifetime.
A client refreshes 30 NAT mappings every 30 seconds. How many refresh datagrams does it send per minute?
60 datagrams per minute
Correct. Each mapping needs a refresh each interval.
A CoAP client maintains 12 NAT mappings and refreshes each every 20 seconds. How many refresh datagrams are sent per minute?
36 datagrams
Correct. Each mapping needs three refreshes per minute.
What’s Next
Now that you understand CoAP server push and the Observe extension, these chapters build directly on your knowledge:
-
CoAP Observe Registration and Freshness Contracts Focus: Token binding, Max-Age freshness, CON liveness checks, sequence ordering, and stale-notification handling Why read it: Turn Observe from a bandwidth-saving pattern into an auditable notification contract that recovers from loss, reordering, NAT expiry, and client restarts.
-
CoAP Advanced Features Focus: Block-wise transfer and large payload handling Why read it: Extend your Observe knowledge to handle firmware-update notifications that exceed a single UDP packet.
-
CoAP API Design Focus: RESTful URI patterns and resource modeling Why read it: Design well-structured observable resources that follow CoAP best practices for naming and content formats.
-
CoAP Security Applications Focus: Securing CoAP with DTLS and OSCORE Why read it: Protect observer registration and notification streams from eavesdropping and injection attacks.
-
CoAP Implementation Labs Focus: Python and ESP32 hands-on examples Why read it: Apply the registry and rate-limiting patterns from this chapter in working code on real hardware.
-
CoAP Message Types Focus: CON vs NON reliability and ACK/RST mechanics Why read it: Understand the reliability layer underpinning Observe notifications and how CON retransmission affects observer cleanup.
-
MQTT Broker and Topics Focus: Topic-based publish-subscribe with a broker Why read it: Compare Observe’s direct server-push model against MQTT’s broker-mediated fan-out for high-subscriber scenarios.
Summary
Decide with a cost record: count polling exchanges, count notifications to every observer, then add refresh and recovery traffic. The temperature example saves counted bytes under its assumptions; the motor example needs an actual send trace because its urgent notifications bypass the routine cap. Verify freshness and observer cleanup separately. Fewer messages alone establish neither a current reading nor physical battery life.
Continue Your Route
This is the last part of this chapter. Return to CoAP Observe: Subscriptions and Updates or continue from the coap module index.



