23 CoAP Lab: Message Construction
23.1 Start With the Decision
A CoAP packet packs its type, token, code, and ID into a few bytes. Build each field before sending it.
23.2 Route Overview
This is part 2 of 2. Review CoAP Lab: Stack and Trace Contracts for the preceding evidence.
23.3 Learning Objectives
- Encode CoAP header fields and token length into message bytes.
- Match tokens and message IDs across replies and retries.
23.4 Chapter Roadmap
- Try It: CoAP Message Byte Builder
- Checkpoint: ESP32 Packet Evidence
- Response Code Reference
- Knowledge Check: CoAP Response Codes
- Basic Simulator: CoAP UDP Communication
- Interactive Simulator: CoAP-Style UDP Communication
- Learning Points
- Visual Reference: CoAP Message Structure
- Visual: CoAP Message Structure
- Alternative View: CoAP vs HTTP Header Comparison
- Common Mistake: Misunderstanding CoAP Token vs Message ID Correlation
- Try It: Token vs Message ID Correlation Visualizer
- Checkpoint: Correlation Boundary
- Common Pitfalls
- 1. Using Confirmable Messages for Every CoAP Request
- 2. Ignoring CoAP Proxy Caching Semantics
- 3. Forgetting DTLS Session Management
- Label the Diagram
- Order the Steps
- Concept Relationships
- See Also
- Match the Concepts
- What’s Next
- Advanced Lab Consolidation Notes
- Checkpoint: Lab Readiness Evidence
- Summary
- Key Takeaway
Checkpoint: ESP32 Packet Evidence
- You now know where the ESP32 callback extracts payload bytes and decodes the response-code class and detail.
- You can connect
coap.get(),coap.put(), andcoap.post()calls to resource paths already tested in Python. - You have a byte-level header view that explains what the library sends before the payload leaves the device.
Once packets can be decoded, response codes become the lab’s acceptance language. Use the table below to make server behavior explicit.
23.5 Response Code Reference
CoAP response codes follow a Class.Detail format similar to HTTP:
| Code | Meaning | HTTP Equivalent |
|---|---|---|
| 2.01 | Created | 201 Created |
| 2.02 | Deleted | 200 OK (for DELETE) |
| 2.03 | Valid | 304 Not Modified |
| 2.04 | Changed | 200 OK (for PUT) |
| 2.05 | Content | 200 OK (for GET) |
| 4.00 | Bad Request | 400 Bad Request |
| 4.01 | Unauthorized | 401 Unauthorized |
| 4.04 | Not Found | 404 Not Found |
| 4.05 | Method Not Allowed | 405 Method Not Allowed |
| 5.00 | Internal Server Error | 500 Internal Server Error |
# Checking response codes in Python
if response.code.is_successful():
print("Success!")
elif response.code == Code.NOT_FOUND:
print("Resource not found")
elif response.code == Code.BAD_REQUEST:
print("Invalid request format")23.6 Basic Simulator: CoAP UDP Communication
What This Simulates: ESP32 demonstrating CoAP’s lightweight UDP request/response pattern
Run it: Before you open the Wokwi build, drive the same request/response in the browser workbench below. Step through a Piggybacked CON GET, then a Separate response and a NON request/response, and watch the Current Packet Format and Event Trace show how CON, NON, ACK, and RST differ on the wire. Try Duplicate CON handling and a Reset message to expose the edge cases the serial monitor alone hides, and adjust Token length, Payload bytes, and Uri-Path segments to feel the 4-byte-header economy. The Wokwi steps below then reproduce the same exchange on the ESP32.
CoAP Communication Pattern:
Client (ESP32)
|
| GET /temperature
| 4-byte header over UDP 5683
v
Server (Simulated)
|
| 2.05 Content
| Temp: 23.5C
v
Client receives response
How to Use:
- Click Start Simulation
- Watch Serial Monitor show UDP request/response cycle
- Observe message types (CON, NON, ACK)
- See CoAP-style resource addressing
- Monitor round-trip times (RTT)
The direct project link stays readable on mobile and avoids the stalled local embed seen during visual review. Open it in a new tab to run the ESP32 simulation and inspect the serial monitor output.
23.7 Visual Reference: CoAP Message Structure
Before decoding a capture, inspect the layout in Figure 23.1 to locate the fixed boundary and the fields whose lengths are carried inside the message. This prevents options or payload bytes from being mistaken for header fields.
Read Figure 23.1 from left to right. The first four bytes always provide version, type, token length, code, and Message ID. TKL then tells the decoder how many token bytes follow; delta-encoded options continue until the optional 0xFF payload marker, after which the remaining bytes are payload. That order is the lab’s packet-decoding checklist.
The 4-byte fixed header contains:
- Ver: Version (always 1)
- Type: CON(0), NON(1), ACK(2), RST(3)
- TKL: Token length (0-8 bytes)
- Code: Method (GET=1, POST=2, PUT=3, DELETE=4) or response code
- Message ID: 16-bit identifier for matching requests/responses
The compact field layout matters only when compared on the wire. Inspect Figure 23.2 to distinguish CoAP’s binary application header from the transport setup and textual headers in the HTTP example.
In Figure 23.2, inspect the HTTP side first: request and response metadata are textual, and TCP plus TLS add connection and security exchanges. Then compare the CoAP side’s four-byte fixed header and optional fields over UDP. The lesson is not that every complete CoAP packet is four bytes; it is that CoAP keeps the fixed application framing small and pays only for fields used by that exchange.
The Error: Developers new to CoAP often confuse Message ID (MID) and Token, attempting to match responses to requests using MID alone. This causes failures with Observe notifications and separate responses.
Why It Happens: HTTP developers expect a single request-response correlation mechanism. CoAP has two: MID for duplicate detection and Token for logical request-response matching.
Example of Failure:
# WRONG: Matching by Message ID only
request_mid = 12345
response = await get_response()
if response.message_id == request_mid: # Fails with Observe!
process(response)The Fix: Always use Token for request-response correlation:
# CORRECT: Matching by Token
import secrets
# Client sends request with unique token
request_token = secrets.token_bytes(4) # e.g., 0xAB12CD34
request = Message(code=Code.GET, uri='coap://sensor/temp')
request.token = request_token
# For Observe, server sends multiple responses with DIFFERENT MIDs
# but SAME token as original request
async for response in request_handle.observation:
if response.token == request_token: # Correct correlation
print(f'Update: {response.payload}')Why This Matters:
- Separate Responses: Server sends empty ACK (MID=12345) immediately, then data CON (MID=12346) later. Both share the same Token.
- Observe Notifications: Each notification has a new MID (12347, 12348, 12349…) but the original Token throughout the subscription lifetime.
- Concurrent Requests: Client sends multiple requests simultaneously. Responses may arrive out-of-order. Token uniquely identifies which request each response answers.
Real Production Impact: A building automation system lost 30% of sensor readings because the client discarded Observe notifications with “mismatched” MIDs. After fixing to use Token matching, all notifications were correctly correlated.
Checkpoint: Correlation Boundary
- You now know the boundary between MID duplicate detection and Token request-response correlation.
- You can explain why Observe notifications keep the original Token even when each notification has a new MID.
- You have a concrete failure pattern to look for when a client drops valid updates from a working server.
The remaining checks turn those protocol facts into operational judgment before you reuse this lab pattern in a production prototype.
Common Pitfalls
CON messages require an ACK roundtrip — on lossy networks with 20% packet loss, a 4-attempt retry with exponential backoff can delay responses by 45 seconds. Use NON for periodic telemetry where data freshness matters more than guaranteed delivery; reserve CON for actuation commands.
CoAP proxies cache GET responses based on Max-Age option — a sensor returning temperature with Max-Age=60 will serve cached values for 60 seconds even if the physical reading changes. Set Max-Age to match your data freshness requirement, not the default 60 seconds.
DTLS handshake (6-8 roundtrips) dominates latency for short-lived CoAP connections — repeatedly creating new DTLS sessions for each request adds 500-2000 ms overhead. Use DTLS session resumption (RFC 5077) to reduce reconnection to 1 roundtrip after the initial handshake.
23.8 Concept Relationships
This implementation chapter bridges theory to practice:
Foundation Knowledge:
Read these points as one connected sequence: start with CoAP Fundamentals and Architecture - REST principles, message types, UDP transport; then CoAP Message Format - Response codes (2.05, 4.04, etc.), Content-Format options; and finish with CoAP Methods and Patterns - GET/PUT/POST usage, CON vs NON selection.
- CoAP Fundamentals and Architecture - REST principles, message types, UDP transport
- CoAP Message Format - Response codes (2.05, 4.04, etc.), Content-Format options
- CoAP Methods and Patterns - GET/PUT/POST usage, CON vs NON selection
Programming Concepts:
Read these points as one connected sequence: start with Python for IoT - Async/await patterns for aiocoap; then Arduino Programming - ESP32 development basics; and finish with Network Programming - UDP communication patterns.
- Python for IoT - Async/await patterns for aiocoap
- Arduino Programming - ESP32 development basics
- Network Programming - UDP communication patterns
Implementation Details:
Read these points as one connected sequence: start with Token matching for request-response correlation; then Resource handler registration (render_get, render_put); then Observe pattern (observe=0 option, notification loop); and finish with Response code selection (2.01 Created, 2.04 Changed, 2.05 Content).
- Token matching for request-response correlation
- Resource handler registration (
render_get,render_put) - Observe pattern (
observe=0option, notification loop) - Response code selection (2.01 Created, 2.04 Changed, 2.05 Content)
Next Steps:
Read these points as one connected sequence: start with CoAP Advanced Features Lab - Full Wokwi simulation with Observe/Block; then CoAP API Design - Production-ready best practices; and finish with Security Implementation - Adding DTLS to CoAP servers.
- CoAP Advanced Features Lab - Full Wokwi simulation with Observe/Block
- CoAP API Design - Production-ready best practices
- Security Implementation - Adding DTLS to CoAP servers
23.9 See Also
Python CoAP Resources:
Read these points as one connected sequence: start with aiocoap Documentation - Official library docs; then Python Async Programming - Asyncio fundamentals; and finish with CoAP.me Public Test Server - Test your client implementations.
- aiocoap Documentation - Official library docs
- Python Async Programming - Asyncio fundamentals
- CoAP.me Public Test Server - Test your client implementations
ESP32 CoAP Resources:
Read these points as one connected sequence: start with ESP32 Arduino Core - Framework documentation; then coap-simple Library - Arduino CoAP library; and finish with ESP32 Network Setup - Wi-Fi configuration.
- ESP32 Arduino Core - Framework documentation
- coap-simple Library - Arduino CoAP library
- ESP32 Network Setup - Wi-Fi configuration
Testing and Debugging:
Read these points as one connected sequence: start with Wireshark CoAP Dissector Reference - Protocol analysis; then Copper Plugin for Firefox - CoAP client/browser; and finish with coap-client CLI Tool - Command-line testing.
- Wireshark CoAP Dissector Reference - Protocol analysis
- Copper Plugin for Firefox - CoAP client/browser
- coap-client CLI Tool - Command-line testing
Alternative Implementations:
Read these points as one connected sequence: start with libcoap (C) - Lightweight C library for embedded systems; then node-coap (JavaScript) - Node.js implementation; and finish with GoCoAP (Go) - High-performance Go library.
- libcoap (C) - Lightweight C library for embedded systems
- node-coap (JavaScript) - Node.js implementation
- GoCoAP (Go) - High-performance Go library
23.10 What’s Next
23.11 Advanced Lab Consolidation Notes
When extending the basic labs into a fuller ESP32-style CoAP server, keep the feature set small enough to test:
| Resource or feature | Lab behavior to prove | Failure mode to observe |
|---|---|---|
/temperature with Observe | Client registers once, then receives updates only when the reading changes | Lost notification, cancelled observation, or stale token |
/firmware with Block2 | Client can fetch a large payload in numbered blocks and resume from the failed block | Restarting the whole transfer after one missing block |
/.well-known/core | Discovery lists only supported resources and content formats | Client assumes hidden or unsupported paths |
Configuration PUT | Idempotent update can be retried without creating duplicate state | Retried POST creates repeated configuration records |
| CON/NON selection | Commands and rare critical alerts use CON; periodic telemetry can use NON when the next value replaces the previous one | High-frequency CON drains batteries, while NON commands lose safety-critical acknowledgements |
For battery labs, write the message policy beside the resource table. A common hybrid is NON for routine telemetry and CON for user actions, safety alerts, firmware blocks, or configuration changes. That preserves the energy benefit of UDP while still proving delivery for messages that cannot be silently lost.
23.11.1 Block Transfer Planning Check
Before using Block2 for firmware, images, or larger JSON payloads, record the block size, total block count, resume behavior, and retry limit. A small planning record prevents the lab from hiding a production risk:
Run it: Fill this planning record from the block-wise workbench below instead of guessing. Pick a direction (Block2 download or Block1 upload) and a Scenario such as Firmware upload or Sensor log download, choose a Requested block size (SZX), and step the transfer while watching NUM, the M bit, and the Byte range advance — that tells you how many blocks a payload needs. Then run the Lossy field link and Server asks for smaller blocks scenarios and read the Event trace to confirm the client retries only the missing block rather than restarting, which is exactly the resume behavior this check must capture.
| Question | Evidence to capture |
|---|---|
| How many blocks are needed? | Payload bytes divided by negotiated block size, with the final partial block noted |
| What happens after packet loss? | Client retries only the missing block instead of restarting the whole payload |
| How is progress verified? | Server logs block number, token, ETag or version, and final checksum |
| When is Block2 not enough? | If payloads are frequent or very large, move the transfer to a gateway or HTTP/TLS path and keep CoAP for control |
Checkpoint: Lab Readiness Evidence
- You now know which records to capture for resource discovery, configuration updates, Observe cancellation, Block2 resume behavior, and security setup.
- You can choose NON for replaceable telemetry and CON for commands, alerts, firmware blocks, and configuration changes.
- You have a final checklist for deciding whether the implementation is tested under constrained-network conditions rather than only compiled.
23.12 Summary
CoAP implementation labs turn protocol rules into testable behavior: resource handlers, confirmable messages, retransmission, Observe notifications, and security setup. The goal is to verify behavior with real requests rather than assuming the stack handles every edge case.
23.13 Key Takeaway
A CoAP implementation is ready only when resource paths, content formats, timeout handling, retransmission, Observe cancellation, and security settings have all been exercised under constrained-network conditions.
23.14 Continue Your Route
This final part closes the route from Try It: CoAP Message Byte Builder through Key Takeaway. Return to CoAP Lab: Stack and Trace Contracts or continue from the coap module index.