23 CoAP Lab: Message Construction
More
Depth: Practitioner
Progress saves as you read. Mark sections complete as you go; your progress appears in the sidebar.
23.1 Start With the Decision
This section: Start With the Decision — CoAP is a compact request and response protocol.
CoAP is a compact request and response protocol. A packet is one unit sent across a network. In a few bytes, a CoAP packet holds a type that says how to handle delivery. A token links a reply to its request; a code names the request or result; and an ID identifies the message. Build each field before sending the packet.
23.2 Route Overview
This section: Route Overview — This is part 2 of 2.
This is part 2 of 2. Review CoAP Lab: Stack and Trace Contracts for the preceding evidence.
23.3 Learning Objectives
This section: Learning Objectives
- Encode CoAP header fields and token length into message bytes.
- Match tokens and message IDs across replies and retries.
23.4 Chapter Roadmap
This section: Chapter Roadmap
- Try It: CoAP Message Byte Builder
- Checkpoint: ESP32 Packet Evidence
- Response Code Reference
- Knowledge Check: CoAP Response Codes
- Basic Simulator: CoAP UDP Communication
- Interactive Simulator: CoAP-Style UDP Communication
- Learning Points
- Visual Reference: CoAP Message Structure
- Visual: CoAP Message Structure
- Alternative View: CoAP vs HTTP Header Comparison
- Common Mistake: Misunderstanding CoAP Token vs Message ID Correlation
- Try It: Token vs Message ID Correlation Visualizer
- Checkpoint: Correlation Boundary
- Common Pitfalls
- 1. Using Confirmable Messages for Every CoAP Request
- 2. Ignoring CoAP Proxy Caching Semantics
- 3. Forgetting DTLS Session Management
- Label the Diagram
- Order the Steps
- Concept Relationships
- See Also
- Match the Concepts
- What’s Next
- Advanced Lab Consolidation Notes
- Checkpoint: Lab Readiness Evidence
- Summary
- Key Takeaway
This section:
Checkpoint: ESP32 Packet Evidence
- You now know where the ESP32 callback extracts payload bytes and decodes the response-code class and detail.
- You can connect
coap.get(),coap.put(), andcoap.post()calls to resource paths already tested in Python. - You have a byte-level header view that explains what the library sends before the payload leaves the device.
Once packets can be decoded, response codes become the lab’s acceptance language. Use the table below to make server behavior explicit.
23.5 Response Code Reference
This section: Response Code Reference
CoAP response codes follow a Class.Detail format similar to HTTP:
| Code | Meaning | HTTP Equivalent |
|---|---|---|
| 2.01 | Created | 201 Created |
| 2.02 | Deleted | 200 OK (for DELETE) |
| 2.03 | Valid | 304 Not Modified |
| 2.04 | Changed | 200 OK (for PUT) |
| 2.05 | Content | 200 OK (for GET) |
| 4.00 | Bad Request | 400 Bad Request |
| 4.01 | Unauthorized | 401 Unauthorized |
| 4.04 | Not Found | 404 Not Found |
| 4.05 | Method Not Allowed | 405 Method Not Allowed |
| 5.00 | Internal Server Error | 500 Internal Server Error |
# Checking response codes in Python
if response.code.is_successful():
print("Success!")
elif response.code == Code.NOT_FOUND:
print("Resource not found")
elif response.code == Code.BAD_REQUEST:
print("Invalid request format")23.6 Basic Simulator: CoAP UDP Communication
This section: Basic Simulator: CoAP UDP Communication
23.6.1 Step 2: Step through the browser message exchange
- Open the CoAP Message Types Workbench below in the browser and select Separate response.
- Press Step once; the run here displayed
Separate responseandStep 2 of 4in the active scenario readout. - In Current Packet Format, compare the empty ACK’s message ID with the original CON before advancing to the later response.
- Continue to the later CON response and compare its token with the original request; token correlation survives the changed message ID.
- This browser simulation teaches the packet sequence; the localhost UDP bytes in Step 1 provide an independently executed packet exchange.
What This Simulates: ESP32 demonstrating CoAP’s lightweight UDP request/response pattern
Run it: Before you open the Wokwi build, drive the same request/response in the browser workbench below. Step through a Piggybacked CON GET, then a Separate response and a NON request/response, and watch the Current Packet Format and Event Trace show how CON, NON, ACK, and RST differ on the wire. Try Duplicate CON handling and a Reset message to expose the edge cases the serial monitor alone hides, and adjust Token length, Payload bytes, and Uri-Path segments to feel the 4-byte-header economy. The Wokwi steps below then reproduce the same exchange on the ESP32.
CoAP Communication Pattern:
Client (ESP32)
|
| GET /temperature
| 4-byte header over UDP 5683
v
Server (Simulated)
|
| 2.05 Content
| Temp: 23.5C
v
Client receives response
23.6.2 Step 3: Inspect the ESP32 serial demonstration if Wokwi is available
- Open the linked Wokwi project below in a browser and click Start Simulation if the service has capacity.
- In Serial Monitor, find the request path, response code, message type labels, and reported round-trip time.
- Compare those printed fields with the real localhost UDP bytes in Step 1 and the browser workbench sequence in Step 2.
- Preserve the project’s serial transcript, but do not treat a printed CoAP-style line as a Wireshark packet capture unless the sketch actually transmits UDP.
- Wokwi was not run in this lane because the CI quota is exhausted; no ESP32 serial or radio result is claimed here.
The direct project link stays readable on mobile and avoids the stalled local embed seen during visual review. Open it in a new tab to run the ESP32 simulation and inspect the serial monitor output.
23.7 Visual Reference: CoAP Message Structure
This section: Visual Reference: CoAP Message Structure — The remaining checks turn those protocol facts into operational judgment before you reuse this lab pattern in a production prototype.
Before decoding a capture, inspect the layout in Figure 23.1 to locate the fixed boundary and the fields whose lengths are carried inside the message. This prevents options or payload bytes from being mistaken for header fields.
In Figure 23.1, the first four bytes always provide version, type, token length, code, and Message ID. TKL then tells the decoder how many token bytes follow; delta-encoded options continue until the optional 0xFF payload marker, after which the remaining bytes are payload. That order is the lab’s packet-decoding checklist.
The 4-byte fixed header contains:
- Ver: Version (always 1)
- Type: CON(0), NON(1), ACK(2), RST(3)
- TKL: Base-format Token length (0-8 bytes); RFC 8974 defines optional extended tokens
- Code: Method (GET=1, POST=2, PUT=3, DELETE=4) or response code
- Message ID: 16-bit identifier for ACK matching and duplicate detection
The compact field layout matters only when compared on the wire. Inspect Figure 23.2 to distinguish CoAP’s binary application header from the transport setup and textual headers in the HTTP example.
In Figure 23.2, inspect the HTTP side first: request and response metadata are textual, and TCP plus TLS add connection and security exchanges. Then compare the CoAP side’s four-byte fixed header and optional fields over UDP. The lesson is not that every complete CoAP packet is four bytes; it is that CoAP keeps the fixed application framing small and pays only for fields used by that exchange.
The Error: Developers new to CoAP often confuse Message ID (MID) and Token, attempting to match responses to requests using MID alone. This causes failures with Observe notifications and separate responses.
Why It Happens: HTTP developers expect a single request-response correlation mechanism. CoAP has two: MID for duplicate detection and Token for logical request-response matching.
Example of Failure:
# WRONG: Matching by Message ID only
request_mid = 12345
response = await get_response()
if response.message_id == request_mid: # Fails with Observe!
process(response)The Fix: Always use Token for request-response correlation:
# CORRECT: Matching by Token
import secrets
# Client sends request with unique token
request_token = secrets.token_bytes(4) # e.g., 0xAB12CD34
request = Message(code=Code.GET, uri='coap://sensor/temp')
request.token = request_token
# For Observe, server sends multiple responses with DIFFERENT MIDs
# but SAME token as original request
async for response in request_handle.observation:
if response.token == request_token: # Correct correlation
print(f'Update: {response.payload}')Why This Matters:
- Separate Responses: Server sends empty ACK (MID=12345) immediately, then data CON (MID=12346) later. Both share the same Token.
- Observe Notifications: Each notification has a new MID (12347, 12348, 12349…) but the original Token throughout the subscription lifetime.
- Concurrent Requests: Client sends multiple requests simultaneously. Responses may arrive out-of-order. Token uniquely identifies which request each response answers.
Illustrative failure: A client that discards Observe notifications with new Message IDs can lose readings. Correlate notifications by the observation Token; use Message ID for ACK and duplicate handling.
Checkpoint: Correlation Boundary
- You now know the boundary between MID duplicate detection and Token request-response correlation.
- You can explain why Observe notifications keep the original Token even when each notification has a new MID.
- You have a concrete failure pattern to look for when a client drops valid updates from a working server.
The remaining checks turn those protocol facts into operational judgment before you reuse this lab pattern in a production prototype.
Common Pitfalls
This section: Common Pitfalls
CON messages require an ACK exchange. With RFC 7252 defaults, four retransmissions occur after randomized waits totaling 30–45 seconds; the final ACK wait can extend failure detection to 62–93 seconds. For example, even at an illustrative 20% packet-loss rate, loss alone does not determine a fixed delay. Use NON for periodic telemetry where data freshness matters more than guaranteed delivery; reserve CON for actuation commands.
CoAP proxies cache GET responses based on Max-Age option — a sensor returning temperature with Max-Age=60 will serve cached values for 60 seconds even if the physical reading changes. Set Max-Age to match your data freshness requirement, not the default 60 seconds.
A fresh DTLS handshake can dominate a short CoAP exchange; its round trips and latency depend on DTLS version, authentication, loss, and link delay. Reuse an established session where appropriate, or use the resumption mechanism supported by the deployed DTLS version (RFC 9147 for DTLS 1.3).
This section:
This section:
This section:
This section:
23.8 Concept Relationships
This section: Concept Relationships — Start with CoAP fundamentals for the request model, then use the message format to inspect response codes and fields.
This implementation chapter bridges theory to practice:
Foundation Knowledge:
Start with CoAP fundamentals for the request model, then use the message format to inspect response codes and fields.
- CoAP Fundamentals and Architecture - REST principles, message types, UDP transport
- CoAP Message Format - Response codes (2.05, 4.04, etc.), Content-Format options
- CoAP Methods and Patterns - GET/PUT/POST usage, CON vs NON selection
Programming Concepts:
Read the Python guide first. Then compare the Arduino examples.
- Python for IoT - Async/await patterns for aiocoap
- Arduino Programming - ESP32 development basics
- Network Programming - UDP communication patterns
Implementation Details:
- Token matching for request-response correlation
- Resource handler registration (
render_get,render_put) - Observe pattern (
observe=0option, notification loop) - Response code selection (2.01 Created, 2.04 Changed, 2.05 Content)
Next Steps:
Use the advanced lab to exercise Observe and block transfer, then apply the API design guidance to the service interface.
- CoAP Advanced Features Lab - Full Wokwi simulation with Observe/Block
- CoAP API Design - Production-ready best practices
- Security Implementation - Adding DTLS to CoAP servers
23.9 See Also
This section: See Also — Start with the aiocoap library guide, then review Python async programming for the event loop used by the examples.
Python CoAP Resources:
Start with the aiocoap library guide, then review Python async programming for the event loop used by the examples.
- aiocoap Documentation - Official library docs
- Python Async Programming - Asyncio fundamentals
- CoAP.me Public Test Server - Test your client implementations
ESP32 CoAP Resources:
Read the ESP32 framework guide before the CoAP library guide to see where the library runs.
- ESP32 Arduino Core - Framework documentation
- coap-simple Library - Arduino CoAP library
- ESP32 Network Setup - Wi-Fi configuration
Testing and Debugging:
Use Wireshark to inspect CoAP traffic, then use the client tool to send requests you can trace.
- Wireshark CoAP Dissector Reference - Protocol analysis
- Copper Plugin for Firefox - CoAP client/browser
- coap-client CLI Tool - Command-line testing
Alternative Implementations:
Compare the C and JavaScript CoAP libraries when choosing an implementation for the target runtime.
- libcoap (C) - Lightweight C library for embedded systems
- node-coap (JavaScript) - Node.js implementation
- GoCoAP (Go) - High-performance Go library
23.10 What’s Next
This section: What’s Next
23.11 Advanced Lab Consolidation Notes
This section: Advanced Lab Consolidation Notes — For battery labs, write the message policy beside the resource table.
When extending the basic labs into a fuller ESP32-style CoAP server, keep the feature set small enough to test:
| Resource or feature | Lab behavior to prove | Failure mode to observe |
|---|---|---|
/temperature with Observe | Client registers once, then receives updates only when the reading changes | Lost notification, cancelled observation, or stale token |
/firmware with Block2 | Client can fetch a large payload in numbered blocks and resume from the failed block | Restarting the whole transfer after one missing block |
/.well-known/core | Discovery lists only supported resources and content formats | Client assumes hidden or unsupported paths |
Configuration PUT | Idempotent update can be retried without creating duplicate state | Retried POST creates repeated configuration records |
| CON/NON selection | Commands and rare critical alerts use CON; periodic telemetry can use NON when the next value replaces the previous one | High-frequency CON drains batteries, while NON commands lose safety-critical acknowledgements |
For battery labs, write the message policy beside the resource table. A common hybrid is NON for routine telemetry and CON for user actions, safety alerts, firmware blocks, or configuration changes. That preserves the energy benefit of UDP while still proving delivery for messages that cannot be silently lost.
23.11.1 Block Transfer Planning Check
Before using Block2 for firmware, images, or larger JSON payloads, record the block size, total block count, resume behavior, and retry limit. A small planning record prevents the lab from hiding a production risk:
23.11.1.1 Step 5: Record a block transfer and loss response
- In the CoAP Block-Wise Transfer Workbench below, select Lossy field link and press Step.
- The browser run here showed
Block2 2:0/1/256 (value 12)and byte range0-255for the first 256-byte block. - Continue stepping through the loss until the event trace shows which NUM is retried, then check that previously received blocks stay complete.
- Fill the planning table with payload size, negotiated block size, count, retry limit, token, and ETag or version before approving resume.
- This is a browser protocol model; firmware or server acceptance still needs a real constrained-link trace with block numbers and a final checksum.
Run it: Fill this planning record from the block-wise workbench below instead of guessing. Pick a direction (Block2 download or Block1 upload) and a Scenario such as Firmware upload or Sensor log download, choose a Requested block size (SZX), and step the transfer while watching NUM, the M bit, and the Byte range advance — that tells you how many blocks a payload needs. Then run the Lossy field link and Server asks for smaller blocks scenarios and read the Event trace to confirm the client retries only the missing block rather than restarting, which is exactly the resume behavior this check must capture.
| Question | Evidence to capture |
|---|---|
| How many blocks are needed? | Payload bytes divided by negotiated block size, with the final partial block noted |
| What happens after packet loss? | Client retries only the missing block instead of restarting the whole payload |
| How is progress verified? | Server logs block number, token, ETag or version, and final checksum |
| When is Block2 not enough? | If payloads are frequent or very large, move the transfer to a gateway or HTTP/TLS path and keep CoAP for control |
This section:
Checkpoint: Lab Readiness Evidence
- You now know which records to capture for resource discovery, configuration updates, Observe cancellation, Block2 resume behavior, and security setup.
- You can choose NON for replaceable telemetry and CON for commands, alerts, firmware blocks, and configuration changes.
- You have a final checklist for deciding whether the implementation is tested under constrained-network conditions rather than only compiled.
23.12 Summary
This section: Summary — CoAP implementation labs turn protocol rules into testable behavior: resource handlers, confirmable messages, retransmission, Observe notifications, and security setup.
CoAP implementation labs turn protocol rules into testable behavior: resource handlers, confirmable messages, retransmission, Observe notifications, and security setup. The goal is to verify behavior with real requests rather than assuming the stack handles every edge case.
23.13 Key Takeaway
This section: Key Takeaway — A CoAP implementation is ready only when resource paths, content formats, timeout handling, retransmission, Observe cancellation, and security settings have all been exercised under constrained-network conditions.
A CoAP implementation is ready only when resource paths, content formats, timeout handling, retransmission, Observe cancellation, and security settings have all been exercised under constrained-network conditions.
23.14 Continue Your Route
This section: Continue Your Route — This is the last part of this chapter.
This is the last part of this chapter. Return to CoAP Lab: Stack and Trace Contracts or continue from the coap module index.
Find help
3 reviewed answers.
Where should I start in a module?
Open the module page from the sidebar to see its chapters in order. Use the module map to return to a topic you need.
Modules and pathsReviewed
What can I try when a project does not work?
Check the symptom, then work through the diagnostic steps. Keep the error message and the smallest example that reproduces it.
When you are stuckReviewed
How do interactive examples work?
Use the interactive guide to check how examples and simulations work before changing your project.
Interactive examples guideReviewed