Chapters

Message Construction: CoAP Lab Reference

These preserved exercises accompany CoAP Lab: Message Construction. Knowledge checks open on the standalone check page. The original instructions and examples below are retained for inspection; their execution claims remain bounded as stated.

Start With the Decision

A temperature client requests /temperature and receives 22.5. Decide whether that reply belongs to this request before accepting the reading. Start with two byte strings, decode them, then change the reply token deliberately.

CoAP is a request and response protocol. Its fixed header contains a message type, token length, code, and Message ID (MID). A token is a client-selected byte string used to correlate a response with a request at the expected endpoint. MID identifies a message for acknowledgement and duplicate detection. A confirmable message (CON) requests an acknowledgement (ACK); a non-confirmable message (NON) does not. A reset (RST) rejects a message the recipient cannot process.

Route Overview

This is part 2 of 2. Review CoAP Lab: Stack and Trace Contracts for the preceding evidence.

Learning Objectives

  • Encode CoAP header fields and token length into message bytes.
  • Match tokens and message IDs across replies and retries.

Chapter Roadmap

  • Try It: CoAP Message Byte Builder
  • Checkpoint: ESP32 Packet Evidence
  • Response Code Reference
  • Knowledge Check: CoAP Response Codes
  • Basic Simulator: CoAP UDP Communication
  • Interactive Simulator: CoAP-Style UDP Communication
  • Learning Points
  • Visual Reference: CoAP Message Structure
  • Visual: CoAP Message Structure
  • Alternative View: CoAP vs HTTP Header Comparison
  • Common Mistake: Misunderstanding CoAP Token vs Message ID Correlation
  • Try It: Token vs Message ID Correlation Visualizer
  • Checkpoint: Correlation Boundary
  • Common Pitfalls
  • 1. Using Confirmable Messages for Every CoAP Request
  • 2. Ignoring CoAP Proxy Caching Semantics
  • 3. Forgetting DTLS Session Management
  • Label the Diagram
  • Order the Steps
  • Concept Relationships
  • See Also
  • Match the Concepts
  • What’s Next
  • Advanced Lab Consolidation Notes
  • Checkpoint: Lab Readiness Evidence
  • Summary
  • Key Takeaway
Try It: CoAP Message Byte Builder

Construct a CoAP message byte-by-byte and see exactly how the 4-byte header is encoded. This helps you understand what the coap-simple library constructs behind the scenes when you call coap.get() or coap.put().

Step 1: Encode and inspect a real localhost UDP datagram

The inherited heading’s “real localhost UDP datagram” wording refers to an unperformed network extension. The required procedure here is an offline fixture test.

  1. Use Python 3 with its standard library. The expected output is derived from the packet bytes; it has not been independently executed on a specific Python version. Save the following as coap_bytes.py. These are fixed byte fixtures from the original example; this exercise opens no socket and requires no server.
  2. Run python3 coap_bytes.py. Inspect the first four bytes, then the two token bytes. The request’s bb after the token encodes Uri-Path option delta 11 and length 11; the following eleven bytes spell temperature. The response’s ff marks the payload boundary.
request = bytes.fromhex("42 01 12 34 aa bb bb 74 65 6d 70 65 72 61 74 75 72 65")
reply = bytes.fromhex("62 45 12 34 aa bb ff 32 32 2e 35")

def fields(packet):
    tkl = packet[0] & 15
    return (packet[0] >> 6, (packet[0] >> 4) & 3, tkl,
            f"{packet[1] >> 5}.{packet[1] & 31:02d}",
            int.from_bytes(packet[2:4], "big"), packet[4:4+tkl].hex())

print("request", fields(request))
print("reply", fields(reply))
print("payload", reply[7:].decode("ascii"))
assert request[4:6] == reply[4:6], "token mismatch"
print("token matches")
  1. Expect exactly:
request (1, 0, 2, '0.01', 4660, 'aabb')
reply (1, 2, 2, '2.05', 4660, 'aabb')
payload 22.5
token matches
  1. Change only the reply token from aa bb to aa bc and run again. Expect AssertionError: token mismatch; the matching MID does not repair the wrong token. Restore aa bb and rerun to recover. If the decoded fields differ, check the copied hex bytes before changing the decoder.
  2. In the byte builder below, set version 1, CON, TKL 2, Code 0.01, and MID 4660. Expect header 42 01 12 34; a different MID changes the last two bytes. Reset controls afterward and delete only your own coap_bytes.py when finished.

The decoder handles these fixed fixtures, not arbitrary CoAP input. It does not validate options or perform network correlation. The builder previews only the header; its generated ESP32 line is illustrative and may not name a valid library call for every control combination.

This is a header-field exercise. Some Type and Code combinations in the controls are not valid complete CoAP exchanges; the preview does not generate Token, option, or payload bytes.

Bex guides the message review.Checkpoint: ESP32 Packet Evidence
  • Record the request and reply header bytes, token, and decoded code from Step 1.
  • Verify that changing only the reply token fails the assertion, then restore it.
  • This record proves fixture decoding; an ESP32 packet claim still requires captured traffic from that implementation.

Once packets can be decoded, response codes become the lab’s acceptance language. Use the table below to make server behavior explicit.

Response Code Reference

The fixture response code byte 45 in hexadecimal is decimal 69: class 2 and detail 5, written 2.05 Content. The class occupies the upper three bits and the detail the lower five. HTTP labels below are analogies, not interchangeable wire codes. For transfer, a POST that creates a new reading resource needs 2.01 Created; replacing an existing reading with PUT needs 2.04 Changed.

CodeMeaningApproximate HTTP analogy
2.01Created201 Created
2.02Deleted200 OK (for DELETE)
2.03Valid304 Not Modified
2.04Changed200 OK (for PUT)
2.05Content200 OK (for GET)
4.00Bad Request400 Bad Request
4.01Unauthorized401 Unauthorized
4.04Not Found404 Not Found
4.05Method Not Allowed405 Method Not Allowed
5.00Internal Server Error500 Internal Server Error
# Checking response codes in Python
if response.code.is_successful():
    print("Success!")
elif response.code == Code.NOT_FOUND:
    print("Resource not found")
elif response.code == Code.BAD_REQUEST:
    print("Invalid request format")
Knowledge Check: CoAP Response Codes

Basic Simulator: CoAP UDP Communication

Step 2: Step through the browser message exchange

  1. Open the CoAP Message Types Workbench below in the browser and select Separate response.
  2. Press Step through the exchange and record each packet type, MID, and token. Expect an empty ACK before the later response; if you see only one ACK carrying data, check that you selected Separate response.
  3. In Current Packet Format, compare the empty ACK’s message ID with the original CON before advancing to the later response.
  4. Continue to the later CON response and compare its token with the original request; token correlation survives the changed message ID.
  5. Reset the scenario afterward. This browser model teaches the sequence; Step 1 decodes fixed bytes and establishes no UDP transport result.
Interactive Simulator: CoAP-Style UDP Communication

What This Simulates: ESP32 demonstrating CoAP’s lightweight UDP request/response pattern

Run it: Before you open the Wokwi build, drive the same request/response in the browser workbench below. Step through a Piggybacked CON GET, then a Separate response and a NON request/response, and watch the Current Packet Format and Event Trace show how CON, NON, ACK, and RST differ on the wire. Try Duplicate CON handling and a Reset message to expose the edge cases the serial monitor alone hides, and adjust Token length, Payload bytes, and Uri-Path segments to feel the 4-byte-header economy. The optional Wokwi step asks you to check what the linked project actually implements; a model sequence does not establish an ESP32 result.

Simulation · interactive

CoAP Message Types Workbench

Explore how the system behaves.

CoAP Communication Pattern:

Client (ESP32)
    |
    | GET /temperature
    | 4-byte header over UDP 5683
    v
Server (Simulated)
    |
    | 2.05 Content
    | Temp: 23.5C
    v
Client receives response

Step 3: Inspect the ESP32 serial demonstration if Wokwi is available

  1. Optional, outside the validated offline core: open the linked Wokwi project below in a browser and click Start Simulation if the service has capacity.
  2. In Serial Monitor, find the request path, response code, message type labels, and reported round-trip time.
  3. Compare the printed fields with the Step 1 fixtures and Step 2 sequence. Missing fields mean the serial log alone cannot support that comparison; inspect the sketch and capture actual traffic before claiming an exchange.
  4. Preserve the project’s serial transcript, but do not treat a printed CoAP-style line as a Wireshark packet capture unless the sketch actually transmits UDP.
  5. Stop your simulation after saving the transcript. The linked project and its tool versions have not been verified here; no serial, UDP, radio, or hardware result is claimed.
Learning Points

What You’ll Observe:

  1. UDP Transport - Connectionless, lightweight communication
  2. 4-Byte Headers - Minimal overhead compared to HTTP
  3. Request/Response - RESTful pattern like HTTP GET
  4. Message IDs - Matching ACKs and detecting duplicates
  5. Tokens - Correlating responses to requests
  6. UDP boundary - UDP itself has no connection handshake; a security protocol such as DTLS adds its own setup

Visual Reference: CoAP Message Structure

Visual: CoAP Message Structure

Before decoding a capture, inspect the layout in Figure 1 to locate the fixed boundary and the fields whose lengths are carried inside the message. This prevents options or payload bytes from being mistaken for header fields.

In Figure 1, the first four bytes always provide version, type, token length, code, and Message ID. TKL then tells the decoder how many token bytes follow; delta-encoded options continue until the optional 0xFF payload marker, after which the remaining bytes are payload. That order is the lab’s packet-decoding checklist.

The 4-byte fixed header contains:

  • Ver: Version (always 1)
  • Type: CON(0), NON(1), ACK(2), RST(3)
  • TKL: Base-format Token length (0-8 bytes); RFC 8974 defines optional extended tokens
  • Code: Method (GET=1, POST=2, PUT=3, DELETE=4) or response code
  • Message ID: 16-bit identifier for ACK matching and duplicate detection
Alternative View: CoAP vs HTTP Header Comparison

The compact field layout matters only when compared on the wire. Inspect Figure 2 to distinguish CoAP’s binary application header from the transport setup and textual headers in the HTTP example.

In Figure 2, inspect the HTTP side first: request and response metadata are textual, and TCP plus TLS add connection and security exchanges. Then compare the CoAP side’s four-byte fixed header and optional fields over UDP. The lesson is not that every complete CoAP packet is four bytes; it is that CoAP keeps the fixed application framing small and pays only for fields used by that exchange.

Common Mistake: Misunderstanding CoAP Token vs Message ID Correlation

The Error: Developers new to CoAP often confuse Message ID (MID) and Token, attempting to match responses to requests using MID alone. This causes failures with Observe notifications and separate responses.

Why It Happens: HTTP developers expect a single request-response correlation mechanism. CoAP has two: MID for duplicate detection and Token for logical request-response matching.

Example of Failure:

# WRONG: Matching by Message ID only
request_mid = 12345
response = await get_response()
if response.message_id == request_mid:  # Fails with Observe!
    process(response)

The Fix: Always use Token for request-response correlation:

# CORRECT: Matching by Token
import secrets

# Client sends request with unique token
request_token = secrets.token_bytes(4)  # e.g., 0xAB12CD34
request = Message(code=Code.GET, uri='coap://sensor/temp')
request.token = request_token

# For Observe, server sends multiple responses with DIFFERENT MIDs
# but SAME token as original request
async for response in request_handle.observation:
    if response.token == request_token:  # Correct correlation
        print(f'Update: {response.payload}')

Why This Matters:

  1. Separate Responses: Server sends empty ACK (MID=12345) immediately, then data CON (MID=12346) later. The empty ACK has Code 0.00 and no token; only the later data response repeats the request token.
  2. Observe Notifications: Each notification has a new MID (12347, 12348, 12349…) but the original Token throughout the subscription lifetime.
  3. Concurrent Requests: Client sends multiple requests simultaneously. Responses may arrive out-of-order. Token uniquely identifies which request each response answers.

Illustrative failure: A client that discards Observe notifications with new Message IDs can lose readings. Correlate notifications by the observation Token; use Message ID for ACK and duplicate handling.

Client C sends CON GET /temperature to endpoint S (address and port), MID 0x2000, token c1 d2. S returns an empty ACK, Code 0.00, MID 0x2000, no token; then S sends CON, Code 2.05 Content, MID 0x3000, token c1 d2, payload 21.8. No duplicate or timeout has occurred. Which decision is correct?

Accept the later response; ACK it with MID 0x3000.

The expected endpoint, request token, and 2.05 code identify the temperature response. The empty ACK acknowledges request MID 0x2000; the later CON needs an ACK using its own MID 0x3000.

An active /temperature observation at endpoint S uses token e3 f4. Its last accepted notification had MID 0x4000. The next fresh, nonduplicate CON notification comes from S with Code 2.05, token e3 f4, MID 0x4001 and a newer Observe sequence value. Which rule accepts this notification and rejects unrelated or unsuccessful replies?

Accept this notification; ACK 0x4001. Require S, token e3 f4, a successful content code, and a fresh Observe value.

The changed MID is valid. Endpoint and observation token correlate the response; code and Observe freshness support accepting the reading. ACK the received CON MID.

Try It: Token vs Message ID Correlation Visualizer

See why Token-based matching works but MID-based matching fails. This simulator shows concurrent requests and Observe notifications where MIDs change but Tokens remain constant.

Step 4: Test the correlation key

  1. In the visualizer below, choose two concurrent requests and three Observe notifications, then compare Token matching with MID matching.
  2. A reply must carry the request’s token to identify the application exchange, while an ACK uses the CON’s message ID to acknowledge that particular transmission.
  3. In the Step 1 fixtures, request and reply both use token aa bb and MID 0x1234. That piggybacked response cannot test whether a client handles a new MID.
  4. Use the Separate response browser scenario from Step 2 to see the changed MID while the token still identifies the request.
  5. Expect eight responses: two initial replies and six notifications. Token matching accepts all eight; MID matching accepts only the two initial replies and drops six notifications. If your counts differ, restore two requests and three notifications, then retry. Finish with Token matching selected.
  6. Transfer the temperature case: an empty ACK with MID 0x1234 acknowledges the request; a later CON response with token aa bb and MID 0x1235 answers it. Acknowledge that CON using 0x1235. Reject a response from the expected endpoint with token aa bc, even if its MID matches the request.
Bex guides the message review.Checkpoint: Correlation Boundary
  • You now know the boundary between MID duplicate detection and Token request-response correlation.
  • You can explain why Observe notifications keep the original Token even when each notification has a new MID.
  • You have a concrete failure pattern to look for when a client drops valid updates from a working server.

The remaining checks turn those protocol facts into operational judgment before you reuse this lab pattern in a production prototype.

Common Pitfalls

CON messages require an ACK exchange. With RFC 7252 defaults, four retransmissions occur after randomized waits totaling 30–45 seconds; the final ACK wait can extend failure detection to 62–93 seconds. For example, even at an illustrative 20% packet-loss rate, loss alone does not determine a fixed delay. Use NON when a later sample can replace a missed one. Use CON when message receipt feedback is needed, with a timeout and application-level outcome check; confirmation cannot guarantee delivery or safe actuation.

A cache may reuse a stored GET response while it is fresh. Max-Age specifies its freshness lifetime in seconds; omission defaults to 60. A cached temperature can therefore differ from the current physical reading. Choose Max-Age from the allowed reading age, and inspect response age before diagnosing an unchanged value as a missing notification.

A fresh DTLS handshake can dominate a short CoAP exchange; its round trips and latency depend on DTLS version, authentication, loss, and link delay. Reuse an established session where appropriate, or use the resumption mechanism supported by the deployed DTLS version (RFC 9147 for DTLS 1.3).

Knowledge check · self-test

Knowledge check

Test your understanding.

Knowledge check · self-test

Knowledge check

Test your understanding.

Concept Relationships

This implementation chapter bridges theory to practice:

Foundation Knowledge:

Start with CoAP fundamentals for the request model, then use the message format to inspect response codes and fields.

Programming Concepts:

Read the Python guide first. Then compare the Arduino examples.

Implementation Details:

  • Token matching for request-response correlation
  • Resource handler registration (render_get, render_put)
  • Observe pattern (observe=0 option, notification loop)
  • Response code selection (2.01 Created, 2.04 Changed, 2.05 Content)

Next Steps:

Use the advanced lab to exercise Observe and block transfer, then apply the API design guidance to the service interface.

See Also

Python CoAP Resources:

Start with the aiocoap library guide, then review Python async programming for the event loop used by the examples.

ESP32 CoAP Resources:

Read the ESP32 framework guide before the CoAP library guide to see where the library runs.

Testing and Debugging:

Use Wireshark to inspect CoAP traffic, then use the client tool to send requests you can trace.

Alternative Implementations:

Compare the C and JavaScript CoAP libraries when choosing an implementation for the target runtime.

What’s Next

CoAP Implementation Stack and Trace Contracts

Focus: Stack selection, handler mapping, trace evidence, Token correlation, CON/NON policy, and DTLS session reuse.

Why read it: Convert the lab code into an acceptance contract you can verify with coap-client, Wireshark, and server logs.

CoAP Advanced Features Lab

Focus: Block-wise transfer, Observe with ESP32, and resource discovery via /.well-known/core.

Why read it: Build and run a full Wokwi simulation that extends the patterns from this chapter.

CoAP Methods and Patterns

Focus: Design decisions for GET, POST, PUT, DELETE, and CON vs NON tradeoffs.

Why read it: Deepen your understanding of when and why each method and message type is chosen.

CoAP Fundamentals and Architecture

Focus: Message structure, token/MID fields, and the UDP transport layer.

Why read it: Revisit the protocol internals now that you have seen them in working code.

CoAP API Design

Focus: REST resource modelling, URI design, and Content-Format selection.

Why read it: Apply production best practices to the servers and clients you have built here.

DTLS and Security

Focus: Datagram TLS for CoAP, OSCORE, and pre-shared keys.

Why read it: Secure the CoAP servers you implemented in this chapter for real deployments.

MQTT Implementation Labs

Focus: Broker-based publish/subscribe with Python and ESP32.

Why read it: Compare broker-mediated MQTT patterns against CoAP's direct request/response and Observe.

Advanced Lab Consolidation Notes

When extending the basic labs into a fuller ESP32-style CoAP server, keep the feature set small enough to test:

Resource or featureLab behavior to proveFailure mode to observe
/temperature with ObserveClient registers, receives notifications under the server policy, then cancelsLost notification, cancelled observation, or stale token
/firmware with Block2Client can fetch a large payload in numbered blocks and resume from the failed blockRestarting the whole transfer after one missing block
/.well-known/coreDiscovery lists only supported resources and content formatsClient assumes hidden or unsupported paths
Configuration PUTIdempotent update can be retried without creating duplicate stateRetried POST creates repeated configuration records
CON/NON selectionCommands and rare critical alerts use CON; periodic telemetry can use NON when the next value replaces the previous oneHigh-frequency CON drains batteries, while NON commands lose safety-critical acknowledgements

For battery labs, write the message policy beside the resource table. A common hybrid is NON for routine telemetry and CON for user actions, safety alerts, firmware blocks, or configuration changes. Record ACK receipt and timeout separately from application success. Neither a simulator result nor an ACK proves that an actuator completed an action safely.

Block Transfer Planning Check

Before using Block2 for firmware, images, or larger JSON payloads, record the block size, total block count, resume behavior, and retry limit. A small planning record prevents the lab from hiding a production risk:

Step 5: Record a block transfer and loss response

  1. Start a fresh CoAP Block-Wise Transfer Workbench below (reload it to clear prior progress). Select direction Block2 download, scenario Lossy field link, and requested block size 256 bytes (SZX 4) before pressing Step. If these controls are unavailable, stop and record the mismatch rather than substituting a different run.
  2. Record the first block number (NUM), the more-blocks flag (M), block size, and byte range. For a 256-byte first block, expect byte range 0-255; check the selected block size if it differs.
  3. Continue stepping through the loss until the event trace shows which NUM is retried, then check that previously received blocks stay complete.
  4. Fill the planning table with payload size, negotiated block size, count, retry limit, token, and ETag or version before approving resume.
  5. Record the final assembled length if the transfer completes. If the trace or size differs, reload and restore the same direction, scenario, and size before retrying. Reload afterward to clear transfer progress. This unverified browser model does not establish firmware or server acceptance; that requires a constrained-link trace and final checksum.

Run it: Use the fixed starting state above for the planning record. Record observed retries rather than assuming resume behavior. Treat other directions and scenarios as optional extensions, reloading between runs.

Simulation · interactive

CoAP Block-Wise Transfer Workbench

Explore how the system behaves.

QuestionEvidence to capture
How many blocks are needed?Payload bytes divided by negotiated block size, with the final partial block noted
What happens after packet loss?Record the missing NUM and its retry; reuse completed blocks only while the representation version remains consistent
How is progress verified?Capture block numbers and representation ETag or version, then verify assembled length and final checksum; a token alone does not identify a stable representation
When is Block2 not enough?If payloads are frequent or very large, move the transfer to a gateway or HTTP/TLS path and keep CoAP for control

Bex guides the message review.Checkpoint: Lab Readiness Evidence
  • You now know which records to capture for resource discovery, configuration updates, Observe cancellation, Block2 resume behavior, and security setup.
  • You can choose NON for replaceable telemetry and CON for commands, alerts, firmware blocks, and configuration changes.
  • You have a final checklist for deciding whether the implementation is tested under constrained-network conditions rather than only compiled.

Summary

The temperature fixtures establish header decoding and a deliberate token-mismatch failure. The transfer case separates an empty ACK from a later response: MID identifies the acknowledged message; token identifies the request answered. Record model results separately from traffic captured from a running implementation.

Key Takeaway

Accept a reading only after checking the expected endpoint, token, and response code. Use MID for acknowledgement and duplicate handling. Carry that distinction into separate responses and observations before testing a real network implementation.

Continue Your Route

This is the last part of this chapter. Return to CoAP Lab: Stack and Trace Contracts or continue from the coap module index.

Detail notes moved from the chapter

Historical protected source blocks retained for the chapter preservation check.

CoAP Implementation Stack and Trace Contracts

Focus: Stack selection, handler mapping, trace evidence, Token correlation, CON/NON policy, and DTLS session reuse.

Why read it: Convert the lab code into an acceptance contract you can verify with coap-client, Wireshark, and server logs.

CoAP Advanced Features Lab

Focus: Block-wise transfer, Observe with ESP32, and resource discovery via /.well-known/core.

Why read it: Build and run a full Wokwi simulation that extends the patterns from this chapter.

CoAP Methods and Patterns

Focus: Design decisions for GET, POST, PUT, DELETE, and CON vs NON tradeoffs.

Why read it: Deepen your understanding of when and why each method and message type is chosen.

CoAP Fundamentals and Architecture

Focus: Message structure, token/MID fields, and the UDP transport layer.

Why read it: Revisit the protocol internals now that you have seen them in working code.

CoAP API Design

Focus: REST resource modelling, URI design, and Content-Format selection.

Why read it: Apply production best practices to the servers and clients you have built here.

DTLS and Security

Focus: Datagram TLS for CoAP, OSCORE, and pre-shared keys.

Why read it: Secure the CoAP servers you implemented in this chapter for real deployments.

MQTT Implementation Labs

Focus: Broker-based publish/subscribe with Python and ESP32.

Why read it: Compare broker-mediated MQTT patterns against CoAP's direct request/response and Observe.

CoAP message format diagram showing the fixed header bit fields (Ver, Type, TKL, Code, Message ID), token, options, payload marker, and payload.
Locate the four-byte header, then use TKL to find the token boundary before decoding options or payload.
Side-by-side comparison of CoAP and HTTP protocol efficiency: HTTP/1.1 textual request and response metadata plus TCP connection setup versus CoAP's compact binary header and datagram transport; the diagram compares framing and setup rather than byte totals.
Compare the CoAP fixed header with the HTTP/1.1 framing and connection setup shown; complete packet size also includes optional fields and transport headers.

Hi, I’m Broker Bex. What do you need?

Find help

Let’s find what you need

Find a lesson or return to your module guide.

Give feedback

Help improve this chapter

Tell us what helped or what needs fixing.

Tips

Try a small next step

A few ways to get more from this chapter.

Ask AI

Continue the conversation

Take the conversation further with your preferred assistant.

AI help is available in supported modules when enabled in Settings.

No live person answers on independent study. Feedback does not promise a personal reply.

For welfare or safety concerns, follow your school’s procedures outside IoTClass.