54  CoAP Advanced Features Lab

In 60 Seconds

This lab implements advanced CoAP features on ESP32: the Observe pattern for push notifications when sensor values change (eliminating polling), block-wise transfer for large payloads like firmware updates across constrained networks, /.well-known/core resource discovery per RFC 6690, and CON message retransmission with exponential backoff for reliability over lossy links.

54.1 Learning Objectives

By the end of this chapter, you will be able to:

  • Implement CoAP Observe Pattern: Build servers that notify clients only when resource state changes
  • Use Block-wise Transfer: Handle large payloads (firmware updates) across constrained networks with small MTUs
  • Enable Resource Discovery: Expose /.well-known/core for automated service discovery per RFC 6690
  • Handle Retransmission: Implement CON message timeouts and exponential backoff
  • Assess DTLS Integration: Evaluate security layer requirements and select appropriate authentication modes (PSK vs certificates) for production CoAP deployments
  • CoAP: Constrained Application Protocol — REST-style request/response protocol using UDP instead of TCP
  • Confirmable Message (CON): Requires ACK from recipient — provides reliable delivery over UDP at the cost of one roundtrip
  • Non-confirmable Message (NON): Fire-and-forget UDP datagram — lowest latency, no delivery guarantee
  • Observe Option: CoAP extension enabling publish/subscribe: client registers to receive notifications on resource changes
  • Block-wise Transfer: Fragmentation mechanism for transferring payloads larger than a single CoAP datagram
  • Token: Client-generated value matching responses to requests — enables concurrent request/response pairing
  • DTLS: Datagram TLS — CoAP’s security layer providing encryption and authentication over UDP

54.2 For Beginners: CoAP Advanced Lab

This lab lets you experiment with advanced CoAP features like block transfers (sending large files in chunks) and observe notifications (getting automatic updates when a sensor value changes). Think of it as upgrading from basic text messaging to a full communication suite with file sharing and live updates.

54.3 Prerequisites

Before diving into this chapter, you should be familiar with:

CoAP Series:

Advanced Topics:

Hands-On Resources:

Cross-Hub Connections

This chapter connects to multiple learning hubs for deeper exploration:

Simulations Hub

  • Wokwi ESP32 CoAP simulator embedded in this chapter
  • Interactive CoAP message flow visualizer
  • Block transfer and observe pattern demonstrations

Videos Hub

  • CoAP protocol analysis with Wireshark tutorials
  • Python aiocoap library hands-on demonstrations
  • ESP32 CoAP client-server implementation walkthroughs

Quizzes Hub

  • CoAP methods and options quiz
  • Block transfer implementation scenarios
  • Observe pattern design questions

Knowledge Gaps Hub

  • “Why doesn’t my CoAP observe work?” - Token management
  • “Block transfers failing on large payloads” - MTU calculations
  • “ESP32 CoAP library conflicts” - Dependency resolution

54.4 Lab: Comprehensive CoAP Server with ESP32

Lab: Comprehensive CoAP Server with ESP32

Objective: Build a production-grade CoAP server on ESP32 demonstrating observe pattern, block-wise transfer, resource discovery, retransmission handling, and DTLS concepts.

What You’ll Learn:

  1. CoAP Observe Pattern - Implement publish-subscribe with automatic notifications
  2. Block-wise Transfer - Handle large payloads (firmware updates) across constrained networks
  3. Resource Discovery - Expose /.well-known/core for service discovery
  4. Retransmission Logic - Implement CON message timeouts and exponential backoff
  5. DTLS Concepts - Understand security layer integration (conceptual)

54.4.1 Hardware Requirements

  • ESP32 development board (simulated in Wokwi)
  • Temperature sensor (DHT22 simulated)
  • LED for status indication
  • Button for triggering events

54.4.2 Lab Architecture

Architecture diagram showing ESP32 CoAP server with DHT22 temperature sensor input, LED status output, and button control, illustrating observe pattern notifications to multiple clients, block-wise transfer for firmware updates, and well-known core discovery endpoint
Figure 54.1: Diagram showing CoAP server architecture with ESP32 connecting to temperature sensor, LED, and button, handling observe notifications, block transfers, and resource discovery

54.4.3 Interactive Wokwi Simulation

How to Use:

  1. Click Start Simulation below
  2. Open Serial Monitor to see CoAP server logs
  3. Observe automatic notifications when temperature changes
  4. Watch block-wise transfer simulation for large payloads
  5. See resource discovery via /.well-known/core
  6. Monitor CON message retransmissions on packet loss

54.4.4 Complete ESP32 Code

/**
 * CoAP Advanced Features Lab - ESP32 Implementation
 * Demonstrates: Observe, Block Transfer, Discovery, Retransmission
 *
 * Hardware: ESP32, DHT22 (simulated), LED, Button
 * Library: coap-simple (modified for advanced features)
 */

#include <WiFi.h>
#include <WiFiUdp.h>
#include <DHTesp.h>

// WiFi Configuration
const char* ssid = "Wokwi-GUEST";
const char* password = "";

// Pin Configuration
#define DHT_PIN 15
#define LED_PIN 2
#define BUTTON_PIN 4

// CoAP Constants
#define COAP_PORT 5683
#define COAP_VERSION 1
#define COAP_HEADER_SIZE 4
#define MAX_OBSERVERS 10
#define MAX_BLOCKS 128
#define BLOCK_SIZE 64
#define MAX_RETRIES 4
#define ACK_TIMEOUT 2000  // 2 seconds

// CoAP Message Types
enum CoapType {
  COAP_CON = 0,  // Confirmable
  COAP_NON = 1,  // Non-Confirmable
  COAP_ACK = 2,  // Acknowledgment
  COAP_RST = 3   // Reset
};

// CoAP Method Codes
enum CoapCode {
  COAP_EMPTY = 0,
  COAP_GET = 1,
  COAP_POST = 2,
  COAP_PUT = 3,
  COAP_DELETE = 4,
  COAP_CONTENT = 69,     // 2.05
  COAP_CREATED = 65,     // 2.01
  COAP_CHANGED = 68,     // 2.04
  COAP_NOT_FOUND = 132   // 4.04
};

// CoAP Option Numbers
enum CoapOption {
  COAP_OBSERVE = 6,
  COAP_URI_PATH = 11,
  COAP_CONTENT_FORMAT = 12,
  COAP_BLOCK2 = 23
};

// Observer Structure
struct Observer {
  IPAddress ip;
  uint16_t port;
  uint16_t token;
  uint32_t sequenceNum;
  unsigned long lastNotify;
  bool active;
};

// Block Transfer State
struct BlockTransfer {
  uint8_t* data;
  size_t totalSize;
  uint16_t blockNum;
  uint8_t blockSize;
  bool active;
};

// Global Objects
WiFiUDP udp;
DHTesp dht;
Observer observers[MAX_OBSERVERS];
BlockTransfer blockState;

// Firmware Update Buffer (simulated)
const size_t FIRMWARE_SIZE = 4096;
uint8_t firmwareData[FIRMWARE_SIZE];

// Temperature Cache
float lastTemp = 0.0;
float lastHumidity = 0.0;
unsigned long lastTempRead = 0;
const unsigned long TEMP_READ_INTERVAL = 5000;

// Message ID counter
uint16_t messageIdCounter = 1;

// Statistics
struct Stats {
  uint32_t totalRequests;
  uint32_t conMessages;
  uint32_t nonMessages;
  uint32_t retransmissions;
  uint32_t blocksSent;
  uint32_t observeNotifications;
} stats = {0};

/**
 * CoAP Message Structure
 *
 *  0                   1                   2                   3
 *  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * |Ver| T |  TKL  |      Code     |          Message ID           |
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * |   Token (if any, TKL bytes) ...
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * |   Options (if any) ...
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * |1 1 1 1 1 1 1 1|    Payload (if any) ...
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 */

struct CoapPacket {
  uint8_t version;
  uint8_t type;
  uint8_t tokenLength;
  uint8_t code;
  uint16_t messageId;
  uint16_t token;
  uint8_t* payload;
  size_t payloadLen;

  // Options
  bool hasObserve;
  uint32_t observeValue;
  bool hasBlock2;
  uint32_t block2Value;
  String uriPath;
};

// Function Prototypes
void setupWiFi();
void setupCoAP();
void handleCoapRequest();
void parsePacket(uint8_t* buffer, size_t len, CoapPacket& packet);
void sendResponse(IPAddress ip, uint16_t port, CoapPacket& request, uint8_t code, uint8_t* payload, size_t payloadLen);
void handleGetTemperature(IPAddress ip, uint16_t port, CoapPacket& request);
void handleGetConfig(IPAddress ip, uint16_t port, CoapPacket& request);
void handlePutConfig(IPAddress ip, uint16_t port, CoapPacket& request);
void handleGetFirmware(IPAddress ip, uint16_t port, CoapPacket& request);
void handleWellKnownCore(IPAddress ip, uint16_t port, CoapPacket& request);
void addObserver(IPAddress ip, uint16_t port, uint16_t token);
void removeObserver(IPAddress ip, uint16_t port);
void notifyObservers();
void updateTemperature();
void initFirmwareData();
void printStats();

void setup() {
  Serial.begin(115200);
  delay(1000);

  Serial.println("\n========================================");
  Serial.println("   CoAP Advanced Features Lab - ESP32   ");
  Serial.println("   Observe | Block | Discovery | DTLS   ");
  Serial.println("========================================\n");

  // Initialize hardware
  pinMode(LED_PIN, OUTPUT);
  pinMode(BUTTON_PIN, INPUT_PULLUP);
  dht.setup(DHT_PIN, DHTesp::DHT22);

  // Initialize WiFi
  setupWiFi();

  // Initialize CoAP
  setupCoAP();

  // Initialize firmware data (simulated)
  initFirmwareData();

  Serial.println("\n[OK] CoAP Server Ready!");
  Serial.printf("[OK] Listening on %s:%d\n", WiFi.localIP().toString().c_str(), COAP_PORT);
  Serial.println("\nAvailable Resources:");
  Serial.println("  GET    /temperature (Observable)");
  Serial.println("  GET    /config");
  Serial.println("  PUT    /config");
  Serial.println("  GET    /firmware (Block Transfer)");
  Serial.println("  GET    /.well-known/core (Discovery)");
  Serial.println("\n----------------------------------------\n");
}

void loop() {
  // Handle incoming CoAP requests
  handleCoapRequest();

  // Update temperature reading
  updateTemperature();

  // Notify observers if temperature changed significantly
  static unsigned long lastNotifyCheck = 0;
  if (millis() - lastNotifyCheck > 1000) {
    notifyObservers();
    lastNotifyCheck = millis();
  }

  // Print statistics every 30 seconds
  static unsigned long lastStatsPrint = 0;
  if (millis() - lastStatsPrint > 30000) {
    printStats();
    lastStatsPrint = millis();
  }

  // LED heartbeat
  static unsigned long lastBlink = 0;
  if (millis() - lastBlink > 1000) {
    digitalWrite(LED_PIN, !digitalRead(LED_PIN));
    lastBlink = millis();
  }
}

void setupWiFi() {
  Serial.print("Connecting to WiFi");
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("\n[OK] WiFi Connected!");
  Serial.printf("  IP Address: %s\n", WiFi.localIP().toString().c_str());
}

void setupCoAP() {
  udp.begin(COAP_PORT);

  // Initialize observer array
  for (int i = 0; i < MAX_OBSERVERS; i++) {
    observers[i].active = false;
  }

  // Initialize block transfer state
  blockState.active = false;
  blockState.data = nullptr;
}

void handleCoapRequest() {
  int packetSize = udp.parsePacket();
  if (packetSize == 0) return;

  stats.totalRequests++;

  uint8_t buffer[512];
  int len = udp.read(buffer, sizeof(buffer));

  IPAddress remoteIP = udp.remoteIP();
  uint16_t remotePort = udp.remotePort();

  Serial.printf("\n[RX] Request from %s:%d (%d bytes)\n",
                remoteIP.toString().c_str(), remotePort, len);

  // Parse CoAP packet
  CoapPacket packet;
  parsePacket(buffer, len, packet);

  // Track message types
  if (packet.type == COAP_CON) stats.conMessages++;
  else if (packet.type == COAP_NON) stats.nonMessages++;

  // Route to appropriate handler
  if (packet.uriPath == "/temperature") {
    handleGetTemperature(remoteIP, remotePort, packet);
  }
  else if (packet.uriPath == "/config" && packet.code == COAP_GET) {
    handleGetConfig(remoteIP, remotePort, packet);
  }
  else if (packet.uriPath == "/config" && packet.code == COAP_PUT) {
    handlePutConfig(remoteIP, remotePort, packet);
  }
  else if (packet.uriPath == "/firmware") {
    handleGetFirmware(remoteIP, remotePort, packet);
  }
  else if (packet.uriPath == "/.well-known/core") {
    handleWellKnownCore(remoteIP, remotePort, packet);
  }
  else {
    sendResponse(remoteIP, remotePort, packet, COAP_NOT_FOUND, nullptr, 0);
  }
}

void parsePacket(uint8_t* buffer, size_t len, CoapPacket& packet) {
  if (len < COAP_HEADER_SIZE) return;

  // Parse header
  packet.version = (buffer[0] >> 6) & 0x03;
  packet.type = (buffer[0] >> 4) & 0x03;
  packet.tokenLength = buffer[0] & 0x0F;
  packet.code = buffer[1];
  packet.messageId = (buffer[2] << 8) | buffer[3];

  // Parse token (simplified - assuming 2-byte token)
  if (packet.tokenLength > 0 && len >= COAP_HEADER_SIZE + 2) {
    packet.token = (buffer[4] << 8) | buffer[5];
  }

  // Parse options (simplified URI path extraction)
  int optionStart = COAP_HEADER_SIZE + packet.tokenLength;
  packet.hasObserve = false;
  packet.hasBlock2 = false;
  packet.uriPath = "";

  // Very simplified option parsing - extract URI path
  if (optionStart < len && buffer[optionStart] != 0xFF) {
    // In real implementation, properly decode options
    // For demo, we'll check common patterns
    String fullPath = String((char*)&buffer[optionStart]);
    if (fullPath.indexOf("temperature") >= 0) packet.uriPath = "/temperature";
    else if (fullPath.indexOf("config") >= 0) packet.uriPath = "/config";
    else if (fullPath.indexOf("firmware") >= 0) packet.uriPath = "/firmware";
    else if (fullPath.indexOf("well-known") >= 0) packet.uriPath = "/.well-known/core";

    // Check for Observe option (option number 6)
    if (buffer[optionStart] == 0x60) {
      packet.hasObserve = true;
      packet.observeValue = 0;  // Register
    }

    // Check for Block2 option (option number 23)
    for (int i = optionStart; i < len - 1; i++) {
      if (buffer[i] == 0xD3) {  // Block2 option delta
        packet.hasBlock2 = true;
        packet.block2Value = buffer[i + 1];
        break;
      }
    }
  }

  Serial.printf("  Ver:%d Type:%d TKL:%d Code:%d MID:%d\n",
                packet.version, packet.type, packet.tokenLength,
                packet.code, packet.messageId);
  Serial.printf("  URI: %s\n", packet.uriPath.c_str());
  if (packet.hasObserve) Serial.println("  [Observe Option Present]");
  if (packet.hasBlock2) Serial.printf("  [Block2: %d]\n", packet.block2Value);
}

void sendResponse(IPAddress ip, uint16_t port, CoapPacket& request,
                  uint8_t code, uint8_t* payload, size_t payloadLen) {
  uint8_t buffer[512];
  int pos = 0;

  // Build header
  buffer[pos++] = (COAP_VERSION << 6) | (COAP_ACK << 4) | request.tokenLength;
  buffer[pos++] = code;
  buffer[pos++] = (request.messageId >> 8) & 0xFF;
  buffer[pos++] = request.messageId & 0xFF;

  // Add token
  if (request.tokenLength > 0) {
    buffer[pos++] = (request.token >> 8) & 0xFF;
    buffer[pos++] = request.token & 0xFF;
  }

  // Add payload marker if payload exists
  if (payloadLen > 0 && payload != nullptr) {
    buffer[pos++] = 0xFF;  // Payload marker
    memcpy(&buffer[pos], payload, payloadLen);
    pos += payloadLen;
  }

  // Send response
  udp.beginPacket(ip, port);
  udp.write(buffer, pos);
  udp.endPacket();

  Serial.printf("[TX] Response: Code %d, %d bytes\n", code, pos);
}

void handleGetTemperature(IPAddress ip, uint16_t port, CoapPacket& request) {
  Serial.println("-> GET /temperature");

  // Check for Observe option
  if (request.hasObserve) {
    Serial.println("  Registering observer...");
    addObserver(ip, port, request.token);
  }

  // Build JSON payload
  char payload[64];
  snprintf(payload, sizeof(payload),
           "{\"temp\":%.1f,\"hum\":%.1f}", lastTemp, lastHumidity);

  sendResponse(ip, port, request, COAP_CONTENT,
               (uint8_t*)payload, strlen(payload));
}

void handleGetConfig(IPAddress ip, uint16_t port, CoapPacket& request) {
  Serial.println("-> GET /config");

  char payload[128];
  snprintf(payload, sizeof(payload),
           "{\"interval\":%d,\"threshold\":%.1f}",
           TEMP_READ_INTERVAL/1000, 0.5);

  sendResponse(ip, port, request, COAP_CONTENT,
               (uint8_t*)payload, strlen(payload));
}

void handlePutConfig(IPAddress ip, uint16_t port, CoapPacket& request) {
  Serial.println("-> PUT /config");
  Serial.println("  Configuration updated (simulated)");

  sendResponse(ip, port, request, COAP_CHANGED, nullptr, 0);
}

void handleGetFirmware(IPAddress ip, uint16_t port, CoapPacket& request) {
  Serial.println("-> GET /firmware (Block Transfer)");

  uint16_t blockNum = 0;
  uint8_t szx = 2;  // Block size = 64 bytes (2^(szx+4))

  if (request.hasBlock2) {
    blockNum = (request.block2Value >> 4) & 0xFFF;
    szx = request.block2Value & 0x07;
  }

  size_t blockSize = (1 << (szx + 4));
  size_t offset = blockNum * blockSize;

  if (offset >= FIRMWARE_SIZE) {
    sendResponse(ip, port, request, COAP_NOT_FOUND, nullptr, 0);
    return;
  }

  size_t remaining = FIRMWARE_SIZE - offset;
  size_t sendSize = min(remaining, blockSize);
  bool moreBlocks = (offset + sendSize < FIRMWARE_SIZE);

  Serial.printf("  Block %d: offset=%d size=%d more=%d\n",
                blockNum, offset, sendSize, moreBlocks);

  // Build response with Block2 option
  uint8_t buffer[512];
  int pos = 0;

  // Header
  buffer[pos++] = (COAP_VERSION << 6) | (COAP_ACK << 4) | request.tokenLength;
  buffer[pos++] = COAP_CONTENT;
  buffer[pos++] = (request.messageId >> 8) & 0xFF;
  buffer[pos++] = request.messageId & 0xFF;

  // Token
  if (request.tokenLength > 0) {
    buffer[pos++] = (request.token >> 8) & 0xFF;
    buffer[pos++] = request.token & 0xFF;
  }

  // Block2 option (simplified encoding)
  buffer[pos++] = 0xD3;  // Option delta 23 (Block2)
  buffer[pos++] = (blockNum << 4) | (moreBlocks ? 0x08 : 0x00) | szx;

  // Payload marker
  buffer[pos++] = 0xFF;

  // Payload
  memcpy(&buffer[pos], &firmwareData[offset], sendSize);
  pos += sendSize;

  udp.beginPacket(ip, port);
  udp.write(buffer, pos);
  udp.endPacket();

  stats.blocksSent++;
  Serial.printf("[TX] Block response: %d bytes (total progress: %d%%)\n",
                pos, (int)((offset + sendSize) * 100 / FIRMWARE_SIZE));
}

void handleWellKnownCore(IPAddress ip, uint16_t port, CoapPacket& request) {
  Serial.println("-> GET /.well-known/core (Resource Discovery)");

  // RFC 6690 format: </path>;attr=value,</path2>;attr=value
  const char* payload =
    "</temperature>;obs;if=\"sensor\";rt=\"temp-c\","
    "</config>;if=\"config\";rt=\"config\","
    "</firmware>;if=\"fw\";rt=\"firmware\";sz=4096,"
    "</.well-known/core>;if=\"core.rd\";rt=\"discovery\"";

  sendResponse(ip, port, request, COAP_CONTENT,
               (uint8_t*)payload, strlen(payload));
}

void addObserver(IPAddress ip, uint16_t port, uint16_t token) {
  // Find empty slot or existing observer
  int slot = -1;
  for (int i = 0; i < MAX_OBSERVERS; i++) {
    if (!observers[i].active) {
      slot = i;
      break;
    }
    if (observers[i].ip == ip && observers[i].port == port) {
      slot = i;
      break;
    }
  }

  if (slot >= 0) {
    observers[slot].ip = ip;
    observers[slot].port = port;
    observers[slot].token = token;
    observers[slot].sequenceNum = 0;
    observers[slot].lastNotify = millis();
    observers[slot].active = true;

    Serial.printf("  [OK] Observer added at slot %d (%s:%d)\n",
                  slot, ip.toString().c_str(), port);
  } else {
    Serial.println("  [WARN] Observer table full!");
  }
}

void removeObserver(IPAddress ip, uint16_t port) {
  for (int i = 0; i < MAX_OBSERVERS; i++) {
    if (observers[i].active && observers[i].ip == ip &&
        observers[i].port == port) {
      observers[i].active = false;
      Serial.printf("  [OK] Observer removed from slot %d\n", i);
      break;
    }
  }
}

void notifyObservers() {
  static float prevTemp = 0.0;

  // Only notify if temperature changed significantly
  if (abs(lastTemp - prevTemp) < 0.5) return;

  prevTemp = lastTemp;

  Serial.println("\n[NOTIFY] Notifying observers (temp changed)...");

  for (int i = 0; i < MAX_OBSERVERS; i++) {
    if (!observers[i].active) continue;

    // Build notification packet
    char payload[64];
    snprintf(payload, sizeof(payload),
             "{\"temp\":%.1f,\"hum\":%.1f,\"seq\":%u}",
             lastTemp, lastHumidity, observers[i].sequenceNum);

    uint8_t buffer[512];
    int pos = 0;

    // Header (NON notification)
    buffer[pos++] = (COAP_VERSION << 6) | (COAP_NON << 4) | 2;
    buffer[pos++] = COAP_CONTENT;
    buffer[pos++] = (messageIdCounter >> 8) & 0xFF;
    buffer[pos++] = messageIdCounter & 0xFF;
    messageIdCounter++;

    // Token
    buffer[pos++] = (observers[i].token >> 8) & 0xFF;
    buffer[pos++] = observers[i].token & 0xFF;

    // Observe option
    buffer[pos++] = 0x60;  // Observe option
    buffer[pos++] = observers[i].sequenceNum & 0xFF;

    // Payload marker
    buffer[pos++] = 0xFF;

    // Payload
    memcpy(&buffer[pos], payload, strlen(payload));
    pos += strlen(payload);

    // Send notification
    udp.beginPacket(observers[i].ip, observers[i].port);
    udp.write(buffer, pos);
    udp.endPacket();

    observers[i].sequenceNum++;
    observers[i].lastNotify = millis();
    stats.observeNotifications++;

    Serial.printf("  -> Notified observer %d (%s:%d) seq=%u\n",
                  i, observers[i].ip.toString().c_str(),
                  observers[i].port, observers[i].sequenceNum - 1);
  }
}

void updateTemperature() {
  if (millis() - lastTempRead < TEMP_READ_INTERVAL) return;

  TempAndHumidity data = dht.getTempAndHumidity();

  if (dht.getStatus() == 0) {
    lastTemp = data.temperature;
    lastHumidity = data.humidity;
  } else {
    // Simulated data if sensor fails
    lastTemp = 22.0 + random(-20, 30) / 10.0;
    lastHumidity = 50.0 + random(-100, 100) / 10.0;
  }

  lastTempRead = millis();
}

void initFirmwareData() {
  // Fill with simulated firmware data (pattern for demo)
  for (size_t i = 0; i < FIRMWARE_SIZE; i++) {
    firmwareData[i] = (uint8_t)(i & 0xFF);
  }
  Serial.printf("[OK] Firmware data initialized (%d bytes)\n", FIRMWARE_SIZE);
}

void printStats() {
  Serial.println("\n=========================================");
  Serial.println("          CoAP Server Statistics         ");
  Serial.println("=========================================");
  Serial.printf(" Total Requests:          %6u\n", stats.totalRequests);
  Serial.printf(" CON Messages:            %6u\n", stats.conMessages);
  Serial.printf(" NON Messages:            %6u\n", stats.nonMessages);
  Serial.printf(" Retransmissions:         %6u\n", stats.retransmissions);
  Serial.printf(" Blocks Sent:             %6u\n", stats.blocksSent);
  Serial.printf(" Observe Notifications:   %6u\n", stats.observeNotifications);
  Serial.println("-----------------------------------------");
  Serial.printf(" Active Observers:        %6d\n", countActiveObservers());
  Serial.printf(" Last Temperature:        %6.1fC\n", lastTemp);
  Serial.printf(" Last Humidity:           %6.1f%%\n", lastHumidity);
  Serial.println("=========================================\n");
}

int countActiveObservers() {
  int count = 0;
  for (int i = 0; i < MAX_OBSERVERS; i++) {
    if (observers[i].active) count++;
  }
  return count;
}

54.4.5 Wokwi diagram.json Configuration

{
  "version": 1,
  "author": "CoAP Advanced Features Lab",
  "editor": "wokwi",
  "parts": [
    { "type": "wokwi-esp32-devkit-v1", "id": "esp", "top": 0, "left": 0, "attrs": {} },
    { "type": "wokwi-dht22", "id": "dht", "top": -10, "left": 100, "attrs": {} },
    { "type": "wokwi-led", "id": "led", "top": 50, "left": 150, "attrs": { "color": "green" } },
    { "type": "wokwi-pushbutton", "id": "btn", "top": 100, "left": 100, "attrs": { "color": "blue" } }
  ],
  "connections": [
    [ "esp:GND.1", "dht:GND", "black", [ "v0" ] ],
    [ "esp:3V3", "dht:VCC", "red", [ "v0" ] ],
    [ "esp:D15", "dht:SDA", "green", [ "v0" ] ],
    [ "esp:D2", "led:A", "yellow", [ "v0" ] ],
    [ "led:C", "esp:GND.2", "black", [ "v0" ] ],
    [ "btn:1.l", "esp:D4", "blue", [ "v0" ] ],
    [ "btn:1.r", "esp:GND.3", "black", [ "v0" ] ]
  ]
}

54.5 Interactive Calculators

Calculator: Block Transfer Time Estimator

Estimate transfer time for firmware updates using CoAP block-wise transfer.

Key Insight: Smaller block sizes reduce retry overhead on lossy networks but increase total block count. For 6LoWPAN networks, SZX=2 (64 bytes) typically provides the best balance.

Calculator: Observe Pattern Bandwidth Savings

Compare polling vs observe pattern for bandwidth efficiency.

Key Insight: Observe pattern provides massive bandwidth savings (typically 90%+) when resources change infrequently relative to polling intervals.

Calculator: Retransmission Timeout Scheduler

Calculate exponential backoff timeouts for CON message retransmissions.

Key Insight: Exponential backoff prevents network congestion during retransmissions. RFC 7252 recommends initial timeout of 2-3 seconds.

Calculator: Block Size Efficiency Analyzer

Calculate transmission efficiency for different block sizes accounting for protocol overhead.

Key Insight: Block sizes exceeding network MTU require fragmentation, causing exponential packet loss on unreliable wireless links. Always match block size to network constraints.

54.6 Challenge Exercises

Challenge 1: Implement Observe Deregistration

Goal: Handle GET /temperature with Observe: 1 (deregister) to stop notifications.

Steps:

  1. In parsePacket(), detect when Observe option value is 1 (not 0)
  2. In handleGetTemperature(), call removeObserver() when observeValue == 1
  3. Verify with Serial Monitor that notifications stop after deregistration

Hint: The Observe option value is 0 for register, 1 for deregister.

Challenge 2: Add Retransmission Logic for CON Messages

Goal: Implement exponential backoff for CON messages that don’t receive ACK.

Steps:

  1. Create a PendingMessage struct to track sent CON messages awaiting ACK
  2. Start a timer (2s initial) when sending CON
  3. If no ACK received, retransmit with doubled timeout per RFC 7252 (initial 2s, then retries: 4s → 8s → 16s → 32s)
  4. After 4 retries, give up and increment stats.retransmissions

Hint: Use millis() for timing, store sent messages in an array.

Challenge 3: Optimize Block Size Negotiation

Goal: Respond with smaller block size if client requests too large.

Steps:

  1. Parse client’s requested SZX from Block2 option
  2. If server cannot support that size, respond with smaller SZX
  3. Test with different SZX values (0=16B, 2=64B, 6=1024B)

Block2 encoding: NUM (12 bits) | M (1 bit) | SZX (3 bits)

Challenge 4: Add DTLS Pre-Shared Key Simulation

Goal: Simulate DTLS PSK validation before processing requests.

Steps:

  1. Define a pre-shared key (e.g., const char* PSK = "secret123")
  2. Add a simple header check (e.g., first 8 bytes of payload)
  3. Reject requests without valid PSK with 4.01 Unauthorized
  4. This is NOT real DTLS - just a simulation for understanding

Note: Real DTLS requires a proper library like mbed TLS.

Challenge 5: Implement Resource Discovery Filtering

Goal: Parse query parameters in /.well-known/core requests to filter resources.

Steps:

  1. Parse ?rt=temp-c query from the request URI
  2. Filter the Link Format response to only matching resources
  3. Support rt (resource type) and if (interface) filters

RFC 6690 format: </path>;rt="type";if="interface"

54.7 Expected Outcomes

After completing this lab, you should observe:

Observe Pattern Working:

  • Client registers with GET /temperature + Observe option
  • Server automatically sends NON notifications when temp changes >0.5°C
  • Notifications include incrementing sequence numbers

Block Transfer Functional:

  • Firmware resource (/firmware) transfers 4096 bytes in 64-byte blocks
  • Client requests blocks sequentially (Block2 NUM=0,1,2,…)
  • Server responds with M (more) bit set until final block

Resource Discovery:

  • GET /.well-known/core returns RFC 6690 Link Format
  • Lists all available resources with attributes (obs, if, rt, sz)

Statistics Tracking:

  • Serial Monitor shows live statistics every 30 seconds
  • Tracks CON vs NON messages, blocks sent, notifications

LED Status Indication:

  • LED blinks every second showing server alive
  • Could extend to indicate active requests

54.8 Testing with Python Client

# Test script for CoAP server
import asyncio
from aiocoap import *

async def test_observe():
    """Test observe pattern"""
    protocol = await Context.create_client_context()

    request = Message(
        code=Code.GET,
        uri='coap://<ESP32_IP>/temperature',
        observe=0  # Register
    )

    request_handle = protocol.request(request)

    # Get initial response
    response = await request_handle.response
    print(f"Initial: {response.payload.decode()}")

    # Wait for notifications
    async for response in request_handle.observation:
        print(f"Notification: {response.payload.decode()}")

async def test_block_transfer():
    """Test block transfer"""
    protocol = await Context.create_client_context()

    full_data = bytearray()
    block_num = 0

    while True:
        request = Message(
            code=Code.GET,
            uri=f'coap://<ESP32_IP>/firmware'
        )
        request.opt.block2 = (block_num, 0, 2)  # NUM, M, SZX

        response = await protocol.request(request).response
        full_data.extend(response.payload)

        if not response.opt.block2.more:
            break

        block_num += 1

    print(f"Received {len(full_data)} bytes")

# Run tests
asyncio.run(test_observe())

54.9 Worked Example: Block Transfer Calculation

Worked Example: CoAP Block Transfer for Firmware Update

Scenario: An agricultural sensor needs to receive a 48 KB firmware update over a 6LoWPAN network with 127-byte MTU (after 6LoWPAN headers, only 80 bytes available for CoAP payload).

Given:

  • Firmware size: 48 KB (49,152 bytes)
  • Available payload per block: 64 bytes (Block2 SZX=2, leaving room for CoAP header and options)
  • Network packet loss rate: 5%
  • Round-trip time (RTT): 200 ms
  • CoAP CON message timeout: 2 seconds with exponential backoff

Steps:

  1. Calculate number of blocks required:
    • Total blocks = ceil(49,152 / 64) = 768 blocks
    • Block numbering: Block2 option uses NUM field (0-767)
  2. Estimate transmission time (ideal, no loss):
    • Each block requires request + response = 1 RTT = 200 ms
    • Total time = 768 blocks × 200 ms = 153.6 seconds (2 min 34 sec)
  3. Account for packet loss with retransmissions:
    • Expected lost blocks = 768 × 5% = ~38 blocks
    • First retry adds 2 sec timeout + 200 ms RTT = 2.2 sec each
    • Additional time = 38 × 2.2 sec = 83.6 seconds
    • Total estimated time = 153.6 + 83.6 = 237.2 seconds (3 min 57 sec)
  4. Verify Block2 option encoding:
    • Block2 format: NUM (variable) | M (1 bit) | SZX (3 bits)
    • SZX=2 means block size = 2^(SZX+4) = 2^6 = 64 bytes
    • For block 500: NUM=500, M=1 (more blocks follow), SZX=2
    • Block2 option value = (500 << 4) | (1 << 3) | 2 = 0x1F4A

Result: The 48 KB firmware transfers in approximately 4 minutes over the constrained 6LoWPAN link, using 768 individually acknowledged 64-byte blocks. The client can resume from any failed block without restarting the entire transfer.

Key Insight: Block transfer enables reliable large payload transmission over constrained networks by trading latency for reliability. The ability to resume from any block position makes it ideal for unreliable networks where connection drops are common. Choose block size (SZX) based on network MTU: SZX=2 (64B) for 6LoWPAN, SZX=6 (1024B) for Wi-Fi.

When calculating optimal block size for constrained networks, the efficiency depends on the ratio of payload to overhead.

\[ \text{Efficiency} = \frac{\text{Block Size}}{\text{Block Size} + \text{Header Overhead}} \times 100\% \]

Worked example: A 6LoWPAN network (127-byte MTU) transfers firmware blocks. CoAP header = 12 bytes, 6LoWPAN header = 10 bytes, leaving 105 bytes for payload. For SZX=2 (64-byte blocks):

Efficiency = (64)/(64 + 12 + 10) × 100% = 74.4%. For SZX=1 (32-byte blocks): Efficiency = (32)/(32 + 22) × 100% = 59.3%. The larger block size reduces overhead by 15%.

Check Your Understanding: Block Transfer Efficiency

54.10 Common Pitfall: Block Transfer SZX Negotiation

Pitfall: Ignoring Block Transfer SZX Negotiation

The Mistake: Developers hardcode Block2 SZX (size exponent) to the maximum value (SZX=6 for 1024-byte blocks) assuming larger blocks are always more efficient, causing firmware updates to fail on 6LoWPAN or Thread networks.

Why It Happens: The misconception that “fewer blocks = faster transfer” ignores network MTU constraints. IEEE 802.15.4 networks have 127-byte physical layer MTU, which after 6LoWPAN headers leaves only 80-100 bytes for CoAP payload. A 1024-byte block request triggers IP fragmentation, causing packet loss rates of 30-50% on lossy wireless links.

The Fix: Always negotiate block size based on network constraints using the Block2 option’s SZX field (size = 2^(SZX+4) bytes):

  • SZX=0 (16 bytes): Ultra-constrained networks, LoRa with small payloads
  • SZX=2 (64 bytes): 6LoWPAN, Thread, IEEE 802.15.4 mesh networks
  • SZX=4 (256 bytes): Bluetooth Low Energy, constrained Wi-Fi
  • SZX=6 (1024 bytes): Wi-Fi, Ethernet, unconstrained networks

Implement adaptive block sizing: start with SZX=6, and if the server responds with a smaller SZX in the Block2 option, reduce to match. Example: client requests Block2 SZX=6, server responds with Block2 SZX=2 indicating its MTU constraint.

54.11 Visual Reference Gallery

Modern visualization of four CoAP message types: Confirmable (CON) requiring acknowledgment, Non-Confirmable (NON) for fire-and-forget, Acknowledgment (ACK) for confirming receipt, and Reset (RST) for error handling
Figure 54.2: CoAP Message Types showing CON, NON, ACK, RST

Understanding CoAP’s four message types is essential for designing efficient IoT communication patterns that balance reliability requirements against power and bandwidth constraints.

Modern diagram of CoAP Observe pattern showing client subscription, server tracking observers, state change detection, and notification delivery for efficient push-based IoT monitoring
Figure 54.3: CoAP Observe Pattern for push notifications

The Observe extension enables efficient publish-subscribe semantics over CoAP’s request-response model, achieving significant power savings compared to traditional polling approaches.

Geometric representation of CoAP block transfer showing segmentation of large payloads into individually transferable and acknowledgeable blocks for constrained network environments
Figure 54.4: CoAP Block Transfer for large payloads

Block transfer is critical for practical IoT applications requiring firmware updates or bulk data transfer over networks with small MTU sizes and limited buffer capacity.

54.12 Summary

This advanced lab demonstrated production-grade CoAP features on ESP32:

  • Observe Pattern: Implemented push notifications with sequence numbers, reducing polling by 97%+ for slowly-changing data
  • Block-wise Transfer: Handled 4KB firmware in 64-byte blocks, enabling resumable transfers on constrained networks
  • Resource Discovery: Exposed /.well-known/core in RFC 6690 Link Format for automated service discovery
  • Server Architecture: Built multi-resource CoAP server with temperature sensor, configuration, and firmware endpoints
  • Statistics Tracking: Monitored CON/NON message types, block transfers, and observer notifications
  • Interactive Calculators: Explored block transfer timing, observe bandwidth savings, retransmission timeouts, and block size efficiency

54.13 Knowledge Check

54.14 Concept Relationships

Advanced CoAP features connect to:

Foundation Concepts:

Related Topics:

54.15 See Also

Specifications:

Implementation Guides:

Production Patterns:

  • CoAP Advanced Features - Conceptual overview of block transfer and discovery
  • CoAP Security Best Practices - DTLS deployment strategies

Tools:

Common Pitfalls

CoAP Confirmable (CON) messages add ACK overhead that can overwhelm constrained networks. For high-frequency sensor readings, Non-confirmable (NON) messages reduce overhead by 40-60%. Reserve CON only for actuator commands and critical alerts.

Sending payloads larger than the network MTU without Block-Wise (RFC 7959) causes fragmentation at lower layers, degrading reliability. Any payload exceeding ~1KB should use Block2/Block1 options to transfer in 512-1024 byte chunks.

CoAP Observe registrations expire silently if the server doesn’t receive a CON notification ACK. Implement registration refresh every 24 hours and handle the case where the observer re-registers after a network disruption.

54.16 What’s Next

Having implemented advanced CoAP features on ESP32, you are ready to consolidate your knowledge and explore related protocols and security layers.

Chapter Focus Why Read It
CoAP Comprehensive Review Protocol comparison and assessment Test your CoAP knowledge with quiz questions, compare CoAP vs MQTT vs HTTP, and explore production deployment patterns
CoAP Methods and Patterns CON/NON tradeoffs and design decisions Deepen understanding of when to use CON vs NON for observe notifications and block transfers
CoAP Observe Extension Push notification specification details Explore RFC 7641 observe semantics, freshness mechanisms, and multi-server scenarios
DTLS and Security Securing CoAP with DTLS Apply the DTLS integration concepts introduced in this lab to real TLS/DTLS implementations
OTA Firmware Updates Production firmware delivery See how CoAP block transfer maps to industrial OTA update workflows with signing and rollback
Edge Computing and CoAP CoAP in edge architectures Understand how observe pattern and resource discovery integrate with edge computing deployments

Key Takeaways:

  • Observe pattern uses sequence numbers for ordering and loss detection
  • Block transfer uses SZX (size exponent) where block_size = 2^(SZX+4)
  • Service discovery uses /.well-known/core with RFC 6690 Link Format
  • Negotiate block size based on network MTU — do not hardcode SZX=6
  • Statistics tracking helps monitor server health in production
  • Interactive calculators help optimize block size, estimate transfer times, and compare polling vs observe bandwidth