1222  CoAP Observe Extension: Server Push

1222.1 Learning Objectives

By the end of this chapter, you will be able to:

• Implement Server Push: Build CoAP Observe for real-time notifications
• Manage Observer Lifecycle: Handle registration, notifications, and deregistration
• Optimize Notification Frequency: Balance freshness with bandwidth and battery
• Compare with Polling: Quantify Observe benefits over traditional polling

1222.2 Prerequisites

Before diving into this chapter, you should be familiar with:

• CoAP Introduction - CoAP basics and design goals
• CoAP Message Types - CON vs NON and reliability patterns

1222.3 The Observe Extension (RFC 7641)

TipMinimum Viable Understanding: CoAP Observe

Core Concept: Observe transforms CoAP from pure request-response into a publish-subscribe pattern. A client registers interest in a resource once, then receives automatic notifications whenever the resource changes - no repeated polling needed.

Why It Matters: Polling wastes energy and bandwidth. If you need temperature updates every 10 seconds, polling requires 8,640 GET requests/day. With Observe, the server pushes only when values change, potentially reducing traffic by 90%+ for slowly-changing resources.

Key Takeaway: Register with Observe: 0 in your GET request, receive notifications with incrementing sequence numbers, and deregister with Observe: 1 or RST when done.

1222.3.1 Traditional Polling vs. Observe

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22', 'tertiaryColor': '#7F8C8D'}}}%%
sequenceDiagram
    participant C as Client
    participant S as Server

    rect rgb(255, 235, 235)
        Note over C,S: POLLING (inefficient)
        C->>S: GET /temperature
        S->>C: 2.05 "23.5"
        Note over C: wait 60 seconds...
        C->>S: GET /temperature
        S->>C: 2.05 "23.6"
        Note over C,S: Repeat every 60 seconds<br/>(wasteful traffic)
    end

    rect rgb(235, 255, 235)
        Note over C,S: OBSERVE (efficient)
        C->>S: GET /temperature (Observe: 0)
        Note over C,S: Client registers for notifications
        S->>C: 2.05 "23.5" (Observe: 1)
        Note over S: ...wait for change...
        S->>C: 2.05 "23.8" (Observe: 2)
        Note over C,S: Server notifies only on change
    end

Traffic comparison for 24 hours of temperature monitoring:

Approach	Messages	Bandwidth	Battery Impact
Polling (every 60s)	2,880 requests + 2,880 responses	~200 KB	High
Observe (10 changes/hour)	1 registration + 240 notifications	~8 KB	96% less

1222.4 Observe Protocol Flow

1222.4.1 Registration

Client sends GET with Observe: 0 to register:

Client -> Server: GET coap://sensor.local/temperature
                  Token: 0xAB12
                  Observe: 0  (register)
                  Accept: text/plain

Server -> Client: 2.05 Content
                  Token: 0xAB12
                  Observe: 1  (sequence number)
                  Max-Age: 60
                  Payload: "23.5"

1222.4.2 Notifications

Server pushes updates when resource changes:

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22'}}}%%
sequenceDiagram
    participant C as Client
    participant S as Server

    Note over S: Temperature changes to 24.1C
    S->>C: NON 2.05 Content<br/>Token: 0xAB12<br/>Observe: 2<br/>Payload: "24.1"

    Note over S: Temperature changes to 24.8C
    S->>C: NON 2.05 Content<br/>Token: 0xAB12<br/>Observe: 3<br/>Payload: "24.8"

    Note over S: Critical threshold exceeded!
    S->>C: CON 2.05 Content<br/>Token: 0xAB12<br/>Observe: 4<br/>Payload: "35.2"
    C->>S: ACK

    Note over C,S: NON for routine, CON for critical

1222.4.3 Deregistration

Three ways to stop receiving notifications:

1. Explicit deregistration (GET with Observe: 1):

Client -> Server: GET /temperature
                  Token: 0xAB12
                  Observe: 1  (deregister)

2. RST response to unwanted notification:

Server -> Client: NON 2.05 Content (notification)
Client -> Server: RST (I don't want this anymore)

3. Timeout (Max-Age expiration):

If client doesn't refresh observation within Max-Age,
server removes observer from list.

1222.5 Observer Management Implementation

1222.5.1 Server-Side Observer Registry

from collections import defaultdict
import time

class ObserverRegistry:
    def __init__(self):
        # Map: resource_uri -> list of Observer objects
        self.observers = defaultdict(list)
        self.observer_timeout = 86400  # 24 hours default

    def register_observer(self, resource_uri, client_addr, token, max_age=None):
        observer = Observer(
            client_addr=client_addr,
            token=token,
            registered_at=time.time(),
            last_notification=time.time(),
            timeout=max_age or self.observer_timeout
        )
        self.observers[resource_uri].append(observer)
        return observer

    def notify_all(self, resource_uri, value, content_format):
        """Send notification to all observers of a resource"""
        expired = []

        for observer in self.observers[resource_uri]:
            # Check if observation expired
            if time.time() - observer.registered_at > observer.timeout:
                expired.append(observer)
                continue

            # Send notification
            self.send_notification(observer, value, content_format)

        # Clean up expired observers
        for observer in expired:
            self.observers[resource_uri].remove(observer)

    def remove_observer(self, resource_uri, client_addr, token):
        """Remove observer on explicit deregistration or RST received"""
        self.observers[resource_uri] = [
            o for o in self.observers[resource_uri]
            if not (o.client_addr == client_addr and o.token == token)
        ]

1222.5.2 Notification Rate Limiting

Prevent notification floods on rapidly-changing resources:

class RateLimitedResource:
    def __init__(self, min_interval=1.0, change_threshold=0.5):
        self.min_interval = min_interval    # Minimum seconds between notifications
        self.change_threshold = change_threshold  # Minimum change to trigger notification
        self.last_notify_time = {}          # Per-observer last notification time
        self.last_notified_value = {}       # Per-observer last sent value

    def on_value_change(self, new_value):
        now = time.time()

        for observer in self.observers:
            last_time = self.last_notify_time.get(observer.token, 0)
            last_value = self.last_notified_value.get(observer.token, None)

            # Check if we should notify
            should_notify = (
                last_value is None or
                abs(new_value - last_value) >= self.change_threshold or
                (now - last_time) >= self.min_interval
            )

            if should_notify:
                self.send_notification(observer, new_value)
                self.last_notify_time[observer.token] = now
                self.last_notified_value[observer.token] = new_value

{
  const container = document.getElementById('kc-coap-observe-1');
  if (container && typeof InlineKnowledgeCheck !== 'undefined') {
    container.innerHTML = '';
    container.appendChild(InlineKnowledgeCheck.create({
      question: "A CoAP server has 100 clients observing a temperature resource. The resource changes rapidly (10 times/second). What problem will occur and how should you fix it?",
      options: [
        {text: "No problem - servers can handle unlimited notifications", correct: false, feedback: "Servers have limited bandwidth and memory. 100 clients x 10 changes/second = 1,000 notifications/second, likely overwhelming constrained devices."},
        {text: "Clients will miss notifications - use CON instead of NON", correct: false, feedback: "Using CON would make it worse - now you're waiting for 100 ACKs per change. The problem is notification rate, not reliability."},
        {text: "Notification flood - implement server-side rate limiting with change threshold", correct: true, feedback: "Correct! Implement MIN_NOTIFY_INTERVAL (e.g., 5 seconds) and CHANGE_THRESHOLD (e.g., 0.5 degrees). Only notify when significant change occurs OR minimum interval passes."},
        {text: "Memory exhaustion - limit number of observers", correct: false, feedback: "While observer limits help, the core issue is notification frequency. Rate limiting addresses the root cause."}
      ],
      difficulty: "hard",
      topic: "coap-observe-advanced"
    }));
  }
}

1222.6 Deep Dive: Observe Internals

NoteDeep Dive: Observe Sequence Numbers and Ordering

The Observe option value is a sequence number that helps clients detect: 1. Out-of-order notifications (UDP doesn’t guarantee ordering) 2. Notification freshness (which update is newer)

Sequence Number Rules (RFC 7641 Section 4.4):

def is_notification_fresh(current_seq, new_seq):
    """
    Determine if new notification is fresher than current.
    Handles 24-bit wraparound.
    """
    # Sequence numbers are 24-bit (0 to 16,777,215)
    MAX_SEQ = (1 << 24) - 1

    # Calculate difference handling wraparound
    diff = (new_seq - current_seq) % (MAX_SEQ + 1)

    # If diff < 2^23, new is fresher (forward direction)
    # If diff >= 2^23, new is older (backward direction - out of order)
    return diff < (1 << 23)

Example scenario:

Notification 1: Observe=100, temp=22.5
Notification 2: Observe=102, temp=23.0  (arrived out of order)
Notification 3: Observe=101, temp=22.8

Client receives: 100 -> 102 -> 101
Client should display: 22.5 -> 23.0 (ignore 101, it's older than 102)

NoteDeep Dive: Observer Removal Conditions

Automatic observer removal triggers:

1. RST received: Client sends RST in response to notification
2. Timeout: No activity within observation lifetime
3. CON notification fails: After 4 retransmissions without ACK
4. Resource deleted: Server removes all observers when resource gone

Retransmission behavior for CON notifications:

Server sends CON notification
Wait 2 seconds for ACK
Retransmit with same Message ID
Wait 4 seconds (exponential backoff)
Retransmit
Wait 8 seconds
Retransmit
Wait 16 seconds
Final attempt
After 4 failures -> Remove observer

1222.7 Edge Cases and Gotchas

1222.7.1 Token Reuse After Client Restart

Problem:

- Client registers observation with Token=0x42
- Client crashes and restarts
- Server sends notification with Token=0x42
- Client doesn't recognize token (state lost) -> sends RST
- Server removes observer

Solutions: 1. Server MUST remove observer when RST received 2. Client should re-register observations after restart 3. Consider persisting observation state to flash

1222.7.2 NAT Timeout Issue

Problem:

UDP NAT mappings expire (typically 30-60 seconds)
- Client behind NAT registers observation
- Server tries to push notification 5 minutes later
- NAT mapping expired -> notification never reaches client

Solutions: 1. Server sends periodic keep-alive NON notifications (every 30 sec) 2. Client sends periodic re-registration (GET with Observe=0) 3. Use Max-Age option to set notification frequency 4. Consider CoAP over TCP for NAT-hostile networks

CautionPitfall: Mismanaging Observe Tokens Across Client Restarts

The Mistake: Clients generate new random tokens after reboot without deregistering previous observations, causing “ghost subscriptions” where the server continues sending notifications to tokens the client no longer recognizes.

Why It Happens: The Observe pattern uses tokens to match notifications to subscriptions. When a client reboots, it loses its token-to-subscription mapping but the server still has the observer registered.

The Fix: Implement proper token lifecycle management:

1. Persist tokens across reboots: Store active observation tokens in EEPROM/Flash
2. Use deterministic token generation: Generate from device ID + resource URI hash
3. Handle orphaned notifications gracefully: When receiving notification with unknown token, send RST
4. Server-side timeout: Configure observer timeout (Max-Age option)

1222.8 Bandwidth Savings Calculation

Example: Temperature sensor with 100 observers

Polling approach (GET every 10 seconds):

Per request:
- Request: 14 bytes (header + token + Uri-Path)
- Response: 16 bytes (header + token + payload)
TOTAL: 30 bytes x 100 clients x 6/min x 60 min = 1.08 MB/hour

Observe approach (notify on change, avg 6 changes/hour):

Per notification:
- CoAP header: 4 bytes
- Token: 2 bytes
- Observe option: 3 bytes
- Content-Format: 2 bytes
- Payload marker: 1 byte
- Payload: 6 bytes ("22.5")
TOTAL: 18 bytes x 100 clients x 6 changes = 10.8 KB/hour

Savings: 99% bandwidth reduction (1.08 MB vs 10.8 KB)

{
  const container = document.getElementById('kc-coap-observe-2');
  if (container && typeof InlineKnowledgeCheck !== 'undefined') {
    container.innerHTML = '';
    container.appendChild(InlineKnowledgeCheck.create({
      question: "A smart thermostat app wants to receive temperature updates whenever the value changes, without polling. Which CoAP feature enables this?",
      options: [
        {text: "Blockwise transfer for streaming large payloads", correct: false, feedback: "Blockwise transfer is for large payloads that exceed MTU, not for push notifications."},
        {text: "Multicast messaging to all subscribers", correct: false, feedback: "Multicast sends one message to many receivers, but doesn't maintain subscription state or push on change."},
        {text: "Observe extension allowing server-push notifications", correct: true, feedback: "Correct! The Observe extension (RFC 7641) enables pub/sub-like behavior: client registers once, server pushes changes automatically."},
        {text: "Separate response mode for async replies", correct: false, feedback: "Separate response handles slow processing, not continuous updates."}
      ],
      difficulty: "easy",
      topic: "coap-observe-basic"
    }));
  }
}

1222.9 Summary

CoAP Observe transforms request-response into publish-subscribe:

• Registration: Client sends GET with Observe: 0
• Notifications: Server pushes updates with incrementing sequence numbers
• Deregistration: GET with Observe: 1, RST, or timeout

Key implementation considerations:

• Rate-limit notifications on rapidly-changing resources
• Use NON for routine updates, CON for critical alerts
• Handle NAT timeout with keep-alive messages
• Track sequence numbers to detect out-of-order delivery

Benefits over polling: 90-99% bandwidth reduction for slowly-changing resources.

1222.10 What’s Next

Now that you understand server push:

• Large Payloads: CoAP Advanced Features - Block-wise transfer for firmware updates
• API Design: CoAP API Design - RESTful patterns and security best practices
• See All CoAP Topics: CoAP Fundamentals and Architecture - Complete chapter index