HTTP Recovery Experiments

HTTP Recovery Experiments

animation
http
protocols

Try It Yourself

Experiment 1: Measure Polling Energy Cost

Calculate battery drain from HTTP polling:

import time
import requests

# Simulate 100 polls
start = time.time()
session = requests.Session()  # Uses keep-alive
for _ in range(100):
    response = session.get("https://httpbin.org/get", timeout=5)
    response.raise_for_status()
    time.sleep(1)  # 1 second interval
elapsed = time.time() - start

# With keep-alive: ~100 seconds (connections reused)
# Without keep-alive: ~145 seconds (connection overhead)

print(f"Time for 100 polls: {elapsed:1f}s")
print(f"Overhead per poll: {(elapsed - 100) / 100 * 1000:0f}ms")

What to Observe:

Connection reuse avoids repeated TCP/TLS setup; compare connection time and total device energy for the chosen cellular network and request interval.

  • With keep-alive: minimal overhead (~5 ms per request)
  • Without keep-alive: ~450 ms overhead per request on cellular
  • Battery impact: repeated connection setup can increase radio-on time; measure the device’s actual energy use.

Experiment 2: WebSocket Reconnection Storm

Simulate thundering herd:

// Run in browser console on 10 tabs simultaneously
const ws = new WebSocket('wss://echo.websocket.org/');

// BAD: Fixed retry (all clients reconnect at once)
ws.onclose = () => setTimeout(() => new WebSocket('wss://echo.websocket.org/'), 5000);

// GOOD: Exponential backoff with jitter
ws.onclose = () => {
    const baseDelay = 1000;
    const maxDelay = 60000;
    const jitter = Math.random() * 1000;
    const delay = Math.min(baseDelay * Math.pow(2, attempt), maxDelay) + jitter;
    setTimeout(() => new WebSocket('wss://echo.websocket.org/'), delay);
};

What to Observe:

  • Without jitter: all 10 clients reconnect simultaneously
  • With jitter: reconnections spread over 0-1 second window
  • Server load: a synchronized 10-client reconnect burst without jitter versus requests spread over the chosen retry window with jitter.

Experiment 3: Chunked Transfer Memory Exhaustion

Test bounded chunk streaming:

import requests

# DANGEROUS: Unbounded chunked response can exhaust memory
def stream_unbounded():
    r = requests.get('https://httpbin.org/stream/10000', stream=True)
    data = r.raw.read()  # Buffers entire 10,000-line response!

# SAFE: Process chunks incrementally
def stream_bounded():
    r = requests.get('https://httpbin.org/stream/10000', stream=True)
    count = 0
    for line in r.iter_lines(chunk_size=1024):
        count += 1
        if count % 1000 == 0:
            print(f"Processed {count} lines")
    return count

What to Observe:

Unbounded reads can fill memory. Read fixed-size chunks, then check how many buffers fit.

  • Unbounded: memory usage grows to ~5MB
  • Bounded: constant ~1KB memory usage
  • 256 MB / 1 kB = 256,000 buffers as a memory-only upper bound; actual concurrent streams are far fewer after network and runtime costs

Challenge: Cost-Benefit Analysis

Calculate annual cellular cost for a fleet management system:

Begin with the tracker count and request frequency, then add per-request overhead to calculate fleet traffic and cost.

  • Fleet size: 5,000 GPS trackers.
  • Current design: a fresh HTTPS POST every 30 seconds.
  • Alternative design: MQTT persistent connection.

HTTP polling inputs

Scale request overhead from one update to a day and month before applying the stated price assumption.

  • Overhead per request: 6.8 kB for TCP, TLS, and HTTP headers.
  • Payload: 0.085 kB of GPS coordinates.
  • Total per request: 6.885 kB.
  • Daily per device: 2,880 requests × 6.885 kB = 19.8288 MB.
  • Fleet daily: 5,000 × 19.8288 MB = 99.144 GB.
  • Monthly traffic: 99.144 GB × 30 = 2,974.32 GB.
  • Annual cost formula: 2,974.32 GB/month × 1,000 MB/GB × EUR 0.01/MB × 12 = EUR 356,918.40.

MQTT persistent inputs

Add MQTT payload and keepalive bytes first. Then scale that sum to fleet cost.

  • Illustrative keepalive traffic allowance: 0.12 kB per minute, including assumed lower-layer and response bytes.
  • Payload: 0.009 kB with binary GPS encoding.
  • Daily per device: (2,880 × 0.009) + (1,440 × 0.12) = 198.72 kB = 0.19872 MB.
  • Fleet daily: 5,000 × 0.19872 MB = 993.6 MB.
  • Monthly traffic: 993.6 MB × 30 = 29.808 GB.
  • Annual cost formula: 29.808 GB/month × 1,000 MB/GB × EUR 0.01/MB × 12 = EUR 3,576.96.

Calculate the savings!

Back to the chapter