HTTP Recovery Experiments
HTTP Recovery Experiments
Try It Yourself
Experiment 1: Measure Polling Energy Cost
Calculate battery drain from HTTP polling:
import time
import requests
# Simulate 100 polls
start = time.time()
session = requests.Session() # Uses keep-alive
for _ in range(100):
response = session.get("https://httpbin.org/get", timeout=5)
response.raise_for_status()
time.sleep(1) # 1 second interval
elapsed = time.time() - start
# With keep-alive: ~100 seconds (connections reused)
# Without keep-alive: ~145 seconds (connection overhead)
print(f"Time for 100 polls: {elapsed:1f}s")
print(f"Overhead per poll: {(elapsed - 100) / 100 * 1000:0f}ms")What to Observe:
Connection reuse avoids repeated TCP/TLS setup; compare connection time and total device energy for the chosen cellular network and request interval.
- With keep-alive: minimal overhead (~5 ms per request)
- Without keep-alive: ~450 ms overhead per request on cellular
- Battery impact: repeated connection setup can increase radio-on time; measure the device’s actual energy use.
Experiment 2: WebSocket Reconnection Storm
Simulate thundering herd:
// Run in browser console on 10 tabs simultaneously
const ws = new WebSocket('wss://echo.websocket.org/');
// BAD: Fixed retry (all clients reconnect at once)
ws.onclose = () => setTimeout(() => new WebSocket('wss://echo.websocket.org/'), 5000);
// GOOD: Exponential backoff with jitter
ws.onclose = () => {
const baseDelay = 1000;
const maxDelay = 60000;
const jitter = Math.random() * 1000;
const delay = Math.min(baseDelay * Math.pow(2, attempt), maxDelay) + jitter;
setTimeout(() => new WebSocket('wss://echo.websocket.org/'), delay);
};What to Observe:
- Without jitter: all 10 clients reconnect simultaneously
- With jitter: reconnections spread over 0-1 second window
- Server load: a synchronized 10-client reconnect burst without jitter versus requests spread over the chosen retry window with jitter.
Experiment 3: Chunked Transfer Memory Exhaustion
Test bounded chunk streaming:
import requests
# DANGEROUS: Unbounded chunked response can exhaust memory
def stream_unbounded():
r = requests.get('https://httpbin.org/stream/10000', stream=True)
data = r.raw.read() # Buffers entire 10,000-line response!
# SAFE: Process chunks incrementally
def stream_bounded():
r = requests.get('https://httpbin.org/stream/10000', stream=True)
count = 0
for line in r.iter_lines(chunk_size=1024):
count += 1
if count % 1000 == 0:
print(f"Processed {count} lines")
return countWhat to Observe:
Unbounded reads can fill memory. Read fixed-size chunks, then check how many buffers fit.
- Unbounded: memory usage grows to ~5MB
- Bounded: constant ~1KB memory usage
- 256 MB / 1 kB = 256,000 buffers as a memory-only upper bound; actual concurrent streams are far fewer after network and runtime costs
Challenge: Cost-Benefit Analysis
Calculate annual cellular cost for a fleet management system:
Begin with the tracker count and request frequency, then add per-request overhead to calculate fleet traffic and cost.
- Fleet size: 5,000 GPS trackers.
- Current design: a fresh HTTPS POST every 30 seconds.
- Alternative design: MQTT persistent connection.
HTTP polling inputs
Scale request overhead from one update to a day and month before applying the stated price assumption.
- Overhead per request: 6.8 kB for TCP, TLS, and HTTP headers.
- Payload: 0.085 kB of GPS coordinates.
- Total per request: 6.885 kB.
- Daily per device: 2,880 requests × 6.885 kB = 19.8288 MB.
- Fleet daily: 5,000 × 19.8288 MB = 99.144 GB.
- Monthly traffic: 99.144 GB × 30 = 2,974.32 GB.
- Annual cost formula: 2,974.32 GB/month × 1,000 MB/GB × EUR 0.01/MB × 12 = EUR 356,918.40.
MQTT persistent inputs
Add MQTT payload and keepalive bytes first. Then scale that sum to fleet cost.
- Illustrative keepalive traffic allowance: 0.12 kB per minute, including assumed lower-layer and response bytes.
- Payload: 0.009 kB with binary GPS encoding.
- Daily per device:
(2,880 × 0.009) + (1,440 × 0.12) = 198.72 kB = 0.19872 MB. - Fleet daily: 5,000 × 0.19872 MB = 993.6 MB.
- Monthly traffic: 993.6 MB × 30 = 29.808 GB.
- Annual cost formula: 29.808 GB/month × 1,000 MB/GB × EUR 0.01/MB × 12 = EUR 3,576.96.
Calculate the savings!