← Back to the chapter · Previous detail · Next detail →

Technical detail · MQTT Python: Reliable Client Patterns

Measuring TLS setup time on ESP32 and Python clients

TLS (Transport Layer Security) protects an MQTT broker connection through a handshake that negotiates protection and validates certificates.

The examples use ESP32 hardware and the Paho-mqtt Python library. TCP (Transmission Control Protocol) supplies connections; CA means certificate authority. MQTT keepalive sets an idle-check interval; exponential backoff increases retry waits. Supply deployment-specific addresses, credentials and certificates.

The Mistake: Developers enable TLS (port 8883) for production security, test on a fast development machine, and then reuse the same timeout settings on constrained Wi-Fi devices. In the field, slower crypto, Wi-Fi reconnects, network latency, and broker load can push the handshake beyond a default timeout and cause intermittent connection failures.

Why It Happens: TLS handshakes perform key exchange and certificate validation using algorithms chosen by the endpoints. Their CPU and elapsed-time cost varies with hardware, network path and broker load. A timeout copied from a laptop test can be too short after Wi-Fi reconnect and TLS negotiation on the target device.

The Fix: Measure connection and TLS handshake times on the device and network, then configure supported timeout and retry settings with margin. TLS session resumption can reduce a repeated handshake when both endpoints support and negotiate it. Provision device certificates before deployment when that is part of the authentication design.

// ESP32: Configure generous TLS timeouts
#include <WiFiClientSecure.h>
#include <PubSubClient.h>

WiFiClientSecure espClient;
PubSubClient mqttClient(espClient);

void setup() {
    // Load CA certificate (or use setInsecure() for testing only)
    espClient.setCACert(root_ca);

    // Increase TCP timeout for TLS handshake (default is often 10s)
    espClient.setTimeout(60000);  // 60 seconds for slow TLS

    // Configure MQTT client with longer keepalive
    mqttClient.setServer(mqtt_server, 8883);
    mqttClient.setKeepAlive(120);  // 2 minutes (allows for slow reconnects)
    mqttClient.setSocketTimeout(60);  // 60 second socket timeout
}

bool connectWithRetry() {
    int attempts = 0;
    while (!mqttClient.connected() && attempts < 5) {
        Serial.printf("TLS connect attempt %d...\n", attempts + 1);
        unsigned long start = millis();

        if (mqttClient.connect(client_id, mqtt_user, mqtt_pass)) {
            unsigned long elapsed = millis() - start;
            Serial.printf("Connected in %lu ms\n", elapsed);
            return true;
        }

        attempts++;
        // Exponential backoff: 10s, 20s, 40s, 80s, 160s
        int delay_ms = 5000 * (1 << attempts);
        Serial.printf("Failed, retrying in %d ms\n", delay_ms);
        delay(delay_ms);
    }
    return false;
}
# Python paho-mqtt: TLS with extended timeout
import ssl
import paho.mqtt.client as mqtt

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)

# Configure certificate validation for TLS
context = ssl.create_default_context()
context.check_hostname = True
context.verify_mode = ssl.CERT_REQUIRED
context.load_verify_locations("/path/to/ca.crt")

# In paho-mqtt 2.1, connect_timeout (default 5 s) bounds the TCP connect.
client.connect_timeout = 5

client.tls_set_context(context)

# paho-mqtt 2.1 sets the socket timeout to keepalive during the TLS handshake.
# Here the TLS handshake can wait up to 120 s; measure and tune both timeouts.
client.connect(broker, 8883, keepalive=120)
client.loop_start()

Timing guidance: Measure handshake time on the real board, network, certificate chain, and broker path. Then configure a timeout with enough margin for Wi-Fi reconnects, broker load, retries, and clock drift instead of copying a laptop default into firmware.

← Back to the chapter · Previous detail · Next detail →