7 Protocol Pioneers: Ethernet and MQTT

7.8.1 The Problem He Solved

In the early 1970s, Xerox PARC (Palo Alto Research Center) had dozens of computers and the world’s first laser printer. Researchers wanted to share the printer, but there was no good way to network the machines. Point-to-point cables didn’t scale — connecting $N$ devices requires $\frac{N(N-1)}{2}$ cables. For 20 devices, that’s 190 cables!

7.8.2 His Breakthrough: Ethernet

In 1973, Bob Metcalfe invented Ethernet — a way for multiple computers to share a single coaxial cable using a cleverly designed media access control protocol:

Key Innovations:

1. Shared Medium All devices connect to the same cable (like houses on a street sharing a postal system). This requires only $N$ connections instead of $\frac{N(N-1)}{2}$.

2. CSMA/CD (Carrier Sense Multiple Access with Collision Detection)

• Carrier Sense: Before transmitting, listen to the cable. If someone else is talking, wait.
• Collision Detection: If two devices start transmitting simultaneously (collision), both detect it and stop.
• Exponential Backoff: After a collision, wait a random time before retrying. Each collision doubles the wait range.

3. 48-Bit MAC Addresses Ethernet interfaces commonly use 48-bit MAC addresses (like 00:1A:2B:3C:4D:5E). Universally administered addresses are intended to be unique, but locally administered addresses can be assigned or randomized.

7.8.3 Metcalfe’s Law: The Network Effect

Perlman made redundant Ethernet survivable by blocking loops. Metcalfe’s story explains why Ethernet then became valuable at scale: each compatible node made the shared ecosystem more useful.

Metcalfe formulated a key principle of network value:

Translation: In Metcalfe’s simple model, potential pairwise connections grow roughly with the square of the number of users. Actual value depends on whether those connections are useful.

Example: A phone network with 10 people supports 45 possible calls ($\frac{10 \times 9}{2}$). Add one person (now 11), and you support 55 calls — a 22% increase in value from adding just one user (10% increase in users).

This explains why social networks grow explosively once they reach critical mass. It also explains why IoT platforms that achieve interoperability (allowing devices from many vendors to connect) are more valuable than proprietary ecosystems.

7.8.4 How to Use: The Impact

Metcalfe founded 3Com Corporation in 1979 to commercialize Ethernet. By the 1990s, Ethernet had become the dominant LAN technology, beating out Token Ring, ARCNET, and others. Today, Ethernet variants include:

• 10BASE-T (10 Mbps over twisted pair)
• 100BASE-TX (Fast Ethernet, 100 Mbps)
• 1000BASE-T (Gigabit Ethernet, 1 Gbps)
• 10GBASE-T (10 Gigabit Ethernet, 10 Gbps)

Metcalfe won the Turing Award (computing’s Nobel Prize) in 2022 for inventing Ethernet.

+Checkpoint: Local Networks and Scale

You now know:

• Perlman’s Spanning Tree Protocol lets Ethernet keep redundancy without broadcast loops.
• Metcalfe’s Ethernet made local sharing practical, then Metcalfe’s Law explained why larger interoperable networks become more valuable.
• This is the same economic reason Matter and other open IoT ecosystems can outgrow isolated proprietary islands.

7.9.1 The Problem He Solved

The Internet needed someone to:

• Document standards — write down how protocols work so everyone implements them the same way
• Manage addresses — assign IP addresses and domain names so no two organizations use the same numbers
• Resolve disputes — mediate when engineers disagreed on protocol design

No company or government controlled the Internet in the 1970s-80s. It was a research project funded by ARPA, built by academics and engineers who volunteered their time. Someone had to coordinate all this chaos.

7.9.2 His Role: Editor of RFCs and IANA Manager

RFC Editor (1969-1998) Jon Postel edited most of the early RFCs (Requests for Comments) — the documents that define Internet standards. He decided which proposals became official specifications and which were rejected. Over 30 years, he edited or authored hundreds of RFCs, including foundational specs like:

• RFC 791: Internet Protocol (IP)
• RFC 793: Transmission Control Protocol (TCP)
• RFC 821: Simple Mail Transfer Protocol (SMTP)
• RFC 959: File Transfer Protocol (FTP)

IANA Manager (1972-1998) Postel managed the Internet Assigned Numbers Authority (IANA) — the organization that assigns:

• IP address blocks to ISPs and organizations
• Protocol numbers (TCP = 6, UDP = 17, ICMP = 1)
• Port numbers (HTTP = 80, HTTPS = 443, MQTT = 1883)

He did this from his desk at USC (University of Southern California) with minimal staff. The Internet community trusted him to make fair decisions, and he did.

7.9.3 Postel’s Law: The Robustness Principle

Postel formulated the most important principle in protocol design:

“Be conservative in what you send, liberal in what you accept.”

Translation:

• Conservative sending: When you implement a protocol, strictly follow the spec. Don’t send malformed packets or ambiguous messages.
• Liberal receiving: When you receive data, tolerate small deviations from the spec. If the message is understandable despite minor errors, process it anyway.

Why this matters: Networks are messy. Different vendors implement protocols slightly differently. If every device rejected messages with minor formatting errors, the Internet would fragment into incompatible islands. Postel’s Law keeps the Internet unified despite vendor diversity.

7.9.4 The 1998 Incident: Postel’s Authority

In 1998, Postel conducted a controversial test. He asked 8 of the 13 root DNS servers to temporarily redirect to a server he controlled — without asking the U.S. government for permission.

The test worked. For a few hours, Postel controlled the majority of the Internet’s root DNS infrastructure. He proved that IANA’s authority was personal — based on trust and technical competence, not legal ownership.

The U.S. government was not amused. Within months, oversight of DNS was transferred to ICANN (Internet Corporation for Assigned Names and Numbers), a new nonprofit. Postel’s era of informal, trust-based governance ended.

7.9.5 How to Use: The Impact

Postel died suddenly in 1998 at age 55. The Internet Engineering Task Force (IETF) published RFC 2468 in his honor, titled “I Remember IANA”:

“Jon Postel was the Internet.”

Every protocol you use — IP, TCP, UDP, ICMP, DNS — was documented in RFCs that Postel edited. Every port number (:443, :1883, :8883) was assigned by Postel’s IANA. His influence is invisible but everywhere.

7.10.1 The Problem He Solved

In 1999, Andy Stanford-Clark (IBM) and Arlen Nipper (Arcom) were building a system to monitor oil pipelines across remote deserts. They needed to transmit sensor data via expensive satellite links with:

• Very low bandwidth (9600 bps or less)
• High latency (500 ms to 2 seconds)
• Unreliable connections (frequent disconnections)

HTTP was too heavy — each HTTP request had 200+ bytes of headers. On a 9600 bps link, just the headers consumed about 167 ms. For thousands of sensors sending frequent updates, HTTP was unusable.

7.10.2 His Breakthrough: MQTT (Message Queuing Telemetry Transport)

Stanford-Clark and Nipper designed MQTT specifically for constrained networks:

Key Design Choices:

1. Minimal Protocol Overhead MQTT headers are just 2 bytes (vs. 200+ for HTTP). The fixed header contains:

• Message type (4 bits)
• QoS level (2 bits)
• Remaining length (variable-length encoding)

2. Publish/Subscribe Pattern Instead of request/response (like HTTP), MQTT uses pub/sub:

• Sensors publish to topics: sensors/building-a/temperature
• Applications subscribe to topics they care about
• A broker routes messages from publishers to subscribers

This decouples senders from receivers — sensors don’t need to know who’s listening.

3. Quality of Service (QoS) Levels MQTT offers three QoS levels:

• QoS 0 (At most once): Fire and forget, no acknowledgment
• QoS 1 (At least once): Acknowledged, may duplicate
• QoS 2 (Exactly once): The MQTT delivery protocol avoids loss and duplication on a negotiated QoS 2 hop, with higher overhead; broker subscription overlap and application retries still need separate duplicate handling. MQTT 5.0

Sensors can choose the right tradeoff between reliability and overhead.

4. Last Will and Testament If a client disconnects unexpectedly (e.g., network failure), the broker automatically publishes a “last will” message the client specified during connection. This enables other devices to detect failures.

7.10.3 The Tweeting House

Stanford-Clark famously connected his home to the Internet in the early 2000s (before “smart home” was a thing):

• His house tweeted when the washing machine finished
• His doorbell sent him SMS messages
• Temperature sensors posted to a web dashboard

This DIY smart home demonstrated MQTT’s versatility — it wasn’t just for industrial oil pipelines. Any system with sensors and actuators could use MQTT.

7.10.4 How to Use: The Impact

MQTT became the de facto standard for IoT messaging:

• AWS IoT Core: Uses MQTT as the primary protocol
• Azure IoT Hub: Supports MQTT alongside AMQP
• Google Cloud IoT: Supported MQTT (service deprecated 2023)
• Home Assistant: MQTT is the standard integration protocol
• Industrial IoT: MQTT is used in manufacturing, automotive, and logistics

In 2013, MQTT became an OASIS standard (not just an IBM project). In 2019, it became an ISO standard (ISO/IEC 20922).

+Checkpoint: IoT Constraints

You now know:

• MQTT came from a 1999 oil-pipeline problem with about 9600 bps bandwidth and high latency.
• Its 2-byte header directly answered HTTP’s 200+ byte overhead on constrained links.
• 6LoWPAN later solved a similar fit problem by compressing IPv6’s 40-byte header for 127-byte 802.15.4 frames.