Event-Driven Architecture

Topic Guide

Architecture & Design
Learn about Event-Driven Architecture in IoT systems – how devices react to real-time events using decoupled producers, brokers, and consumers for scalable and resilient IoT deployments.

Event-Driven Architecture

Architecture & Design

Learning Objectives

By the end of this topic guide, you will be able to:

  • Define Event-Driven Architecture (EDA): Explain the core principles of event production, detection, and reaction in IoT systems
  • Identify EDA Components: Distinguish producers, consumers, brokers, and event channels in an IoT deployment
  • Compare Messaging Patterns: Differentiate between point-to-point, publish-subscribe, and event streaming approaches
  • Apply Event Processing: Understand simple event processing, event filtering, and Complex Event Processing (CEP) for IoT analytics
  • Select Appropriate Technologies: Choose between MQTT, Kafka, AMQP, and other event brokers based on IoT requirements
Minimum Viable Understanding

If you take away only three things from this topic:

  1. Events decouple producers from consumers – a temperature sensor publishes a reading without knowing (or caring) which services consume it. This decoupling lets you add new consumers (dashboard, alert service, analytics) without modifying the sensor or existing consumers.
  2. Brokers are the backbone – the event broker (MQTT broker, Kafka cluster, RabbitMQ) handles routing, buffering, and delivery guarantees. Choosing the right broker for your IoT scenario determines latency, throughput, reliability, and cost.
  3. Event-driven is not always the right choice – EDA excels for real-time reactions and fan-out scenarios, but adds complexity (eventual consistency, debugging difficulty, message ordering). Use request-response for simple query/command patterns where you need an immediate, synchronous answer.

Hey Sensor Squad! Today our four friends are building an event-driven alarm system for their school garden.

Temperature Terry is the garden’s temperature watcher. Instead of shouting the temperature every second (“72 degrees! 72 degrees! Still 72 degrees!”), Sammy only speaks up when something CHANGES: “Alert! Temperature just hit 95 degrees – the tomatoes are in danger!”

Lila the Lightbulb is the event broker – the messenger in the middle. When Sammy sends an alert, Lila does NOT just pass it to one person. She broadcasts it to EVERYONE who signed up for garden alerts. Think of her like a school PA system!

the microcontroller subscribed to temperature alerts. When he hears “95 degrees!” from Lila, he automatically turns on the garden sprinklers. He does not care WHERE the alert came from – he just reacts to the event.

the battery also subscribed, but she does something different with the SAME alert. She sends a text message to the teacher: “Garden is too hot! Sprinklers activated.” Same event, different reaction!

Why is this so cool?

  • Sammy does NOT need to know who Max and Bella are (they are “decoupled”)
  • If a new helper joins (say, a fan controller), they just subscribe to Lila – nobody else changes!
  • Sammy saves energy by only talking when something important happens
  • If Lila is busy, she remembers the message and delivers it when ready (no lost events!)

The Event-Driven Rule: Do NOT keep asking “anything happen yet?” – instead, WAIT and REACT only when something actually happens. It saves energy, reduces noise, and scales to hundreds of helpers!

The simple version: Event-Driven Architecture (EDA) is a design approach where software components communicate by producing and reacting to events – things that happen. Instead of one component directly calling another (“Hey, give me the temperature!”), a sensor simply announces “temperature changed!” and any interested service can listen and react independently.

A real-world analogy: Think about a newspaper subscription. The newspaper (producer) publishes articles without knowing exactly who reads them. Subscribers (consumers) receive the paper and each does something different – one clips coupons, another reads sports, a third checks the weather. The delivery service (broker) handles routing. No one needs to know about anyone else.

Three core components:

Component Role IoT Example
Event Producer Creates events when something happens Temperature sensor detects threshold breach
Event Broker Routes events to interested consumers MQTT broker, Apache Kafka, RabbitMQ
Event Consumer Reacts to events it has subscribed to Dashboard updates, alert service triggers SMS

When to use EDA:

  • You need real-time reactions (alerts when temperature exceeds a limit)
  • Multiple consumers need the same data (dashboard + analytics + logging)
  • Systems should be independently deployable and scalable
  • You want to add new features without changing existing code

When NOT to use EDA:

  • Simple request-response queries (“What is the current temperature?”)
  • When you need immediate, synchronous confirmation of an action
  • Small systems with only 2-3 components that rarely change

Overview

Key Concepts: events, reactive, asynchronous, decoupling, publish-subscribe, event sourcing, Complex Event Processing

Event-Driven Architecture (EDA) is a software design paradigm in which the flow of a program is determined by events – significant changes in state such as a sensor reading crossing a threshold, a device going offline, or a user pressing a button. In IoT systems, EDA provides the foundation for building scalable, real-time, and loosely coupled systems that can process millions of sensor events per second.

The fundamental insight of EDA is temporal and spatial decoupling: event producers do not need to know which consumers exist, and consumers do not need to be online at the moment an event is produced. The event broker mediates all communication, enabling independent development, deployment, and scaling of each component.

High-level event-driven architecture diagram showing IoT event producers on the left (temperature sensor, motion detector, smart meter) sending events through a central event broker in teal, which routes events via topic channels to multiple event consumers on the right (dashboard, alert service, analytics engine, automation rules). Arrows illustrate the fan-out pattern where a single event reaches multiple consumers.

Core EDA Patterns in IoT

There are three primary messaging patterns used in event-driven IoT systems, each suited to different requirements:

Pattern Comparison

Pattern Delivery Persistence Use Case
Point-to-Point Exactly one consumer Until consumed Command dispatch, task queues
Publish-Subscribe All subscribers Typically transient Real-time alerts, fan-out
Event Streaming Consumer-controlled Long-term (days/weeks) Analytics, audit trails, replay

Event Processing Levels

IoT systems apply different levels of event processing depending on the complexity of the insights they need to extract:

Three-tier event processing pyramid for IoT. At the base, Simple Event Processing handles individual events like threshold checks. In the middle, Event Stream Processing handles windowed aggregations across continuous streams. At the top, Complex Event Processing detects patterns across multiple event streams over time for sophisticated analytics. Each tier shows example operations and typical latency.

  • Simple Event Processing (SEP): Each event is evaluated independently. Example: if temperature > 40C, trigger an alert. No correlation between events.
  • Event Stream Processing (ESP): Continuous queries over sliding windows of events. Example: compute the average temperature over the last 5 minutes and alert if it exceeds 38C. Uses frameworks like Apache Flink or Kafka Streams.
  • Complex Event Processing (CEP): Detects patterns across multiple event streams over time. Example: if motion is detected AND temperature rises AND the door has not been opened in 30 minutes, infer a potential fire. Uses pattern matching engines like Esper or Apache Siddhi.

IoT Event Broker Selection

Selecting the right event broker is one of the most consequential architectural decisions in an IoT system. The following decision tree helps guide this choice:

Decision tree for selecting an IoT event broker. Starting from the question 'What are your primary requirements?', the tree branches into three paths: constrained devices and low bandwidth leads to MQTT, high throughput and event replay leads to Apache Kafka, and complex routing and delivery guarantees leads to RabbitMQ with AMQP. Each leaf node lists the broker's key strengths for IoT scenarios.

Broker Comparison Table

Feature MQTT Apache Kafka RabbitMQ (AMQP)
Protocol overhead Very low (2-byte header) Moderate Moderate
Message persistence Optional (retained messages) Built-in (log-based) Optional (durable queues)
Throughput ~100K msg/sec ~1M+ msg/sec ~50K msg/sec
Message replay No Yes (offset-based) No (consumed = gone)
IoT device support Excellent (constrained devices) Limited (heavy client) Good (multiple protocols)
QoS levels 0, 1, 2 At-least-once (default) At-most-once to exactly-once
Best for Sensors, edge devices Analytics pipelines Enterprise workflows

Worked Example: Smart Building Event Pipeline

Scenario: A commercial building has 200 temperature sensors, 50 occupancy sensors, and 30 air quality sensors. The building management system must: (a) display live readings on a dashboard, (b) trigger HVAC adjustments in real-time, (c) generate daily energy reports, and (d) detect anomalies indicating equipment failure.

Step 1: Define Event Types

Event Type Source Payload Example Frequency
temperature.reading Temp sensor {"zone": "3F-A", "value": 23.5, "unit": "C"} Every 60s
occupancy.change PIR sensor {"zone": "3F-A", "count": 12, "status": "occupied"} On change
airquality.reading AQ sensor {"zone": "3F-A", "co2_ppm": 850, "pm25": 12} Every 300s
hvac.command Control system {"zone": "3F-A", "action": "cool", "target": 22} On demand

Step 2: Design the Event Flow

Sensors --> MQTT Broker --> Kafka (persistence) --> Consumers
                |                    |
                v                    v
          Live Dashboard      Stream Processing
          (WebSocket)         (Flink/Kafka Streams)
                                     |
                              +------+------+
                              |      |      |
                              v      v      v
                           HVAC   Anomaly  Daily
                           Control Detect   Report

Step 3: Select Brokers

  • MQTT (Mosquitto): Edge-level broker. The 280 sensors publish via MQTT because they are constrained devices on a local network. MQTT’s small overhead (2-byte header) minimizes bandwidth usage.
  • Apache Kafka: Cloud-level broker. MQTT messages are bridged into Kafka topics for persistence, replay, and stream processing. Kafka retains 7 days of events for anomaly detection model retraining.

Step 4: Calculate Event Volume

  • Temperature: 200 sensors x 1 event/min = 200 events/min = 12,000 events/hour
  • Occupancy: 50 sensors x ~10 changes/hour = 500 events/hour
  • Air quality: 30 sensors x 12 events/hour = 360 events/hour
  • Total: ~12,860 events/hour = ~308,640 events/day

At an average payload of 150 bytes, daily data volume = 308,640 x 150 = ~44 MB/day – well within MQTT and Kafka capacity.

Step 5: Implement CEP Rules

Anomaly detection rule (pseudocode):

WHEN temperature.reading.value > zone.setpoint + 5
AND  occupancy.change.count < 3
AND  hvac.command was issued > 30 minutes ago
THEN trigger "possible HVAC failure" alert
WITH severity = "high", affected_zone = event.zone

This CEP rule correlates three event streams (temperature, occupancy, HVAC commands) to detect when a zone is overheating despite low occupancy and a recent HVAC command – indicating potential equipment failure rather than simply a hot day with many people.

Common Pitfalls

  1. Treating every sensor reading as an event: Publishing raw sensor data at high frequency (e.g., accelerometer at 100 Hz) as discrete events overwhelms brokers and consumers. Instead, apply edge-side filtering – only publish when values change meaningfully (dead-band filtering) or summarize windows of readings.

  2. Ignoring message ordering: Events may arrive out of order due to network latency, retransmissions, or parallel processing. A “door closed” event arriving before “door opened” causes incorrect state. Use sequence numbers, timestamps, and idempotent consumers to handle this.

  3. Conflating events with commands: Events describe something that happened (“temperature reached 40C”). Commands describe something that should happen (“turn on cooling”). Mixing these in the same topic/queue leads to confusing ownership – who is responsible for acting? Keep event topics and command topics separate.

  4. Missing dead-letter queues: When a consumer cannot process an event (malformed payload, downstream service down), the event must go somewhere. Without a dead-letter queue (DLQ), failed events are silently dropped. Always configure DLQs and monitor them.

  5. Assuming exactly-once delivery is free: True exactly-once processing across distributed systems requires idempotent consumers, deduplication, and transactional producers – all of which add complexity and latency. Evaluate whether at-least-once with idempotent handlers is sufficient for your IoT use case (it usually is).

Knowledge Check

Test your understanding of Event-Driven Architecture concepts.

Learning Resources

Key chapters that expand on the concepts in this guide:

Pub/Sub and Topic Routing

Core chapter covering broker-mediated publish-subscribe delivery and topic-based routing that underpins EDA.

Communication Models for IoT

Foundational chapter comparing request-response, publish-subscribe, and push/pull patterns that underpin EDA.

SOA and Microservices for IoT

How event-driven communication connects microservices in IoT platforms, including choreography vs. orchestration.

MQTT Publish-Subscribe Basics

Deep dive into the most widely used IoT messaging protocol, covering QoS levels, topic design, and retained messages.

Summary

Event-Driven Architecture is a foundational pattern for building scalable, real-time IoT systems. The key takeaways from this topic guide are:

  • Decoupling is the core benefit: Producers and consumers operate independently. Adding a new analytics service or dashboard requires zero changes to sensors or existing consumers.
  • Three messaging patterns serve different needs: point-to-point for commands, publish-subscribe for real-time fan-out, and event streaming for persistent, replayable event logs.
  • Event processing ranges from simple to complex: Threshold checks (SEP), windowed aggregations (ESP), and multi-stream pattern detection (CEP) form a processing hierarchy.
  • Broker selection depends on your constraints: MQTT for constrained IoT devices, Kafka for high-throughput analytics pipelines, RabbitMQ for enterprise-grade routing and delivery guarantees.
  • Hybrid architectures are common: Most production IoT systems combine MQTT at the edge with Kafka or similar in the cloud, bridging between them for the best of both worlds.
  • Pitfalls are predictable and avoidable: Separate events from commands, implement dead-letter queues, handle out-of-order delivery, and do not flood brokers with raw high-frequency data.

What’s Next

After understanding Event-Driven Architecture, consider exploring:

Back to top