1573  Hardware-in-the-Loop (HIL) Testing for IoT

1573.1 Learning Objectives

By the end of this chapter, you will be able to:

• Understand HIL Architecture: Design HIL test setups for IoT devices
• Build Sensor Simulators: Create DAC-based sensor simulation for automated testing
• Implement Failure Injection: Test device behavior under fault conditions
• Integrate HIL with CI/CD: Automate HIL tests in continuous integration pipelines

1573.2 Prerequisites

Before diving into this chapter, you should be familiar with:

• Integration Testing: Understanding integration test concepts
• Testing Fundamentals: The IoT testing pyramid

NoteKey Takeaway

In one sentence: HIL testing runs real firmware against simulated sensors, enabling automated validation of hardware-software interactions.

Remember this rule: If you can’t test it in CI, you can’t prevent regressions. HIL makes hardware testing as automated as software testing.

---

1573.3 What is Hardware-in-the-Loop Testing?

HIL testing involves running your actual device (the Device Under Test, or DUT) while simulating the external world through controlled inputs and monitored outputs.

HIL system architecture

Figure 1573.1: HIL test architecture: Test PC controls interface board, which simulates sensors and monitors device outputs

1573.3.1 Why HIL Testing Matters

Testing Approach	Coverage	Speed	Cost	Repeatability
Unit Tests (host)	Logic only	Fast	Free	Perfect
Manual Testing	Full system	Slow	High	Poor
Field Testing	Real-world	Very slow	Very high	Impossible
HIL Testing	Firmware + HW	Medium	Medium	Excellent

HIL fills the gap: You can test firmware behavior under conditions that are dangerous, expensive, or impossible to create manually (e.g., sensor failure, extreme temperatures, network outages).

---

1573.4 HIL Hardware Setup

1573.4.1 Minimum HIL Components

Component	Purpose	Example Products	Cost
Test Host	Run test scripts	Any PC/laptop	Existing
Interface Board	Generate/read signals	Arduino Mega, Raspberry Pi Pico	$25-50
DAC Module	Simulate analog sensors	MCP4725 (12-bit)	$5-10
ADC Module	Monitor DUT outputs	ADS1115 (16-bit)	$10-15
Relay Module	Control power cycling	4-channel relay	$10
Logic Analyzer	Debug protocol issues	Saleae Logic 8 (optional)	$500

1573.4.2 Wiring Architecture

Test Host (USB) ────► Interface Board (Arduino Mega)
                              │
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
        ▼                     ▼                     ▼
     DAC (I2C)           GPIO Pins            Relay Module
        │                     │                     │
        ▼                     ▼                     ▼
  DUT Analog In        DUT Digital I/O       DUT Power Supply

1573.4.3 Example: Simulating a Temperature Sensor (NTC Thermistor)

Real NTC thermistors output analog voltage based on temperature. Your HIL can simulate this:

# On Interface Board (Arduino/Python)
# MCP4725 DAC connected via I2C

import board
import busio
import adafruit_mcp4725
import math

i2c = busio.I2C(board.SCL, board.SDA)
dac = adafruit_mcp4725.MCP4725(i2c)

def simulate_temperature(celsius):
    """
    Simulate NTC thermistor voltage for given temperature.
    Assumes 10k NTC with 10k pull-up, 3.3V reference.
    """
    # Steinhart-Hart approximation
    T = celsius + 273.15
    B = 3950  # B-constant for typical NTC
    R25 = 10000  # Resistance at 25C
    R_pullup = 10000

    R = R25 * math.exp(B * (1/T - 1/298.15))
    voltage = 3.3 * R / (R + R_pullup)

    # Convert to DAC value (12-bit, 0-4095)
    dac_value = int((voltage / 3.3) * 4095)
    dac.raw_value = dac_value
    return voltage

---

1573.5 HIL Test Framework

1573.5.1 Python Test Framework Structure

hil_tests/
├── conftest.py           # pytest fixtures
├── fixtures/
│   ├── interface.py      # Arduino communication
│   ├── sensors.py        # Sensor simulators
│   └── network.py        # Network emulator
├── tests/
│   ├── test_temperature.py
│   ├── test_wifi_recovery.py
│   └── test_power_cycle.py
└── requirements.txt

1573.5.2 conftest.py - Shared Fixtures

import pytest
import serial
import time

class HILInterface:
    def __init__(self, port='/dev/ttyUSB0', baud=115200):
        self.serial = serial.Serial(port, baud, timeout=1)
        time.sleep(2)  # Wait for Arduino reset

    def send_command(self, cmd):
        self.serial.write(f"{cmd}\n".encode())
        return self.serial.readline().decode().strip()

    def set_temperature(self, celsius):
        return self.send_command(f"TEMP:{celsius:.1f}")

    def power_cycle_dut(self, off_seconds=2):
        self.send_command("RELAY:OFF")
        time.sleep(off_seconds)
        self.send_command("RELAY:ON")

    def read_dut_led(self, pin):
        return self.send_command(f"GPIO_READ:{pin}") == "HIGH"

@pytest.fixture(scope="session")
def hil():
    interface = HILInterface()
    yield interface
    interface.serial.close()

@pytest.fixture
def mqtt_monitor():
    """Monitor MQTT messages from DUT."""
    import paho.mqtt.client as mqtt
    messages = []

    def on_message(client, userdata, msg):
        messages.append({
            'topic': msg.topic,
            'payload': msg.payload.decode(),
            'timestamp': time.time()
        })

    client = mqtt.Client()
    client.on_message = on_message
    client.connect("localhost", 1883)
    client.subscribe("dut/#")
    client.loop_start()

    yield messages

    client.loop_stop()
    client.disconnect()

---

1573.6 Writing HIL Test Cases

1573.6.1 Temperature Sensor Tests

import pytest
import time

class TestTemperatureSensor:
    """Test DUT temperature reading and reporting behavior."""

    def test_normal_reading(self, hil, mqtt_monitor):
        """DUT should report correct temperature within tolerance."""
        # Simulate 25C
        hil.set_temperature(25.0)
        time.sleep(5)  # Wait for DUT to read and publish

        # Check MQTT message
        temp_msgs = [m for m in mqtt_monitor if 'temperature' in m['topic']]
        assert len(temp_msgs) > 0, "No temperature message received"

        reported_temp = float(temp_msgs[-1]['payload'])
        assert abs(reported_temp - 25.0) < 1.0, \
            f"Temperature {reported_temp}C not within 1C of expected 25C"

    def test_sensor_out_of_range(self, hil, mqtt_monitor):
        """DUT should flag invalid readings outside sensor range."""
        # Simulate impossible temperature (sensor failure)
        hil.set_temperature(-100.0)  # Below NTC range
        time.sleep(5)

        status_msgs = [m for m in mqtt_monitor if 'status' in m['topic']]
        assert any('sensor_error' in m['payload'] for m in status_msgs), \
            "DUT should report sensor error for out-of-range reading"

    @pytest.mark.parametrize("temp", [-40, 0, 25, 50, 85])
    def test_full_range(self, hil, mqtt_monitor, temp):
        """DUT should accurately read across full operating range."""
        hil.set_temperature(temp)
        time.sleep(5)

        temp_msgs = [m for m in mqtt_monitor if 'temperature' in m['topic']]
        reported = float(temp_msgs[-1]['payload'])
        assert abs(reported - temp) < 2.0, \
            f"At {temp}C, reported {reported}C (off by {abs(reported-temp):.1f}C)"

1573.6.2 Power Resilience Tests

class TestPowerResilience:
    """Test DUT behavior during power failures."""

    def test_clean_boot_after_power_loss(self, hil, mqtt_monitor):
        """DUT should boot cleanly and resume operation after power loss."""
        mqtt_monitor.clear()

        # Power cycle
        hil.power_cycle_dut(off_seconds=5)
        time.sleep(60)  # Allow full boot sequence

        # Verify operational
        assert len(mqtt_monitor) > 0, \
            "DUT should resume MQTT publishing after power cycle"

    def test_power_during_ota_update(self, hil, mqtt_monitor):
        """DUT should recover if power lost during OTA update."""
        # Trigger OTA update
        hil.send_command("TRIGGER_OTA")
        time.sleep(5)  # Wait for update to start

        # Pull power mid-update
        hil.power_cycle_dut(off_seconds=3)
        time.sleep(90)  # Allow recovery boot

        # Check DUT is operational (either old or new firmware)
        assert len(mqtt_monitor) > 0, \
            "DUT should recover to operational state after interrupted OTA"

    @pytest.mark.parametrize("off_duration", [0.1, 0.5, 1.0, 2.0, 5.0])
    def test_brownout_recovery(self, hil, mqtt_monitor, off_duration):
        """DUT should handle various brownout durations."""
        mqtt_monitor.clear()
        hil.power_cycle_dut(off_seconds=off_duration)
        time.sleep(60)

        assert len(mqtt_monitor) > 0, \
            f"DUT should recover from {off_duration}s power loss"

---

1573.7 Failure Injection Scenarios

1573.7.1 Advanced HIL Scenarios

def test_i2c_sensor_nack(hil):
    """DUT should handle I2C sensor not responding (NACK)."""
    hil.send_command("I2C_DISCONNECT:TEMP_SENSOR")
    time.sleep(10)

    # DUT should report error, not crash
    assert hil.read_dut_led(STATUS_LED_PIN) == True, \
        "Status LED should indicate error"

def test_ntp_sync_failure(hil, mqtt_monitor):
    """DUT should handle NTP server unreachable."""
    hil.send_command("DNS_BLOCK:pool.ntp.org")
    time.sleep(3600)  # Run for 1 hour

    # Check timestamps are still monotonic
    timestamps = [m['timestamp'] for m in mqtt_monitor]
    assert all(t1 <= t2 for t1, t2 in zip(timestamps, timestamps[1:])), \
        "Timestamps should remain monotonic even without NTP"

---

1573.8 Arduino Interface Firmware

// interface_board.ino
#include <Wire.h>
#include <Adafruit_MCP4725.h>

Adafruit_MCP4725 dac;
const int RELAY_PIN = 7;
const int DUT_LED_PIN = 8;

void setup() {
    Serial.begin(115200);
    dac.begin(0x62);
    pinMode(RELAY_PIN, OUTPUT);
    pinMode(DUT_LED_PIN, INPUT);
    digitalWrite(RELAY_PIN, HIGH);  // DUT power ON by default
}

void loop() {
    if (Serial.available()) {
        String cmd = Serial.readStringUntil('\n');
        handleCommand(cmd);
    }
}

void handleCommand(String cmd) {
    if (cmd.startsWith("TEMP:")) {
        float temp = cmd.substring(5).toFloat();
        setTemperature(temp);
        Serial.println("OK");
    }
    else if (cmd == "RELAY:OFF") {
        digitalWrite(RELAY_PIN, LOW);
        Serial.println("OK");
    }
    else if (cmd == "RELAY:ON") {
        digitalWrite(RELAY_PIN, HIGH);
        Serial.println("OK");
    }
    else if (cmd.startsWith("GPIO_READ:")) {
        int pin = cmd.substring(10).toInt();
        Serial.println(digitalRead(pin) == HIGH ? "HIGH" : "LOW");
    }
    else {
        Serial.println("ERROR:UNKNOWN_COMMAND");
    }
}

void setTemperature(float celsius) {
    // Convert temperature to DAC value for NTC simulation
    float T = celsius + 273.15;
    float B = 3950;
    float R25 = 10000;
    float R_pullup = 10000;

    float R = R25 * exp(B * (1/T - 1/298.15));
    float voltage = 3.3 * R / (R + R_pullup);
    int dacValue = (voltage / 3.3) * 4095;

    dac.setVoltage(dacValue, false);
}

---

1573.9 CI/CD Integration

1573.9.1 GitHub Actions Workflow for HIL Tests

name: HIL Tests

on:
  push:
    branches: [main]
  schedule:
    - cron: '0 2 * * *'  # Nightly at 2 AM

jobs:
  hil-test:
    runs-on: [self-hosted, hil-runner]  # Physical machine with HIL setup
    steps:
      - uses: actions/checkout@v3

      - name: Flash DUT with latest firmware
        run: |
          pio run -e esp32dev
          pio run -e esp32dev -t upload

      - name: Run HIL Tests
        run: pytest hil_tests/ -v --tb=short --junitxml=hil-results.xml

      - name: Upload Test Results
        uses: actions/upload-artifact@v3
        with:
          name: hil-test-results
          path: hil-results.xml

---

1573.10 Common HIL Pitfalls and Solutions

Pitfall	Symptoms	Solution
Timing issues	Flaky tests	Add explicit waits, use message-based sync
Ground loops	Noise in analog signals	Use optoisolators, common ground
USB disconnect	Interface board resets	Use powered USB hub, monitor connection
DUT boot timing	Tests fail on first message	Wait for “ready” message from DUT
DAC settling time	Wrong readings	Add 10ms delay after DAC write
Relay bounce	Multiple power cycles	Add debounce delay (50ms)

---

1573.11 HIL Test Coverage Checklist

Sensor Simulation:
[ ] Normal operating range readings
[ ] Edge of range (min/max)
[ ] Out of range (sensor failure)
[ ] Noisy sensor (add random noise)
[ ] Disconnected sensor (I2C NACK / no response)

Network Scenarios:
[ ] Normal connectivity
[ ] Wi-Fi disconnect/reconnect
[ ] Cloud server unreachable
[ ] DNS failure
[ ] High latency (1000ms+)
[ ] Packet loss (10%, 50%)

Power Scenarios:
[ ] Clean boot
[ ] Power cycle (various durations)
[ ] Brownout (voltage dip)
[ ] Power during OTA update
[ ] Power during flash write

Environmental:
[ ] Temperature extremes (via thermal chamber)
[ ] Humidity (if applicable)
[ ] EMI (via signal injection)

---

1573.12 Knowledge Check

InlineKnowledgeCheck({
  questionId: "kc-testing-hil-1",
  question: "Your smart home hub supports 15 different Zigbee sensors from 8 manufacturers. Manual testing requires 2 hours per sensor (pairing, commands, firmware update, recovery). You have 3 firmware releases per month. Total monthly testing: 15 sensors x 2 hours x 3 releases = 90 hours. Your team proposes building a device farm with automated testing. Upfront cost: $8,000 hardware + 120 hours development. Which factors justify the investment?",
  options: [
    "The $8,000 upfront cost is too high - continue manual testing to save budget",
    "Device farm pays for itself after 3 months (90h/month manual vs automated), enables faster releases, catches integration bugs",
    "Device farms are only valuable for companies with 100+ devices - your 15 sensors are too few",
    "Automated testing can't replace manual testing for IoT because it can't detect real-world issues like range or interference"
  ],
  correctAnswer: 1,
  feedback: [
    "Incorrect. Let's calculate ROI: Manual testing costs 90h/month at $50/h = $4,500/month. Device farm: $8,000 one-time + (120h x $50/h) = $14,000 upfront. Break-even: 14,000 / 4,500 = 3.1 months. After 3 months, device farm saves $4,500/month indefinitely.",
    "Correct! Device farm justification has multiple dimensions: 1) Direct cost savings: After 3-month payback, saves $4,500/month in testing labor. 2) Velocity: Manual 90h = 2.25 weeks per release. Automated = overnight. Enables weekly releases instead of monthly. 3) Quality: Humans forget steps, skip tests under deadline pressure. Automation runs 100% of tests every time.",
    "Incorrect. Device farms scale to any size. Even 5 devices benefit from automation if testing is repetitive and frequent.",
    "Incorrect. While device farms can't fully test RF range, they handle 80% of testing: pairing, commands, firmware updates, power cycling. Supplement with targeted field trials for RF validation."
  ],
  hint: "Calculate the break-even point: upfront cost divided by monthly savings."
})

---

1573.13 Summary

Hardware-in-the-Loop testing enables automated firmware validation:

• HIL Architecture: Test PC + Interface Board + DUT enables controlled testing
• Sensor Simulation: DACs simulate analog sensors for repeatable tests
• Failure Injection: Test device behavior under power loss, sensor failure, network issues
• CI/CD Integration: Run HIL tests nightly on self-hosted runners
• Coverage: Use checklists to ensure comprehensive failure mode testing

---

1573.14 What’s Next?

Continue your testing journey with these chapters:

• Environmental Testing: Validate operation under temperature, humidity, EMC
• Field Testing and Deployment: Beta programs and production validation
• Testing Overview: Return to the complete testing guide