%%{init: {'theme': 'base', 'themeVariables': {'primaryColor':'#2C3E50','primaryTextColor':'#fff','primaryBorderColor':'#16A085','lineColor':'#16A085','secondaryColor':'#E67E22','tertiaryColor':'#7F8C8D','background':'#ffffff','mainBkg':'#2C3E50','secondBkg':'#16A085','tertiaryBkg':'#E67E22','textColor':'#2C3E50'}}}%%
graph TD
Number["32-bit Number<br/>0x12345678"]
subgraph BigEndian["Big-Endian (Network Byte Order)"]
BE0["Address 0x00:<br/><b>0x12</b><br/>Most Significant"]
BE1["Address 0x01:<br/><b>0x34</b>"]
BE2["Address 0x02:<br/><b>0x56</b>"]
BE3["Address 0x03:<br/><b>0x78</b><br/>Least Significant"]
end
subgraph LittleEndian["Little-Endian (Intel x86, ARM)"]
LE0["Address 0x00:<br/><b>0x78</b><br/>Least Significant"]
LE1["Address 0x01:<br/><b>0x56</b>"]
LE2["Address 0x02:<br/><b>0x34</b>"]
LE3["Address 0x03:<br/><b>0x12</b><br/>Most Significant"]
end
Number --> BigEndian
Number --> LittleEndian
BE0 --> BE1 --> BE2 --> BE3
LE0 --> LE1 --> LE2 --> LE3
style Number fill:#E67E22,stroke:#2C3E50,stroke-width:3px,color:#fff
style BE0 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style BE1 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style BE2 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style BE3 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style LE0 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style LE1 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style LE2 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style LE3 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
48 Bitwise Operations and Endianness
48.1 Learning Objectives
By the end of this chapter, you will be able to:
- Apply endianness concepts: Identify big-endian vs little-endian byte ordering
- Perform bitwise operations: Use AND, OR, XOR, shifts, and masks for IoT programming
- Manipulate hardware registers: Set, clear, and toggle individual bits
- Handle cross-platform communication: Convert between network and host byte orders
NoteRelated Chapters, Products, and Tools
Foundation Topics: - Data Representation Fundamentals - Overview and index - Number Systems and Data Units - Binary, decimal, hexadecimal - Text Encoding for IoT - ASCII, Unicode, and UTF-8 - Packet Structure and Framing - How data is packaged
Apply These Concepts: - Sensor Circuits and Signals - Signal processing - Encryption Architecture - Cryptographic data - Prototyping Hardware - Microcontroller programming
Learning Hubs: - Quiz Navigator - Test your understanding - Simulation Playground - Interactive tools
48.2 Prerequisites
Before reading this chapter, you should understand:
- Binary and hexadecimal number systems from Number Systems and Data Units
- What bytes are and how they store data (bits 0-7, MSB/LSB)
48.3 Endianness and Byte Ordering
When storing multi-byte numbers (16-bit, 32-bit, 64-bit), the order of bytes in memory matters. This is called endianness.
48.3.1 Big-Endian vs Little-Endian
Example: The 32-bit hexadecimal number 0x12345678
{fig-alt=“Endianness comparison diagram showing a 32-bit hexadecimal value 0x12345678 stored in memory using two different byte ordering schemes. The top section illustrates big-endian (network byte order) with four memory addresses 0x00 through 0x03 storing bytes in most-significant-first order: address 0x00 contains 0x12, 0x01 contains 0x34, 0x02 contains 0x56, and 0x03 contains 0x78. The bottom section shows little-endian (Intel x86 and ARM) ordering with the same addresses storing bytes in least-significant-first order: address 0x00 contains 0x78, 0x01 contains 0x56, 0x02 contains 0x34, and 0x03 contains 0x12. Both representations derive from the same source value, demonstrating how processor architecture determines memory layout.”} {#fig-endianness}
%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22', 'tertiaryColor': '#7F8C8D'}}}%%
graph LR
subgraph BIG["Big-Endian Protocols"]
NET["<b>TCP/IP</b><br/>Internet protocols<br/>htonl() required"]
MOD["<b>Modbus TCP</b><br/>Industrial control<br/>Standard network"]
JAVA["<b>Java/JVM</b><br/>Cross-platform<br/>Consistent order"]
end
subgraph LITTLE["Little-Endian Protocols"]
BLE["<b>Bluetooth LE</b><br/>Wearables, sensors<br/>ARM native"]
USB["<b>USB</b><br/>Peripherals<br/>x86/ARM native"]
I2C["<b>Most I2C sensors</b><br/>Check datasheet!<br/>Varies by vendor"]
end
subgraph DANGER["Danger Zone"]
MISMATCH["<b>ESP32 to Cloud</b><br/>Little-endian CPU<br/>Big-endian network<br/>Must convert!"]
end
NET --> MISMATCH
BLE --> MISMATCH
style NET fill:#E67E22,stroke:#2C3E50,stroke-width:2px,color:#000
style MOD fill:#E67E22,stroke:#2C3E50,stroke-width:2px,color:#000
style JAVA fill:#E67E22,stroke:#2C3E50,stroke-width:2px,color:#000
style BLE fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style USB fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style I2C fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style MISMATCH fill:#7F8C8D,stroke:#E67E22,stroke-width:3px,color:#fff
Big-Endian (Most Significant Byte First): - Used by: Network protocols (TCP/IP, Modbus), Java Virtual Machine - Why “Big”? The biggest (most significant) byte comes first - Memory layout: 0x12 at lowest address, 0x78 at highest address - Human-readable: Matches how we write numbers left-to-right
Little-Endian (Least Significant Byte First): - Used by: Intel x86, ARM Cortex-M (ESP32, STM32), Bluetooth LE - Why “Little”? The littlest (least significant) byte comes first - Memory layout: 0x78 at lowest address, 0x12 at highest address - Performance advantage: Processor can read lower bytes first during arithmetic
48.3.2 Endianness in IoT Protocols
Network protocols use big-endian (also called “network byte order”):
// Sending 16-bit temperature value 1234 (0x04D2) over TCP/IP
uint16_t temp = 1234;
uint16_t temp_network = htons(temp); // Host TO Network Short
// Bytes sent: 0x04, 0xD2 (big-endian, regardless of CPU)Bluetooth LE uses little-endian:
// BLE GATT characteristic: heart rate 72 BPM (0x0048)
uint16_t heart_rate = 72;
// Bytes sent: 0x48, 0x00 (little-endian, LSB first)
NoteReal-World Example: Multi-Sensor Gateway Data Aggregation
Scenario: A smart building gateway collects data from 50 sensors every minute. Each sensor sends a 32-bit timestamp (Unix epoch seconds).
Problem: Temperature sensors use ESP32 (little-endian ARM), but the central server expects network byte order (big-endian). Without conversion:
Sensor sends timestamp 1609459200 (Jan 1, 2021, 00:00:00 UTC): - Correct big-endian bytes: 0x60 0x5E 0x00 0x00 - ESP32 sends (WRONG): 0x00 0x00 0x5E 0x60 (little-endian) - Server interprets as: 96 seconds (Jan 1, 1970, 00:01:36 UTC) - Error: 51 years off! Data appears to be from 1970 instead of 2021
Impact: - Historical analytics show impossible negative time deltas - Energy usage graphs show 51 years of missing data - Triggered false “clock drift” alerts across all 50 sensors - Cost: 8 hours of debugging time ($800+ engineering cost)
Fix: Use htonl() (Host TO Network Long) function:
uint32_t timestamp = 1609459200;
uint32_t network_order = htonl(timestamp); // Converts to big-endian
client.write((uint8_t*)&network_order, 4); // Sends: 60 5E 00 00Prevention: Always use network byte order functions (htonl, htons, ntohl, ntohs) for multi-byte values in protocols.
WarningEndianness Mismatch Bug
Real scenario: ESP32 (little-endian) sends sensor data to Python server expecting network byte order:
// ESP32 code (WRONG - sends little-endian)
uint32_t timestamp = 1609459200;
client.write((uint8_t*)×tamp, 4); // Sends: 00 00 5E 60
// Python server receives
data = sock.recv(4)
timestamp = struct.unpack('>I', data)[0] # Expects big-endian
# Result: 96 (wrong!) instead of 1609459200Fix: Use explicit byte order conversion:
// ESP32 code (CORRECT)
uint32_t timestamp = 1609459200;
uint32_t timestamp_be = htonl(timestamp); // Convert to big-endian
client.write((uint8_t*)×tamp_be, 4); // Sends: 60 5E 00 00Practical rule: Always check protocol specifications for endianness. When in doubt, network protocols use big-endian.
48.4 Bitwise Operations
Bitwise operations manipulate individual bits within bytes, essential for: - Setting/clearing hardware register flags - Packing multiple boolean values into single bytes - Extracting protocol header fields - Efficient sensor data encoding
48.4.1 Basic Bitwise Operators
%%{init: {'theme': 'base', 'themeVariables': {'primaryColor':'#2C3E50','primaryTextColor':'#fff','primaryBorderColor':'#16A085','lineColor':'#16A085','secondaryColor':'#E67E22','tertiaryColor':'#7F8C8D','background':'#ffffff','mainBkg':'#2C3E50','secondBkg':'#16A085','tertiaryBkg':'#E67E22','textColor':'#2C3E50'}}}%%
graph TB
subgraph AND["AND Operation (&)<br/>Both bits must be 1"]
A1["1010 1100<br/>& 1111 0000"]
A2["= 1010 0000"]
end
subgraph OR["OR Operation (|)<br/>At least one bit is 1"]
O1["1010 1100<br/>| 0000 1111"]
O2["= 1010 1111"]
end
subgraph XOR["XOR Operation (^)<br/>Bits are different"]
X1["1010 1100<br/>^ 0101 0101"]
X2["= 1111 1001"]
end
subgraph NOT["NOT Operation (~)<br/>Flip all bits"]
N1["~1010 1100"]
N2["= 0101 0011"]
end
A1 --> A2
O1 --> O2
X1 --> X2
N1 --> N2
style A1 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style A2 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style O1 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style O2 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style X1 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style X2 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
style N1 fill:#2C3E50,stroke:#16A085,stroke-width:2px,color:#fff
style N2 fill:#16A085,stroke:#2C3E50,stroke-width:2px,color:#fff
{fig-alt=“Four-panel diagram illustrating fundamental bitwise operations with binary examples. The AND operation panel shows 1010 1100 & 1111 0000 resulting in 1010 0000, demonstrating that output bits are 1 only where both inputs are 1. The OR operation panel displays 1010 1100 | 0000 1111 producing 1010 1111, showing that output is 1 when at least one input is 1. The XOR operation panel presents 1010 1100 ^ 0101 0101 yielding 1111 1001, illustrating that output is 1 when inputs differ. The NOT operation panel shows ~1010 1100 resulting in 0101 0011, demonstrating bitwise inversion where each 0 becomes 1 and each 1 becomes 0. Each panel includes input values and resulting output with clear visual separation.”} {#fig-bitwise-operations}
%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#2C3E50', 'primaryTextColor': '#fff', 'primaryBorderColor': '#16A085', 'lineColor': '#16A085', 'secondaryColor': '#E67E22', 'tertiaryColor': '#7F8C8D'}}}%%
graph TB
subgraph AND_USE["AND: Extract / Read"]
AND1["<b>Check if sensor ready</b><br/>status & 0x80<br/>Is bit 7 set?"]
AND2["<b>Mask upper nibble</b><br/>value & 0x0F<br/>Keep only lower 4 bits"]
end
subgraph OR_USE["OR: Set / Enable"]
OR1["<b>Enable I2C peripheral</b><br/>register |= 0x04<br/>Turn ON bit 2"]
OR2["<b>Set multiple flags</b><br/>status |= (ERROR | WARN)<br/>Set error + warning"]
end
subgraph XOR_USE["XOR: Toggle / Flip"]
XOR1["<b>Toggle LED state</b><br/>led_reg ^= 0x01<br/>Flip bit 0 ON to OFF"]
XOR2["<b>Simple checksum</b><br/>crc ^= byte<br/>XOR all bytes"]
end
subgraph SHIFT_USE["Shift: Pack / Extract"]
SHIFT1["<b>Extract temp from 16-bit</b><br/>temp = raw >> 4<br/>Shift right 4 bits"]
SHIFT2["<b>Pack 2 values in byte</b><br/>(high << 4) | low<br/>Combine nibbles"]
end
style AND1 fill:#16A085,stroke:#2C3E50,stroke-width:1px,color:#fff
style AND2 fill:#16A085,stroke:#2C3E50,stroke-width:1px,color:#fff
style OR1 fill:#E67E22,stroke:#2C3E50,stroke-width:1px,color:#000
style OR2 fill:#E67E22,stroke:#2C3E50,stroke-width:1px,color:#000
style XOR1 fill:#2C3E50,stroke:#16A085,stroke-width:1px,color:#fff
style XOR2 fill:#2C3E50,stroke:#16A085,stroke-width:1px,color:#fff
style SHIFT1 fill:#7F8C8D,stroke:#2C3E50,stroke-width:1px,color:#fff
style SHIFT2 fill:#7F8C8D,stroke:#2C3E50,stroke-width:1px,color:#fff
Truth tables:
| A | B | A AND B | A OR B | A XOR B |
|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 |
| 0 | 1 | 0 | 1 | 1 |
| 1 | 0 | 0 | 1 | 1 |
| 1 | 1 | 1 | 1 | 0 |
48.4.2 Bit Shifts
Left shift (<<): Moves bits left, fills with zeros (multiplies by 2^n)
0b00001101 << 2 = 0b00110100
(13 << 2 = 52 = 13 x 4)Right shift (>>): Moves bits right, fills with sign bit or zero (divides by 2^n)
0b00001100 >> 2 = 0b00000011
(12 >> 2 = 3 = 12 / 4)48.4.3 Common Bit Manipulation Patterns
Setting a bit (turn ON):
// Set bit 3 of register to 1 (enable I2C)
register |= (1 << 3); // OR with mask 0b00001000Clearing a bit (turn OFF):
// Clear bit 5 of status byte (disable LED)
status &= ~(1 << 5); // AND with inverted mask 0b11011111Toggling a bit (flip state):
// Toggle bit 7 (invert MSB)
value ^= (1 << 7); // XOR with mask 0b10000000Checking if a bit is set:
// Check if error flag (bit 2) is set
if (status & (1 << 2)) {
printf("Error detected!\n");
}Extracting multi-bit fields:
// Extract bits 4-6 from sensor reading
// Example: 0b11010110, bits 4-6 = 101 (value 5)
uint8_t sensor = 0xD6; // 11010110
uint8_t field = (sensor >> 4) & 0x07; // Shift right 4, mask with 0b00000111
// Result: 0b00000101 = 548.4.4 Real IoT Example: Sensor Status Byte
Many sensors pack multiple flags into a single status byte:
// BME280 sensor status register (0xF3)
// Bit 7-4: Reserved
// Bit 3: measuring (1 = conversion running)
// Bit 2-1: Reserved
// Bit 0: im_update (1 = NVM data copying)
uint8_t status = i2c_read(0xF3);
// Check if sensor is busy
bool is_measuring = status & (1 << 3);
bool is_updating = status & (1 << 0);
if (is_measuring || is_updating) {
delay(10); // Wait before reading data
}
// More efficient: check both bits at once
if (status & 0b00001001) { // Mask for bits 3 and 0
delay(10);
}
NoteReal-World Example: LoRaWAN Device Status Byte
Scenario: A battery-powered agricultural soil sensor transmits data via LoRaWAN (51-byte payload limit). The device needs to report multiple status flags with every reading.
Traditional approach (wasteful):
bool temp_sensor_ok = true; // 1 byte
bool moisture_ok = true; // 1 byte
bool battery_low = false; // 1 byte
bool calibration_needed = false; // 1 byte
bool valve_open = true; // 1 byte
bool network_joined = true; // 1 byte
bool gps_fix = false; // 1 byte
bool rain_detected = true; // 1 byte
// Total: 8 bytes for status flagsOptimized approach (bitwise packing):
// Pack 8 status flags into 1 byte
uint8_t status = 0; // Start with all flags cleared
// Set individual flags using bitwise OR
status |= (1 << 0); // Bit 0: temp_sensor_ok = 1
status |= (1 << 1); // Bit 1: moisture_ok = 1
// Bit 2: battery_low = 0 (keep cleared)
// Bit 3: calibration_needed = 0
status |= (1 << 4); // Bit 4: valve_open = 1
status |= (1 << 5); // Bit 5: network_joined = 1
// Bit 6: gps_fix = 0
status |= (1 << 7); // Bit 7: rain_detected = 1
// Result: status = 0b10110011 = 0xB3
// Total: 1 byte for all status flagsSavings: - Payload size: 8 bytes to 1 byte (87.5% reduction) - Energy: Fewer bytes = faster transmission = 3.2 mAh saved per day (7 fewer bytes x 96 transmissions/day x 5 uA/byte) - Battery life: 7 extra days per year on a 2,000 mAh battery - Cost: $2 battery replacement avoided annually per device (x1,000 devices = $2,000/year savings)
Reading flags:
if (status & (1 << 2)) {
printf("Battery low - replace soon\n");
}Why this matters: With 1,000 agricultural sensors transmitting every 15 minutes, bitwise packing saves 4.03 GB/year of LoRaWAN bandwidth and extends field battery life significantly.
NoteBitwise Operations for Energy Efficiency
Why use bitwise operations in IoT? - Fast: Single CPU instruction (2-4 cycles) - Low power: No multiplication/division (expensive) - Compact: Pack 8 boolean flags into 1 byte instead of 8 bytes
Example: Status byte with 8 flags (temp OK, humidity OK, battery OK, etc.) uses 1 byte instead of 8 separate boolean variables.
48.4.5 Python Struct Module for Binary Data
When IoT devices communicate with Python servers/scripts, use the struct module:
import struct
# Pack 32-bit integer as big-endian (network byte order)
timestamp = 1609459200
packed = struct.pack('>I', timestamp) # '>' = big-endian, 'I' = uint32
# Result: b'\x60\x5e\x00\x00'
# Pack multiple values into binary message
# Format: '>H f B' = big-endian, uint16, float32, uint8
sensor_id = 42
temperature = 23.5
status = 0b10010001
message = struct.pack('>H f B', sensor_id, temperature, status)
# Result: 9 bytes (2 + 4 + 1 + 2 padding)
# Unpack received binary data
data = b'\x00\x2a\x41\xbc\x00\x00\x91'
sensor_id, temp, status = struct.unpack('>H f B', data)
print(f"Sensor {sensor_id}: {temp}C, Status: {status:08b}")48.5 Knowledge Check
TipQuiz 1: IoT Memory Address Decoding
Scenario: You’re programming an ESP32 microcontroller for a smart agriculture sensor. The datasheet shows the I2C peripheral control register at memory address 0x3FF53000. During debugging, you need to set bit 7 (the I2C enable bit) of the byte at offset 0x10 from this base address.
Think about: 1. What is the complete memory address in hexadecimal where you’ll write? (Base + offset) 2. What binary value should you write to enable I2C while keeping other bits unchanged? (Use bitwise OR) 3. If you read back 0x8F from that address, is I2C successfully enabled?
Key Insights: - Address calculation: 0x3FF53000 + 0x10 = 0x3FF53010 (hex addition) - Enable bit 7: Write 0x80 (binary 10000000) using bitwise OR to preserve other bits - Verification: 0x8F = 10001111 in binary - bit 7 is 1, so yes, I2C is enabled! - Real impact: Incorrect bit manipulation could disable other peripherals or crash the sensor, requiring a field technician visit costing $200+ per device
This is why every IoT developer must master hexadecimal and bitwise operations - a single bit error can mean the difference between a working $50 sensor and a $250 warranty replacement.
NoteScenario 1: Debugging a Sensor Reading
You’re debugging a temperature sensor connected to an ESP32. The serial monitor shows: Received bytes: 01 90. The sensor uses 16-bit big-endian format where the value represents temperature in tenths of a degree Celsius.
Question: What is the actual temperature reading?
Question: What temperature does 01 90 represent?
Explanation: B. 0x0190 = 400 in decimal, and the sensor reports tenths of a degree, so 400 = 40.0C. See the full walkthrough below.
TipAnswer
Temperature: 40.0C
Step-by-step solution:
- Combine bytes (big-endian): 0x01 is MSB, 0x90 is LSB
- Hexadecimal value: 0x0190
- Convert to decimal: 0x0190 = 1 x 256 + 9 x 16 + 0 = 256 + 144 = 400
- Apply scaling: 400 tenths = 40.0C
Key insight: Big-endian means the most significant byte comes first. If this were little-endian (like Intel/ARM), you’d read it as 0x9001 = 36,865 tenths = 3686.5C (obviously wrong!). Always check the datasheet for byte order.
NoteScenario 2: Packing Sensor Status Flags
You’re designing a battery-powered environmental sensor that needs to transmit 6 status flags over LoRaWAN (where every byte matters):
- Bit 0: Temperature sensor OK
- Bit 1: Humidity sensor OK
- Bit 2: Battery low warning
- Bit 3: Calibration needed
- Bit 4: Wi-Fi connected
- Bit 5: SD card present
Question: If the temperature sensor is OK, humidity sensor is OK, battery is low, and Wi-Fi is connected, what single byte value should you transmit?
Question: What byte value should you transmit?
Explanation: A. Bits 0, 1, 2, and 4 are set = 00010111 = 0x17.
TipAnswer
Byte value: 0x17 (decimal 23, binary 00010111)
Step-by-step solution:
Identify set bits:
- Bit 0: Temperature OK = 1
- Bit 1: Humidity OK = 1
- Bit 2: Battery low = 1
- Bit 3: Calibration needed = 0
- Bit 4: Wi-Fi connected = 1
- Bit 5: SD card present = 0
Build binary value: 00010111 (reading right to left: bits 0,1,2,4 are set)
Convert to hex: 0001 0111 = 0x17
Code implementation:
uint8_t status = 0; status |= (1 << 0); // Temp OK status |= (1 << 1); // Humidity OK status |= (1 << 2); // Battery low status |= (1 << 4); // Wi-Fi connected // Result: status = 0x17
Key insight: Bitwise packing saves 5 bytes compared to sending 6 separate boolean bytes. Over thousands of transmissions, this dramatically extends battery life.
NoteScenario 3: Cross-Platform Communication
Your ESP32 sensor (little-endian ARM) sends a 32-bit Unix timestamp to a cloud server. The timestamp value is 1704067200 (January 1, 2024, 00:00:00 UTC).
Question: 1. What bytes will the ESP32 send if you just write the raw memory bytes (wrong approach)? 2. What bytes should be sent for correct network byte order (big-endian)? 3. What C function converts between these formats?
Question: Which byte sequence is the correct network byte order for the timestamp 0x659A3480?
Explanation: B. Network byte order is big-endian (most-significant byte first).
TipAnswer
1. Raw little-endian bytes (WRONG):
First, convert 1704067200 to hexadecimal: - 1704067200 = 0x659A3480
Little-endian stores LSB first: 80 34 9A 65
2. Network byte order (CORRECT):
Big-endian stores MSB first: 65 9A 34 80
3. Conversion function:
#include <arpa/inet.h> // or <netinet/in.h>
uint32_t timestamp = 1704067200;
uint32_t network_order = htonl(timestamp); // Host TO Network LongWhat happens with wrong byte order: - Server receives: 80 34 9A 65 (little-endian) - Server interprets as big-endian: 0x80349A65 = 2,150,013,541 - Wrong date: May 2038 instead of January 2024!
Key insight: Always use htonl() (32-bit), htons() (16-bit), ntohl(), and ntohs() when sending multi-byte values over networks. This ensures correct communication regardless of processor architecture.
NoteScenario 4: Reading a Sensor Datasheet
You’re reading the datasheet for an I2C accelerometer. It says: - “X-axis acceleration is stored in registers 0x32 (low byte) and 0x33 (high byte)” - “Resolution: 10-bit, range: +/-2g, left-justified in 16-bit format”
You read register 0x32 = 0x40 and register 0x33 = 0x01.
Question: What is the actual acceleration value in g?
Question: What acceleration does the register pair 0x0140 represent (10-bit, +/-2g, left-justified)?
Explanation: B. Combine bytes = 0x0140, shift right 6 (left-justified) = 5, then scale 5 x (4g/1024) = 0.0195g.
TipAnswer
Acceleration: +0.0195g (approx 0.02g)
Step-by-step solution:
- Combine bytes (little-endian since low byte is at lower address):
- Low byte 0x32 = 0x40
- High byte 0x33 = 0x01
- Combined 16-bit value: 0x0140
- Handle left-justification:
- “Left-justified in 16-bit” means the 10-bit value occupies the upper 10 bits
- Shift right by 6 to get the actual 10-bit value: 0x0140 >> 6 = 5
- Interpret as signed value:
- 10-bit range: -512 to +511 (two’s complement)
- Value 5 is positive (MSB = 0)
- Convert to g:
- +/-2g range means: -2g to +2g over 1024 levels
- Resolution: 4g / 1024 = 0.00390625 g/LSB
- Acceleration: 5 x 0.00390625 = 0.0195g (or about 0.02g)
Alternative calculation: - Full scale: +/-2g = 4g total range - 10-bit resolution: 2^10 = 1024 levels - Value of 5: (5/512) x 2g = 0.0195g
Key insight: Datasheets often describe “left-justified” or “right-justified” data. Left-justified means you need to shift right to extract the actual value. Always check the bit layout diagram in the datasheet!
Knowledge Check: Bitwise Operations Quick Check
Concept: Using bitwise operations for hardware control.
Question: To set bit 3 of a register without changing other bits, which operation do you use?
Explanation: A is correct. OR with a mask sets bits: x | 1 = 1 (sets bit), x | 0 = x (preserves bit). 0x08 = 0b00001000 sets only bit 3. AND would clear other bits; XOR would toggle; assign would overwrite everything.
48.6 Visual Reference Gallery
NotePacket Encapsulation Layers (Alternative View)
Each network layer adds its own header to the data payload, creating a layered encapsulation structure. This visualization shows how a small sensor reading (perhaps 4 bytes) grows as it passes through the protocol stack: application layer adds topic/QoS information, transport adds port numbers and checksums, network adds IP addresses, and data link adds MAC addresses and frame delimiters. Understanding encapsulation helps explain why LoRaWAN’s 13-byte overhead can dwarf a 2-byte temperature reading.
48.7 Summary
Endianness and bitwise operations are essential for IoT programming:
- Endianness: Big-endian (network byte order, MSB first) vs little-endian (Intel/ARM, LSB first)
- Conversion Functions:
htonl(),htons(),ntohl(),ntohs()for cross-platform communication - Bitwise AND: Extract bits, check flags, mask values
- Bitwise OR: Set bits, enable features, combine flags
- Bitwise XOR: Toggle bits, simple checksums
- Bit Shifts: Pack/unpack data, multiply/divide by powers of 2
Key Takeaways:
- Always check protocol specs for endianness - network protocols use big-endian
- Bitwise operations are fast (single CPU instruction) and memory-efficient
- Pack multiple boolean flags into single bytes to save bandwidth and battery
- Use Python’s
structmodule for binary data parsing in server-side code - A single bit error can cause 51-year timestamp errors or crash sensors
48.8 What’s Next
Now that you understand data representation fundamentals, continue to:
- Packet Structure and Framing - Learn how data gets packaged for network transmission
- Data Formats for IoT - Explore JSON, CBOR, Protocol Buffers for IoT messaging
Or return to the Data Representation overview for the complete topic map.