8  Bitwise Operations and Endianness

Byte Order, Masks, Shifts, Packed Flags, and Safe Decoding

8.1 In 60 Seconds

Bitwise decoding turns compact bytes into named fields. Endianness decides how multi-byte values are assembled, masks select individual bits, shifts move packed fields into place, and read-modify-write protects neighboring register settings. The safe habit is to treat byte order, bit positions, width, scaling, units, and reserved bits as one documented contract.

8.2 Start With the Story

Start with a tiny payload that says 0x01F4, where one team reads 500 and another reads the wrong value because the width, order, or encoding was never written down. The core idea in Bitwise Operations and Endianness is simple: data representation turns bits into agreed meaning through units, signedness, byte order, text encoding, and payload contracts. This page focuses that idea on IoT data representation endianness, network byte order, bit masks, shifts, register-safe read-modify-write patterns, packed status bytes. In everyday IoT, a meter reading, firmware flag, topic name, or diagnostic string is only trustworthy when both sender and receiver decode it the same way. Start simple: write the value, its units, its byte width, and the decoding rule before adding clever packing or optimization.

8.3 Decode Bytes by Contract

IoT devices often exchange data as bytes rather than text. A payload may carry a two-byte temperature, a four-byte timestamp, and a one-byte status register. To decode it safely, both sides must agree on byte order, bit positions, scaling, and field width.

Two ideas do most of the work. Endianness tells you which byte of a multi-byte value comes first. Bitwise operations let you test, set, clear, toggle, pack, and extract individual bits.

The core rule is precise: bits are numbered positions inside a byte, and a mask selects the positions you care about. AND checks or extracts, OR sets, XOR toggles, and shifts move fields into place. Any multi-byte field needs an agreed byte order before it crosses a device, gateway, file, or network boundary.

Think of a multi-digit number written by two people who never agreed which digit goes first. The same ink can read as two different numbers. A row of light switches is the other half of the picture: a mask is the list of switches you are allowed to touch, leaving the rest alone.

For example, a battery-powered sensor might send one status byte where bit 0 means the temperature sample is valid, bit 2 means low battery, and bit 4 means the gateway link is ready. The decoder should not judge the decimal value of the whole byte. It should apply the documented masks, test each flag, and leave reserved bits untouched so future firmware can add meanings without breaking old readers.

The value 0x12345678 in big-endian (12 34 56 78) and little-endian (78 56 34 12) byte order.

The One-Minute View

Resolve byte order first

The same bytes assemble into different numbers if the receiver picks the wrong first byte. Decide endianness before any value is read.

A mask selects bits

AND with a mask reads or extracts; OR sets; AND with an inverted mask clears; XOR toggles. Neighbors stay untouched.

Document the contract

Byte order, bit positions, width, scaling, and reserved bits are part of the data contract, not implementation trivia.

Beginner Examples

• Test whether bit 2 is set in a status byte by applying a mask that selects only that bit.
• Read a two-byte big-endian value by placing the first byte in the high position and the second in the low position.
• Enable one feature bit with OR so the other settings in the same register are preserved.

Bit Mask Knowledge Check

A safe decoder starts by treating byte order and masks as part of the payload contract, not as details to infer later.

8.4 Apply It: Decode Packed Bytes Safely

Bitwise work is safe when every operation is tied to a documented field width, byte order, and bit position. The workflow is the same for a packet field and a hardware register.

Walkthrough: Decode a Field at a Time

1. Freeze the layout. Write down byte order, field width, bit numbering, masks, scaling, and reserved bits before decoding.
2. Extract one field at a time. Use masks and shifts to isolate a value, then convert it to its documented unit.
3. Preserve neighbors. For register writes, read the current value, change only the intended bits, and write the combined value back.
4. Validate boundaries. Test all-zero, all-one, minimum, maximum, and reserved-bit examples.

In code review, the operator should match the intent without guesswork: status & 0x80 reads a ready flag, control |= 0x04 enables a peripheral, led_reg ^= 0x01 toggles a bit, and a shift moves packed fields into the documented position before comparison or scaling.

Real IoT code uses AND to read, OR to set, XOR to toggle, and shifts to pack or extract bit fields.

Operator

Purpose

Pattern

Typical IoT Use

AND

Keep only selected bits.

value & mask

Check status flags or extract a field.

OR

Set selected bits to one.

value | mask

Enable a feature bit while preserving neighbors.

XOR

Flip selected bits.

value ^ mask

Toggle a state or update a parity-style check.

NOT

Invert a mask.

~mask

Clear selected bits with value & ~mask.

Shift

Move a field left or right.

value << n, value >> n

Pack two nibbles or extract a field from a register.

Register-Safe Read-Modify-Write

Hardware registers often place several settings in one byte. A write that changes one bit must not change the others, so read the register, modify only the target bits, and write it back.

#define BIT(n)        (1u << (n))
#define MODE_MASK     (0x03u << 4)   /* bits 5:4 */

uint8_t reg = read_control_register();

reg |= BIT(2);             /* set bit 2 */
reg &= (uint8_t)~BIT(5);   /* clear bit 5 */
reg ^= BIT(0);             /* toggle bit 0 */

reg = (reg & (uint8_t)~MODE_MASK) | (0x02u << 4); /* replace bits 5:4 */

write_control_register(reg);

Set Use OR with a mask. Bits that are 0 in the mask stay unchanged.

Clear Use AND with an inverted mask. The target bit becomes 0; neighbors stay unchanged.

Toggle Use XOR when the state should flip. Do not use it when the bit must be guaranteed on or off.

Replace field Clear the field first, then OR in the shifted new value.

Worked Decoder: A Big-Endian Fixed-Point Field

A two-byte temperature field is specified as big-endian signed fixed point in tenths of a degree. The received bytes are 01 90:

1. Assemble big-endian: 0x0190.
2. Convert to decimal: 400.
3. Apply the scale: 400 / 10 = 40.0.
4. Validate against the sensor's expected operating range.

int16_t raw = (int16_t)(((uint16_t)bytes[0] << 8) | bytes[1]);
float temperature = raw / 10.0f;   /* 40.0 */

The same two bytes decoded little-endian would give 0x9001, a very different value, which is why byte order is part of the data contract.

Try It: Extract a Field by Hand

Given the byte 0b10110100, choose a mask for bits 2 through 4, AND it with the byte, then shift the result down to bit zero. Write the decimal value of the extracted three-bit field. (No special hardware needed; pencil and paper is enough.)

Incremental Practice

Beginner

Test whether bit 2 is set in a status byte by applying a mask that selects only that bit.

Intermediate

Extract a three-bit mode field by masking it and shifting it down to bit zero before comparing values.

Advanced

Decode a multi-byte packed payload by applying the agreed endianness first, then extracting scaled fields and reserved flags.

Byte Order Knowledge Check

The same discipline that decodes one field also protects shared hardware registers: isolate only the intended bits and prove the boundary cases.

8.5 Under the Hood: Packed Layouts, the Pipeline, and Validation

Packed-layout work treats bit packing as a precise, testable contract. A reliable decoder follows the same path every time, and a packed byte is only useful when its layout is documented and tested.

Packed Status Bytes

Packing flags into one byte is common when every byte must be justified. Each bit gets a name, a meaning, and a decode expression.

Bit

Meaning

Set When

Decode Expression

0

Temperature reading valid.

The temperature sample passed range and checksum checks.

status & BIT(0)

1

Humidity reading valid.

The humidity sample is ready for use.

status & BIT(1)

2

Battery warning.

The measured supply is below the warning threshold.

status & BIT(2)

4

Gateway link ready.

The next uplink can be attempted.

status & BIT(4)

If bits 0, 1, 2, and 4 are set, the byte adds up as 1 + 2 + 4 + 16 = 23:

bit positions:  7 6 5 4 3 2 1 0
status bits:    0 0 0 1 0 1 1 1
hex value:      0x17

The Decode Pipeline

Load the bytes, assemble fields in the agreed order, mask and shift, scale the value, then validate.

The decode pipeline: load, order, mask and shift, scale, then validate.

Stage

Action

Evidence

Failure Mode

Load

Read the exact bytes from the register, packet, or file.

Hex dump and byte count.

Off-by-one offsets or missing bytes.

Order

Assemble multi-byte values in the specified byte order.

Known input and expected numeric value.

Plausible-looking but wrong values.

Mask and shift

Select the bit field and move it into position.

A worked binary example.

Wrong field width or shift distance.

Scale and validate

Apply the unit conversion, then check range and reserved bits.

Unit notes plus boundary and invalid examples.

Correct bits but wrong unit, or silent corruption.

Validation Checklist

• Record byte order for every multi-byte field, and name each bit and reserved bit.
• Define field width, signedness, scale, and offset.
• Include a normal example and each boundary value; test invalid and reserved-bit states.
• Confirm write operations preserve neighboring register bits, and keep decoder tests with the source.

Common Pitfalls

1. Sending raw multi-byte memory. Memory layout can differ from the wire format; assemble values explicitly before crossing a boundary.
2. Overwriting neighbor bits. Direct assignment clears unrelated settings; use masks and read-modify-write.
3. Forgetting reserved bits. Ignore them on read and preserve them on write unless the spec says otherwise.
4. Shifting before defining width. Without field width, signedness, scale, and offset, the bit math can look right while the value is wrong.

Read-Modify-Write Knowledge Check

In production, bitwise work is precise engineering. Endianness defines how bytes assemble, masks select meaning, shifts move fields, read-modify-write protects neighbors, and validation catches wrong order, wrong width, and impossible states before they become silent corruption.

8.6 Summary

• Endianness defines how a multi-byte value is assembled; resolve it before reading any number.
• Masks select the bits that carry meaning: AND tests or extracts, OR sets, AND with an inverted mask clears, and XOR toggles.
• Shifts move fields into and out of packed positions.
• Read-modify-write protects neighboring register bits when changing a single setting.
• Packed payloads need documentation, a worked example, and decoder tests kept with the source.
• Validation catches wrong byte order, wrong field width, reserved-bit violations, and impossible states.

Key Takeaway

Bitwise operations are powerful for flags, masks, registers, and compact payloads, but they are safe only when bit positions, endianness, units, reserved bits, and versioning are documented and tested.

8.7 See Also

Number Systems and Data Units

Review binary, hexadecimal, bit positions, and fixed width behind these operations.

Data Representation Fundamentals

See how byte-interpretation contracts prevent decode errors across the wider topic.

Packet Anatomy

See how packed fields sit inside network frames with headers and trailers.