IoT Fundamentals · Study deck
Packet Framing
Picture a meter stream where one damaged byte makes every later reading slide into the wrong field.
Physics Phoebe is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: Framing answers "where is the message?" Error detection answers "are the located bytes intact?" A robust receiver does both: it finds the boundary, then runs the checksum or CRC over the framed bytes.
- Explain: Walkthrough: From Stream to Reliable Boundaries Pick the boundary method.: Fixed-length for uniform records, length-prefix for variable binary messages, or a delimiter for text and streams where resynchronization matters.
- Explain: The practical job is to choose a boundary method, handle the cases where data could imitate a boundary, and define how large messages are split and rebuilt.
Major section
In 60 Seconds · Start With the Story
The receiver needs a clear way to find the next valid message.
- JavaScript Object Notation is a text format for named values; it is shortened to JSON.
- A payload means the part of a message that carries the application data.
- Framing is how a receiver finds message boundaries in a continuous byte stream.
Major section
Finding Message Boundaries in a Stream
Framing adds those spaces and full stops to a byte stream.
- A communication link often delivers a continuous run of bytes with no built-in marks between messages.
- Framing is the agreement that lets a receiver split that stream back into the discrete packets the sender intended.
- Both work, and each carries different trade-offs.
Major section
Finding Message Boundaries in a Stream (continued)
Without framing, the receiver has data but no idea where one message stops and the next begins.
- The letters are all there, but you can only read it once you agree where words begin and end.
- In practice, the boundary rule is visible in the trace.
- A text log protocol may use newline delimiters.
Major section
Finding Message Boundaries in a Stream (continued)
You can tell the receiver how long each message is with a length field, or you can place a special delimiter marker between messages.
- A UART sensor protocol may start with a sync byte and a length byte.
- A BLE notification, LoRaWAN payload, or low-power serial link may choose a compact binary layout to avoid wasting bytes on repeated markers.
- The receiver can only parse safely when that rule is explicit.
Major section
Apply It: Choose and Implement a Framing Scheme
The practical job is to choose a boundary method, handle the cases where data could imitate a boundary, and define how large messages are split and rebuilt.
- A clear, written framing rule lets two independent implementations interoperate.
- Bound the maximum so a corrupt or hostile length cannot request an unreasonable read or buffer.
Major section
Apply It: Choose and Implement a Framing Scheme (continued)
That last column is the exact transformation performed in the worked example above.
- Walkthrough: From Stream to Reliable Boundaries Pick the boundary method.: Fixed-length for uniform records, length-prefix for variable binary messages, or a delimiter for text and streams where resynchronization matters.
- The result is that the flag value only ever appears as a real boundary on the wire, never inside payload data.
- Every frame is the same known size.
Major section
Apply It: Choose and Implement a Framing Scheme (continued)
If delimiter, define escaping.: Choose the marker, then specify how payload bytes equal to the marker, and to the escape byte itself, are escaped on send and restored on receive.
- A length field makes the header count authoritative but can lose synchronization when that count is corrupt.
- Delimiters make frame edges visible and help the receiver resynchronize.
- Wastes space for short messages and cannot carry variable data.
Major section
Apply It: Choose and Implement a Framing Scheme (continued)
Byte stuffing preserves those visible edges when the payload itself contains 0x7E, at the cost of variable overhead.
- A corrupted length desynchronizes the stream until recovery logic acts.
- Escaping adds bytes and processing to every frame.
- If your job is to define and implement a framing scheme, you can stop here.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly
The deeper layer explains how a receiver first locks onto the bits, how stuffing keeps a marker unique, and how a corrupted boundary recovers.
- Bit Synchronization Comes First Before bytes can even be read, the receiver must lock onto the bit timing.
- The receiver removes the stuffed bit.
- The largest frame the link can carry.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
Both techniques guarantee the boundary marker is something only the framing layer produces.
- A preamble, a known alternating pattern, lets the receiver's clock synchronize, and a start-frame delimiter then marks the first real bit of the frame.
- The cost is the escaping overhead on every frame.
- Fragments may arrive out of order.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
Oversize messages are dropped or silently truncated.
- Without this step, even uncorrupted bytes can be sampled at the wrong boundaries and decoded as nonsense.
- Byte stuffing is the byte-oriented version of the same idea, escaping any payload byte that matches the flag or escape value.
- Fragments are reassembled in the wrong order.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
A way to know a message exceeds the limit and must be split.
- Losing and Regaining Frame Synchronization Length-prefix framing is efficient but fragile to a corrupted length.
- After corruption, the receiver can discard bytes until it finds the next delimiter and resume at the next frame.
- Knowing when the set is whole.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
The bottom release checks require evidence that fragment cost, loss, and buffer use fit the target.
- A reassembly timeout and a buffer limit.
- Framing Versus Error Detection Framing and error detection are different layers that are easy to conflate.
- Documenting each step is what lets two independent implementations exchange packets reliably.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
Framing answers "where is the message?" Error detection answers "are the located bytes intact?" A robust receiver does both: it finds the boundary, then runs the checksum or CRC over the framed bytes.
- A correct boundary with corrupt contents, or correct contents read at the wrong boundary, are distinct failures.
- Common Pitfalls Trusting a length field without bounding it.: A corrupt or hostile length can request a huge read or buffer allocation.
- Escaping only the delimiter.: The escape byte itself must also be escaped, or its appearance in data becomes ambiguous.
Major section
Under the Hood: Synchronization, Stuffing, and Reassembly (continued)
The receiver uses offsets to restore byte order, but a missing range prevents delivery of the whole datagram.
- Confusing framing with error detection.: Finding a boundary does not prove the bytes inside it are correct.
- Omitting a reassembly timeout.: A single missing fragment can hold buffers open until they are exhausted.
- Assuming byte alignment without bit synchronization.: Without a preamble and start marker, even correct bytes can be mis-framed.
Major section
Summary
Framing is the agreement that lets a receiver split a continuous byte stream into discrete messages.
- The common boundary methods are fixed length, a length-prefix field, and a delimiter marker.
- Delimiter framing needs escaping, through byte or bit stuffing, so payload data cannot imitate the boundary marker.
- A length field must be bounded so a corrupt value cannot desynchronize the stream or request an unreasonable buffer.
Deck summary
Key takeaways
The receiver needs a clear way to find the next valid message.
- Framing adds those spaces and full stops to a byte stream.
- Without framing, the receiver has data but no idea where one message stops and the next begins.
- You can tell the receiver how long each message is with a length field, or you can place a special delimiter marker between messages.
- The practical job is to choose a boundary method, handle the cases where data could imitate a boundary, and define how large messages are split and rebuilt.
Retrieval practice
Recall check 1 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q1What core problem does framing solve?
Show answer
Answer: A Framing supplies the boundaries that turn an undifferentiated byte stream back into discrete messages.
Retrieval practice
Recall check 2 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q2A delimiter-framed protocol uses a flag byte to mark boundaries, but payloads sometimes contain that exact byte value. What must the design do?
Show answer
Answer: A Byte stuffing keeps the flag value unique to real boundaries while still allowing arbitrary payload data.
Retrieval practice
Recall check 3 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q3In a length-prefix scheme with no other protection, a single bit flips inside one frame's length field. What is the most likely consequence?
Show answer
Answer: A A wrong length shifts the next boundary, which cascades into every subsequent frame until resynchronization.
Print reference
Answers
Answer key.
- A · Framing supplies the boundaries that turn an undifferentiated byte stream back into discrete messages.
- A · Byte stuffing keeps the flag value unique to real boundaries while still allowing arbitrary payload data.
- A · A wrong length shifts the next boundary, which cascades into every subsequent frame until resynchronization.