16 Packet Framing

16.1 In 60 Seconds

Find One Message After a Broken Byte

Picture a meter stream where one damaged byte makes every later reading slide into the wrong field. The receiver needs a clear way to find the next valid message.

JavaScript Object Notation is a text format for named values; it is shortened to JSON. A payload means the part of a message that carries the application data.

Send two marked frames, corrupt or remove one byte in the first, and check whether the second is found and decoded. Keep the raw bytes, boundary rule, length, check result, rejection reason, and decoder version.

This test proves recovery for chosen faults, not every link failure. The deeper sections compare fixed size, length fields, delimiters, escaping, checks, and fragmentation.

Framing is how a receiver finds message boundaries in a continuous byte stream. Use a fixed size, a length field, or a delimiter; escape any payload bytes that can imitate a delimiter; and define fragmentation when one message is too large for the link frame.

16.2 Start With the Story

You will choose a framing rule and explain how a receiver finds message boundaries and recovers after losing alignment. Start with a byte stream and mark how lengths, delimiters, or escaped bytes identify each frame.

Track the byte stream across four beats to see how one explicit framing rule keeps the receiver aligned.

16.3 Finding Message Boundaries in a Stream

A communication link often delivers a continuous run of bytes with no built-in marks between messages. Framing is the agreement that lets a receiver split that stream back into the discrete packets the sender intended. Without framing, the receiver has data but no idea where one message stops and the next begins.

Picture a sentence written with no spaces or punctuation, such as thecatsatonthemat. The letters are all there, but you can only read it once you agree where words begin and end. Framing adds those spaces and full stops to a byte stream.

There are two common ways to mark boundaries. You can tell the receiver how long each message is with a length field, or you can place a special delimiter marker between messages. Both work, and each carries different trade-offs.

In practice, the boundary rule is visible in the trace. A UART sensor protocol may start with a sync byte and a length byte. A TCP application protocol may length-prefix each binary record because TCP is a stream, not a message service. A text log protocol may use newline delimiters. A BLE notification, LoRaWAN payload, or low-power serial link may choose a compact binary layout to avoid wasting bytes on repeated markers. The receiver can only parse safely when that rule is explicit.

The One-Minute Framing Decision

Beginner Examples

• A sensor that always sends the same fixed-size reading can use fixed-length frames, so the boundaries are implied by the constant length.
• A text-style protocol often ends each message with a newline, using that newline as a delimiter.
• A binary protocol often places a length field in the header so the receiver knows exactly how many payload bytes follow.

Boundary Knowledge Check

If this gives you the core idea, you can stop here. Continue to Practitioner when you need to choose and implement a framing scheme.

16.4 Apply It: Choose and Implement a Framing Scheme

The practical job is to choose a boundary method, handle the cases where data could imitate a boundary, and define how large messages are split and rebuilt. A clear, written framing rule lets two independent implementations interoperate.

Walkthrough: From Stream to Reliable Boundaries

1. Pick the boundary method. Fixed-length for uniform records, length-prefix for variable binary messages, or a delimiter for text and streams where resynchronization matters.
2. If length-prefix, define the field. Set its size, units, byte order, and whether it counts the header. Bound the maximum so a corrupt or hostile length cannot request an unreasonable read or buffer.
3. If delimiter, define escaping. Choose the marker, then specify how payload bytes equal to the marker, and to the escape byte itself, are escaped on send and restored on receive.
4. Add synchronization if the link needs it. A preamble and start marker let the receiver lock onto the bit timing before the frame body begins.
5. Define the MTU and a fragmentation rule. State how a large message is split, numbered, and rebuilt, including how missing, duplicate, and out-of-order fragments are handled and when reassembly times out.
6. Keep framing separate from error detection. Framing locates the message; a checksum or CRC then judges whether the located bytes are intact.

Worked Example: Byte Stuffing a Delimiter

Suppose a protocol uses a flag value to mark frame boundaries and an escape value to signal substitution. The transmitter scans the payload before sending:

• Wherever a payload byte equals the flag, replace it with the escape value followed by a substituted byte.
• Wherever a payload byte equals the escape value, replace it with the escape value followed by a different substituted byte.
• The receiver reverses the rule: when it sees the escape value, it consumes the next byte and restores the original.

The result is that the flag value only ever appears as a real boundary on the wire, never inside payload data.

The worked byte-stuffing example is easiest to place beside the other boundary choices in Figure 16.1. Compare how a receiver uses the Length Field, Delimiters, and Byte Stuffing columns before choosing a rule for a real payload. The diagram makes parser simplicity, resynchronization behavior, and payload transparency visible as separate trade-offs rather than one generic framing choice.

Read Figure 16.1 left to right. A length field makes the header count authoritative but can lose synchronization when that count is corrupt. Delimiters make frame edges visible and help the receiver resynchronize. Byte stuffing preserves those visible edges when the payload itself contains 0x7E, at the cost of variable overhead. That last column is the exact transformation performed in the worked example above.

Incremental Practice

Delimiter Escaping Knowledge Check

If your job is to define and implement a framing scheme, you can stop here. Continue to Under the Hood for synchronization, stuffing, and reassembly mechanics.

16.5 Under the Hood: Synchronization, Stuffing, and Reassembly

The deeper layer explains how a receiver first locks onto the bits, how stuffing keeps a marker unique, and how a corrupted boundary recovers.

Bit Synchronization Comes First

Before bytes can even be read, the receiver must lock onto the bit timing. A preamble, a known alternating pattern, lets the receiver's clock synchronize, and a start-frame delimiter then marks the first real bit of the frame. Without this step, even uncorrupted bytes can be sampled at the wrong boundaries and decoded as nonsense.

Stuffing Keeps a Marker Unique

When a frame marker is a specific bit pattern, the transmitter can use bit stuffing: it inserts an extra 0 bit after a run of consecutive 1 bits in the data, for example after five ones, so the data can never reproduce the marker pattern. The receiver removes the stuffed bit. Byte stuffing is the byte-oriented version of the same idea, escaping any payload byte that matches the flag or escape value. Both techniques guarantee the boundary marker is something only the framing layer produces.

Losing and Regaining Frame Synchronization

• Length-prefix framing is efficient but fragile to a corrupted length. If the length field flips, the receiver reads the wrong number of bytes, and every following boundary is wrong until a recovery mechanism, such as a timeout, an idle gap, or a resynchronization marker, restores alignment.
• Delimiter framing degrades more gracefully. After corruption, the receiver can discard bytes until it finds the next delimiter and resume at the next frame. The cost is the escaping overhead on every frame.

Read Figure 16.2 from the frame budget at the top to fragment placement and bounded reassembly below. The receiver uses offsets to restore byte order, but a missing range prevents delivery of the whole datagram. The bottom release checks require evidence that fragment cost, loss, and buffer use fit the target.

Inspect Figure 16.2 at Measured after target-stack IPv6/UDP compression and note that it highlights Measured after target-stack IPv6/UDP compression. Compare Application payload, which uses Application payload to show the next hand-off, before reading FRAG headers repeat datagram size: 300 B as the place that uses FRAG headers repeat datagram size: 300 B to control parsing and delivery. The comparison turns Reassembly needs state: fragments share a datagram tag and size, later fragments add an offset, and the receiver buffers and orders them, delivering only if all arrive before the 60-second timeout into a bounded Losing and Regaining Frame Synchronization choice.

Framing Versus Error Detection

Framing and error detection are different layers that are easy to conflate. Framing answers "where is the message?" Error detection answers "are the located bytes intact?" A robust receiver does both: it finds the boundary, then runs the checksum or CRC over the framed bytes. A correct boundary with corrupt contents, or correct contents read at the wrong boundary, are distinct failures.

Common Pitfalls

1. Trusting a length field without bounding it. A corrupt or hostile length can request a huge read or buffer allocation.
2. Escaping only the delimiter. The escape byte itself must also be escaped, or its appearance in data becomes ambiguous.
3. Confusing framing with error detection. Finding a boundary does not prove the bytes inside it are correct.
4. Omitting a reassembly timeout. A single missing fragment can hold buffers open until they are exhausted.
5. Assuming byte alignment without bit synchronization. Without a preamble and start marker, even correct bytes can be mis-framed.

Reassembly Knowledge Check

At this depth, framing is a chain of agreements: synchronize the bits, mark the boundary in a way data cannot imitate, recover when a boundary is lost, and split and rebuild messages too large for one frame. Documenting each step is what lets two independent implementations exchange packets reliably.

16.6 Summary

• Framing is the agreement that lets a receiver split a continuous byte stream into discrete messages.
• The common boundary methods are fixed length, a length-prefix field, and a delimiter marker.
• Delimiter framing needs escaping, through byte or bit stuffing, so payload data cannot imitate the boundary marker.
• A length field must be bounded so a corrupt value cannot desynchronize the stream or request an unreasonable buffer.
• Messages larger than the link’s MTU must be fragmented and reassembled with identifiers, ordering, completion markers, and a timeout.
• Framing locates a message, while error detection with a checksum or CRC is a separate layer that judges whether the bytes are intact.

16.7 See Also