IoT Fundamentals · Study deck
Choosing an IoT Data Format
Picture a temperature device sending 215 to a dashboard.
Physics Phoebe is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: The golden vector at the bottom is the executable bridge between specification and implementation: firmware and backend tests must both recover version 1, 37.5% VWC, 18.6 C, 23.4 C, and 3.612 V.
- Explain: In Figure: VineyardPayloadV1 is an eight-byte little-endian application body with, the upper band keeps identity and session evidence outside the application body, preventing an eight-byte payload claim from quietly omitting receiver context.
- Explain: A Sigfox tracker with a roughly 12-byte uplink may force a hand-packed layout with a version byte, bit flags, and scaled integers.
Major section
In 60 Seconds · Start With the Story
The number could mean 215 degrees, 21.5 degrees, a raw count, or an error code.
- The format is useful only when every receiver recovers the same field, unit, scale, time, and version.
- Bandwidth means how much data a link can carry in a given time.
Major section
Match Format to Link Budget
A data format is the language a device uses to write down a reading before it leaves the chip.
- The same temperature value can be sent as the text {"temp":23.5} or as a few packed bytes.
- Both carry the measurement, but they cost very different amounts of bandwidth, radio time, and battery.
Major section
Match Format to Link Budget (continued)
Labelling every item ("blue shirt", "toothbrush") is easy to unpack later but wastes space; that is JSON.
- Compressing everything into the smallest possible bundle is the most efficient but you must remember exactly where each item went; that is custom binary.
- A real format choice starts with the device's link, not the developer's preference.
- A soil probe on LoRaWAN at a low data rate may need CBOR or integer scaling because the available payload is much smaller.
Major section
Match Format to Link Budget (continued)
A Sigfox tracker with a roughly 12-byte uplink may force a hand-packed layout with a version byte, bit flags, and scaled integers.
- Beginner Examples A mains-powered home hub on Wi-Fi can send JSON freely; the link is comfortable and readability speeds development.
- A battery field sensor on a low-rate LoRaWAN profile may benefit from CBOR or scaled integers when its measured application-payload or airtime budget is tight.
- A Sigfox asset tracker with a roughly 12-byte uplink often needs hand-packed custom binary, because even CBOR may not fit.
Major section
Apply It: Choose and Size a Payload
The practical workflow starts with the link budget and ends with a short decode record.
- If it fits the budget with margin, you may be done.
- The gateway and wearable cases introduce tighter payload or energy limits, so measured JSON, CBOR, or a shared schema become candidates rather than automatic answers.
Major section
Apply It: Choose and Size a Payload (continued)
Under that profile, the application-payload budget is up to 51 bytes; other regions, data rates, MAC-command use, and device profiles have different limits.
- The order connects the workflow to a defensible stopping rule: move to a less inspectable format only when the measured constraint eliminates the simpler choice.
- A valid 25-byte CBOR example uses short text keys with the same integer-scaled values.
- The custom layout is exactly 8 bytes.
Major section
Apply It: Choose and Size a Payload (continued)
An "8-byte custom binary" claim is credible only after every byte has a stable meaning.
- Worked Example: An 8-Byte Vineyard Application Contract This example is deliberately bounded to an EU863-870 LoRaWAN uplink at DR0 -- LoRa SF12 at 125 kHz -- with no FOpts.
- Full field names are readable but exceed this declared application budget.
- Leaves radio-budget margin, but every byte needs an immutable decode contract.
Major section
Apply It: Choose and Size a Payload (continued)
The exact sample values are version 1, soil moisture 37.5% VWC, soil temperature 18.6 C, air temperature 23.4 C, and battery voltage 3.612 V.
- The eight application bytes carry a layout version and four measurements; the trusted ingestion envelope carries stable device identity, session and frame evidence, FPort, and receive time.
- DevAddr is session- and network-scoped, so the ingestion service maps it to a stable registry identity rather than treating it as a permanent business key.
- Encoding resolution is not sensor accuracy: 0.1 C storage does not prove a sensor is accurate to 0.1 C.
Major section
Apply It: Choose and Size a Payload (continued)
In Figure: VineyardPayloadV1 is an eight-byte little-endian application body with, the upper band keeps identity and session evidence outside the application body, preventing an eight-byte payload claim from quietly omitting receiver context.
- The golden vector at the bottom is the executable bridge between specification and implementation: firmware and backend tests must both recover version 1, 37.5% VWC, 18.6 C, 23.4 C, and 3.612 V.
- Receive windows, retries, sensor and MCU work, sleep current, and battery self-discharge are excluded.
- The calculation reports charge; energy in joules additionally requires supply voltage.
Major section
Apply It: Choose and Size a Payload (continued)
The 8-byte contract uses about 43.6% less TX charge than the 45-byte JSON body and about 24.9% less than the 25-byte CBOR body -- not five to six times less.
- Standards note:: LoRaWAN application-payload limits are region- and data-rate-specific and may shrink when FOpts carries MAC commands.
- Packet airtime should be verified with the selected radio library or the Semtech LoRa Calculator.
- A format that fits today still needs a versioned contract so future fields and older receivers can coexist.
Major section
Under the Hood: Encoding Mechanics, Schema, and Cost
The mechanics explain why the formats differ in size and why some changes are safe while others break deployed devices.
- Each format makes a different trade between readability, self-description, and bytes.
- Replaces text syntax with compact binary type bytes; keeps the JSON data model.
- No, needs the schema.
- Format-to-Protocol Fit Format.
Major section
Under the Hood: Encoding Mechanics, Schema, and Cost (continued)
Still grows with many fields.
- A deployable estimate uses the complete frame and selected radio profile, then adds receive windows, retries, sensing, MCU work, sleep current, and power-conversion losses as required by the system boundary.
- MessagePack is a similar binary-JSON bridge that needs no schema.
- Both sides need the compiled schema.
Major section
Under the Hood: Encoding Mechanics, Schema, and Cost (continued)
CBOR is an IETF standard (RFC 8949, originally RFC 7049) and pairs naturally with CoAP on constrained networks.
- Protocol Buffers are a schema-compiled format common in gRPC services where the contract is stable and versioned.
- XML remains in some legacy industrial and enterprise systems but is too verbose for tight radio links.
- A common waste is sending every number as a 64-bit float.
Major section
Under the Hood: Encoding Mechanics, Schema, and Cost (continued)
A temperature stored as 23.5 in IEEE 754 double precision takes 8 bytes.
- CBOR removes text syntax while retaining a self-describing data model; Protocol Buffers replaces names with schema field numbers; custom binary removes nearly all in-band description and relies on the external byte contract.
- Schema Evolution IoT fleets are rarely updated all at once.
- A trustworthy payload records each one instead of treating the bytes as self-explanatory.
Major section
Under the Hood: Encoding Mechanics, Schema, and Cost (continued)
Devices, gateways, and cloud consumers may run different versions for a long time, so the format has to survive staged change.
- Common Pitfalls Encoding numbers as strings.: Sending "23.5" instead of 23.5 forces string parsing, breaks numeric queries, and adds bytes.
- Using floats for every field.: Eight bytes per value is rarely needed; integer scaling keeps the precision and cuts the size.
- Ignoring schema evolution.: Adding or renaming a field without a versioning rule can break consumers that have not been updated yet.
Major section
Summary
Format selection is an evidence chain, not a contest for the fewest bytes.
- When measurement proves that a more compact encoding is necessary, move deliberately: integer scaling, compact standard formats, shared schemas, and custom layouts each save bytes by relocating descriptive information into a stronger external contract.
- The vineyard example shows how those decisions stay connected.
- Its link profile sets the application budget, measured encodings reveal which representations fit, and the byte diagram makes the smallest candidate independently decodable.
Deck summary
Key takeaways
The number could mean 215 degrees, 21.5 degrees, a raw count, or an error code.
- A data format is the language a device uses to write down a reading before it leaves the chip.
- Labelling every item ("blue shirt", "toothbrush") is easy to unpack later but wastes space; that is JSON.
- A Sigfox tracker with a roughly 12-byte uplink may force a hand-packed layout with a version byte, bit flags, and scaled integers.
- The practical workflow starts with the link budget and ends with a short decode record.
Retrieval practice
Recall check 1 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q1A smart-parking sensor uses NB-IoT with a 200-byte payload limit and sends a four-field message (spot ID, occupied flag, arrival time, battery level). Which format is the most appropriate first choice?
Show answer
Answer: B The message is roughly 90 to 100 bytes in JSON, well under 200, so the readable format is fine.
Retrieval practice
Recall check 2 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q2A sensor's application body shrinks from JSON at 95 bytes to CBOR at 50 bytes. What can you conclude before calculating the selected radio profile and complete packet?
Show answer
Answer: C Byte reduction is known immediately, but actual packet airtime and transmit charge require the radio settings, full frame length, fixed overhead, and rounding rules.
Retrieval practice
Recall check 3 of 3
Physics Phoebe says: answer from memory, then check your reasoning.
Q3Which situation most justifies the engineering effort of a hand-packed custom binary format?
Show answer
Answer: D A 12-byte limit can require exact bit-packing that even CBOR may not satisfy, so custom binary earns its complexity.
Print reference
Answers
Answer key.
- B · The message is roughly 90 to 100 bytes in JSON, well under 200, so the readable format is fine.
- C · Byte reduction is known immediately, but actual packet airtime and transmit charge require the radio settings, full frame length, fixed overhead, and rounding rules.
- D · A 12-byte limit can require exact bit-packing that even CBOR may not satisfy, so custom binary earns its complexity.