1318  Advanced Serialization Analysis

Trade-off Matrices, Network Simulation, and Schema Evolution

1318.1 Learning Objectives

By the end of this section, you will be able to:

• Use weighted trade-off matrices to make format decisions based on multiple priorities
• Understand how compression algorithms interact with different serialization formats
• Simulate network conditions to predict transmission performance
• Evaluate schema evolution capabilities for long-term system maintenance
• Assess memory footprint requirements for constrained devices

1318.2 Introduction

Choosing a serialization format involves balancing multiple competing priorities. This chapter provides advanced analysis tools including weighted decision matrices, compression synergy analysis, network condition simulation, schema evolution comparison, and memory footprint analysis for embedded systems.

1318.3 Trade-off Decision Matrix

Use this weighted scoring system to find the best format based on your specific priorities.

ieeeColors = ({
  navy: "#2C3E50",
  teal: "#16A085",
  orange: "#E67E22",
  gray: "#7F8C8D",
  red: "#E74C3C",
  green: "#27AE60",
  purple: "#8E44AD",
  blue: "#3498DB",
  yellow: "#F1C40F",
  darkGray: "#34495E",
  lightGray: "#ECF0F1"
})

// Format definitions
formats = [
  { id: "json", name: "JSON", color: ieeeColors.orange, overhead: 1.0, encodeSpeed: 0.8, schemaRequired: false, humanReadable: true },
  { id: "msgpack", name: "MessagePack", color: ieeeColors.teal, overhead: 0.6, encodeSpeed: 0.4, schemaRequired: false, humanReadable: false },
  { id: "protobuf", name: "Protocol Buffers", color: ieeeColors.blue, overhead: 0.4, encodeSpeed: 0.3, schemaRequired: true, humanReadable: false },
  { id: "cbor", name: "CBOR", color: ieeeColors.purple, overhead: 0.55, encodeSpeed: 0.35, schemaRequired: false, humanReadable: false },
  { id: "avro", name: "Apache Avro", color: ieeeColors.green, overhead: 0.45, encodeSpeed: 0.5, schemaRequired: true, humanReadable: false },
  { id: "binary", name: "Raw Binary", color: ieeeColors.red, overhead: 0.25, encodeSpeed: 0.1, schemaRequired: true, humanReadable: false }
]

viewof priorities = Inputs.form({
  sizeWeight: Inputs.range([0, 10], { label: "Size Priority", value: 7, step: 1 }),
  speedWeight: Inputs.range([0, 10], { label: "Speed Priority", value: 5, step: 1 }),
  flexWeight: Inputs.range([0, 10], { label: "Flexibility Priority", value: 4, step: 1 }),
  compatWeight: Inputs.range([0, 10], { label: "Compatibility Priority", value: 6, step: 1 }),
  debugWeight: Inputs.range([0, 10], { label: "Debuggability Priority", value: 3, step: 1 })
})

weightedScores = formats.map(f => {
  const sizeScore = (1 - f.overhead) * 10;
  const speedScore = (1 - f.encodeSpeed) * 10;
  const flexScore = f.schemaRequired ? 4 : 8;
  const compatScore = f.id === "json" ? 10 : f.id === "cbor" ? 7 : f.id === "msgpack" ? 6 : 5;
  const debugScore = f.humanReadable ? 10 : 3;

  const totalWeight = priorities.sizeWeight + priorities.speedWeight + priorities.flexWeight + priorities.compatWeight + priorities.debugWeight;

  const weightedScore = totalWeight > 0 ? (
    (sizeScore * priorities.sizeWeight +
     speedScore * priorities.speedWeight +
     flexScore * priorities.flexWeight +
     compatScore * priorities.compatWeight +
     debugScore * priorities.debugWeight) / totalWeight
  ) : 0;

  return {
    format: f,
    scores: { size: sizeScore, speed: speedScore, flex: flexScore, compat: compatScore, debug: debugScore },
    weightedScore: weightedScore.toFixed(2)
  };
}).sort((a, b) => parseFloat(b.weightedScore) - parseFloat(a.weightedScore))

html`
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">

  <div style="background: linear-gradient(135deg, ${weightedScores[0].format.color}, ${ieeeColors.navy}); color: white; padding: 20px; border-radius: 12px; margin-bottom: 20px; text-align: center;">
    <div style="font-size: 14px; opacity: 0.9;">Based on Your Priorities</div>
    <div style="font-size: 28px; font-weight: bold; margin: 10px 0;">${weightedScores[0].format.name}</div>
    <div style="font-size: 16px;">Score: ${weightedScores[0].weightedScore}/10</div>
  </div>

  <div style="display: grid; gap: 10px;">
    ${weightedScores.map((item, i) => html`
      <div style="display: flex; align-items: center; gap: 15px; background: ${i === 0 ? ieeeColors.green + '15' : 'white'}; border: 1px solid ${i === 0 ? ieeeColors.green : ieeeColors.gray + '30'}; border-radius: 8px; padding: 15px;">
        <div style="width: 30px; height: 30px; background: ${item.format.color}; border-radius: 50%; display: flex; align-items: center; justify-content: center; color: white; font-weight: bold; font-size: 14px;">
          ${i + 1}
        </div>
        <div style="flex: 1;">
          <div style="font-weight: bold; color: ${item.format.color};">${item.format.name}</div>
          <div style="font-size: 11px; color: ${ieeeColors.gray};">
            Size: ${item.scores.size.toFixed(1)} | Speed: ${item.scores.speed.toFixed(1)} | Flex: ${item.scores.flex} | Compat: ${item.scores.compat} | Debug: ${item.scores.debug}
          </div>
        </div>
        <div style="background: ${item.format.color}; color: white; padding: 5px 15px; border-radius: 20px; font-weight: bold;">
          ${item.weightedScore}
        </div>
      </div>
    `)}
  </div>

</div>
`

(a)

(b)

(c)

(d)

(e)

Figure 1318.1: Interactive decision support for choosing serialization format

1318.4 Implementation Considerations

NoteLibrary Support by Platform

Format	Arduino/ESP32	Python	Node.js	Go	Rust
JSON	ArduinoJson	json (built-in)	JSON (built-in)	encoding/json	serde_json
MessagePack	msgpack-arduino	msgpack	msgpack5	msgpack	rmp-serde
Protocol Buffers	Nanopb	protobuf	protobufjs	protobuf	prost
CBOR	cn-cbor	cbor2	cbor	go-cbor	ciborium
Avro	Limited	fastavro	avsc	goavro	apache-avro
Raw Binary	Native	struct	Buffer	encoding/binary	byteorder

WarningCommon Pitfalls

1. Ignoring Schema Evolution: Protobuf and Avro handle schema changes; raw binary does not
2. Forgetting Endianness: Binary formats must agree on byte order
3. Overhead in Small Messages: Schema-based formats add per-message overhead
4. Compression Interaction: Some formats compress better than others with gzip/lz4
5. Security Considerations: Binary formats can hide malicious payloads; validate all inputs

1318.5 Compression Synergy Analysis

Different serialization formats interact differently with compression algorithms. Text formats like JSON compress much better than already-binary formats.

compressionAlgorithms = [
  { id: "none", name: "No Compression", ratio: 1.0, cpuCost: 0 },
  { id: "gzip", name: "gzip", ratio: 0.4, cpuCost: 5 },
  { id: "lz4", name: "LZ4", ratio: 0.6, cpuCost: 1 },
  { id: "zstd", name: "Zstandard", ratio: 0.35, cpuCost: 3 },
  { id: "snappy", name: "Snappy", ratio: 0.65, cpuCost: 0.5 }
]

// Base sizes for calculations
baseJsonSize = 100

viewof selectedCompression = Inputs.select(compressionAlgorithms, {
  label: "Additional Compression",
  format: d => d.name,
  value: compressionAlgorithms[0]
})

// Simulated serialization results
serializationResults = formats.map(f => ({
  format: f,
  size: Math.round(baseJsonSize * f.overhead),
  encodeTime: f.encodeSpeed
}))

compressionSynergyData = {
  return serializationResults.map(r => {
    // Text formats compress better than binary
    const compressibility = r.format.humanReadable ? 1.0 :
                           r.format.id === "msgpack" ? 0.7 :
                           r.format.id === "protobuf" ? 0.5 :
                           r.format.id === "cbor" ? 0.6 :
                           r.format.id === "avro" ? 0.55 : 0.4;

    const effectiveRatio = 1 - (1 - selectedCompression.ratio) * compressibility;
    const compressedSize = Math.round(r.size * effectiveRatio);
    const totalCpu = r.encodeTime + (selectedCompression.cpuCost * r.size / 100);

    return {
      format: r.format,
      originalSize: r.size,
      compressedSize: compressedSize,
      effectiveReduction: ((1 - compressedSize / baseJsonSize) * 100).toFixed(1),
      totalCpu: totalCpu.toFixed(2),
      worthIt: compressedSize < r.size * 0.9
    };
  }).sort((a, b) => a.compressedSize - b.compressedSize);
}

html`
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">

  <div style="background: white; border: 1px solid ${ieeeColors.gray}30; border-radius: 12px; padding: 20px; margin-bottom: 20px;">
    <h4 style="margin: 0 0 15px 0; color: ${ieeeColors.navy};">Serialization + ${selectedCompression.name} Combined Size</h4>

    <div style="display: grid; gap: 12px;">
      ${compressionSynergyData.map((item, i) => html`
        <div style="display: flex; align-items: center; gap: 15px;">
          <div style="width: 100px; font-weight: 500; color: ${item.format.color};">${item.format.name}</div>

          <!-- Before compression bar -->
          <div style="flex: 1; position: relative;">
            <div style="background: ${ieeeColors.lightGray}; border-radius: 4px; height: 24px; overflow: hidden;">
              <!-- Original size (faded) -->
              <div style="position: absolute; background: ${item.format.color}30; height: 24px; width: ${(item.originalSize / compressionSynergyData[compressionSynergyData.length - 1].originalSize) * 100}%; border-radius: 4px;"></div>
              <!-- Compressed size -->
              <div style="position: absolute; background: ${item.format.color}; height: 24px; width: ${(item.compressedSize / compressionSynergyData[compressionSynergyData.length - 1].originalSize) * 100}%; border-radius: 4px; display: flex; align-items: center; justify-content: flex-end; padding-right: 8px;">
                <span style="color: white; font-size: 11px; font-weight: bold; text-shadow: 0 1px 2px rgba(0,0,0,0.3);">${item.compressedSize}B</span>
              </div>
            </div>
          </div>

          <div style="width: 80px; text-align: right;">
            <div style="font-size: 12px; color: ${ieeeColors.green}; font-weight: bold;">-${item.effectiveReduction}%</div>
            <div style="font-size: 10px; color: ${ieeeColors.gray};">vs JSON raw</div>
          </div>
        </div>
      `)}
    </div>

    <div style="margin-top: 15px; padding-top: 15px; border-top: 1px solid ${ieeeColors.lightGray}; font-size: 12px; color: ${ieeeColors.gray};">
      <strong>Note:</strong> Faded bars show uncompressed size; solid bars show size after ${selectedCompression.name} compression.
      Text formats like JSON compress much better than already-binary formats.
    </div>
  </div>

  ${selectedCompression.id !== "none" ? html`
  <div style="background: ${ieeeColors.orange}10; border-left: 4px solid ${ieeeColors.orange}; padding: 15px; border-radius: 0 8px 8px 0;">
    <strong style="color: ${ieeeColors.orange};">Compression Trade-off:</strong>
    <span style="color: ${ieeeColors.darkGray};">
      ${selectedCompression.name} adds ~${selectedCompression.cpuCost}ms CPU overhead per KB.
      ${compressionSynergyData.filter(d => !d.worthIt).length > 0 ?
        `For already-compact formats like ${compressionSynergyData.filter(d => !d.worthIt).map(d => d.format.name).join(', ')}, additional compression may not be worthwhile.` :
        'All formats benefit from additional compression in this configuration.'}
    </span>
  </div>
  ` : ''}

</div>
`

(a)

(b)

(c)

(d)

(e)

(f)

Figure 1318.2: How serialization formats interact with compression algorithms

1318.6 Network Condition Simulator

Simulate how your serialization choice affects transmission under different network conditions including cellular, NB-IoT, LoRa, and satellite.

networkConditions = [
  { id: "wifi", name: "Wi-Fi (Good)", bandwidth: 10000, latency: 5, packetLoss: 0.01 },
  { id: "4g", name: "4G Cellular", bandwidth: 1000, latency: 50, packetLoss: 0.02 },
  { id: "nbiot", name: "NB-IoT", bandwidth: 30, latency: 1500, packetLoss: 0.05 },
  { id: "lora", name: "LoRa SF10", bandwidth: 1, latency: 2000, packetLoss: 0.1 },
  { id: "satellite", name: "Satellite", bandwidth: 10, latency: 600, packetLoss: 0.03 }
]

viewof selectedNetwork = Inputs.select(networkConditions, {
  label: "Network Condition",
  format: d => d.name,
  value: networkConditions[1]
})

networkSimulation = {
  return serializationResults.map(r => {
    const transmissionTime = (r.size * 8) / (selectedNetwork.bandwidth * 1000) * 1000; // ms
    const totalTime = transmissionTime + selectedNetwork.latency;
    const successRate = Math.pow(1 - selectedNetwork.packetLoss, Math.ceil(r.size / 100)); // Simplified

    // Retry estimate
    const expectedRetries = 1 / successRate;
    const effectiveTime = totalTime * expectedRetries;

    return {
      format: r.format,
      size: r.size,
      transmissionTime: transmissionTime.toFixed(1),
      totalTime: totalTime.toFixed(1),
      successRate: (successRate * 100).toFixed(1),
      effectiveTime: effectiveTime.toFixed(1),
      batteryImpact: (r.size * 0.1 + transmissionTime * 0.05).toFixed(2) // Simplified mAh
    };
  }).sort((a, b) => parseFloat(a.effectiveTime) - parseFloat(b.effectiveTime));
}

html`
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">

  <div style="background: ${ieeeColors.navy}; color: white; padding: 20px; border-radius: 12px 12px 0 0;">
    <div style="display: grid; grid-template-columns: repeat(4, 1fr); gap: 15px; text-align: center;">
      <div>
        <div style="font-size: 12px; opacity: 0.8;">Bandwidth</div>
        <div style="font-size: 20px; font-weight: bold;">${selectedNetwork.bandwidth} kbps</div>
      </div>
      <div>
        <div style="font-size: 12px; opacity: 0.8;">Base Latency</div>
        <div style="font-size: 20px; font-weight: bold;">${selectedNetwork.latency} ms</div>
      </div>
      <div>
        <div style="font-size: 12px; opacity: 0.8;">Packet Loss</div>
        <div style="font-size: 20px; font-weight: bold;">${(selectedNetwork.packetLoss * 100).toFixed(1)}%</div>
      </div>
      <div>
        <div style="font-size: 12px; opacity: 0.8;">Network Type</div>
        <div style="font-size: 20px; font-weight: bold;">${selectedNetwork.name}</div>
      </div>
    </div>
  </div>

  <div style="background: white; border: 1px solid ${ieeeColors.gray}30; border-radius: 0 0 12px 12px; overflow: hidden;">
    <!-- Header -->
    <div style="display: grid; grid-template-columns: 120px 1fr 100px 100px 100px; background: ${ieeeColors.lightGray}; padding: 10px; font-size: 12px; font-weight: bold; color: ${ieeeColors.navy};">
      <div>Format</div>
      <div>Effective Time (with retries)</div>
      <div style="text-align: center;">Success Rate</div>
      <div style="text-align: center;">TX Time</div>
      <div style="text-align: center;">Battery</div>
    </div>

    ${networkSimulation.map((item, i) => html`
      <div style="display: grid; grid-template-columns: 120px 1fr 100px 100px 100px; padding: 12px 10px; border-bottom: 1px solid ${ieeeColors.lightGray}; align-items: center; background: ${i === 0 ? ieeeColors.green + '10' : 'white'};">
        <div style="font-weight: 500; color: ${item.format.color};">${item.format.name}</div>
        <div style="padding-right: 20px;">
          <div style="background: ${ieeeColors.lightGray}; border-radius: 4px; height: 20px; overflow: hidden;">
            <div style="background: ${item.format.color}; height: 100%; width: ${Math.min(100, (parseFloat(item.effectiveTime) / Math.max(...networkSimulation.map(n => parseFloat(n.effectiveTime)))) * 100)}%; display: flex; align-items: center; padding-left: 8px;">
              <span style="color: white; font-size: 11px; font-weight: bold;">${item.effectiveTime} ms</span>
            </div>
          </div>
        </div>
        <div style="text-align: center; font-size: 13px; color: ${parseFloat(item.successRate) > 90 ? ieeeColors.green : parseFloat(item.successRate) > 70 ? ieeeColors.orange : ieeeColors.red}; font-weight: bold;">
          ${item.successRate}%
        </div>
        <div style="text-align: center; font-size: 13px; color: ${ieeeColors.darkGray};">
          ${item.transmissionTime} ms
        </div>
        <div style="text-align: center; font-size: 13px; color: ${ieeeColors.purple};">
          ${item.batteryImpact} mAh
        </div>
      </div>
    `)}
  </div>

  <div style="margin-top: 15px; display: grid; grid-template-columns: 1fr 1fr; gap: 15px;">
    <div style="background: ${ieeeColors.green}10; border-left: 4px solid ${ieeeColors.green}; padding: 12px; border-radius: 0 8px 8px 0;">
      <strong style="color: ${ieeeColors.green};">Best for ${selectedNetwork.name}:</strong>
      <span style="color: ${ieeeColors.darkGray};">${networkSimulation[0].format.name} with ${networkSimulation[0].effectiveTime}ms effective time</span>
    </div>
    <div style="background: ${ieeeColors.purple}10; border-left: 4px solid ${ieeeColors.purple}; padding: 12px; border-radius: 0 8px 8px 0;">
      <strong style="color: ${ieeeColors.purple};">Battery Savings:</strong>
      <span style="color: ${ieeeColors.darkGray};">${((parseFloat(networkSimulation[networkSimulation.length-1].batteryImpact) - parseFloat(networkSimulation[0].batteryImpact)) / parseFloat(networkSimulation[networkSimulation.length-1].batteryImpact) * 100).toFixed(0)}% reduction vs JSON</span>
    </div>
  </div>

</div>
`

(a)

(b)

(c)

(d)

Figure 1318.3: Simulate how serialization choice affects transmission under different network conditions

1318.7 Schema Evolution Comparison

Understanding how different formats handle schema changes over time is critical for long-lived IoT deployments.

schemaEvolutionData = [
  {
    format: "JSON",
    addField: { support: "Automatic", impact: "None", notes: "New fields ignored by old readers" },
    removeField: { support: "Automatic", impact: "Low", notes: "Old readers may fail on required fields" },
    renameField: { support: "None", impact: "Breaking", notes: "Treated as add+remove" },
    changeType: { support: "None", impact: "Breaking", notes: "Runtime errors possible" }
  },
  {
    format: "MessagePack",
    addField: { support: "Automatic", impact: "None", notes: "Same as JSON" },
    removeField: { support: "Automatic", impact: "Low", notes: "Same as JSON" },
    renameField: { support: "None", impact: "Breaking", notes: "Same as JSON" },
    changeType: { support: "None", impact: "Breaking", notes: "Binary type markers help detect" }
  },
  {
    format: "Protocol Buffers",
    addField: { support: "Native", impact: "None", notes: "New field numbers forward compatible" },
    removeField: { support: "Native", impact: "None", notes: "Reserved field numbers" },
    renameField: { support: "Native", impact: "None", notes: "Only number matters, not name" },
    changeType: { support: "Limited", impact: "Varies", notes: "Some safe promotions (int32 to int64)" }
  },
  {
    format: "CBOR",
    addField: { support: "Automatic", impact: "None", notes: "Like JSON but with tags" },
    removeField: { support: "Automatic", impact: "Low", notes: "Unknown keys skipped" },
    renameField: { support: "None", impact: "Breaking", notes: "Key-based like JSON" },
    changeType: { support: "Limited", impact: "Varies", notes: "Type tags can help" }
  },
  {
    format: "Apache Avro",
    addField: { support: "Native", impact: "None", notes: "Default values for new fields" },
    removeField: { support: "Native", impact: "None", notes: "Schema resolution handles" },
    renameField: { support: "Native", impact: "None", notes: "Aliases support renames" },
    changeType: { support: "Native", impact: "None", notes: "Schema resolution promotes types" }
  },
  {
    format: "Raw Binary",
    addField: { support: "None", impact: "Breaking", notes: "Fixed structure" },
    removeField: { support: "None", impact: "Breaking", notes: "All positions shift" },
    renameField: { support: "N/A", impact: "N/A", notes: "No field names" },
    changeType: { support: "None", impact: "Breaking", notes: "Size changes break parsing" }
  }
]

getImpactColor = (impact) => {
  switch(impact) {
    case "None": return ieeeColors.green;
    case "Low": return ieeeColors.teal;
    case "Varies": return ieeeColors.orange;
    case "Breaking": return ieeeColors.red;
    default: return ieeeColors.gray;
  }
}

html`
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">
  <h3 style="color: ${ieeeColors.navy}; margin-bottom: 15px;">Schema Evolution Compatibility</h3>

  <div style="overflow-x: auto;">
    <table style="width: 100%; border-collapse: collapse; font-size: 13px;">
      <thead>
        <tr style="background: ${ieeeColors.navy}; color: white;">
          <th style="padding: 12px; text-align: left;">Format</th>
          <th style="padding: 12px; text-align: center;">Add Field</th>
          <th style="padding: 12px; text-align: center;">Remove Field</th>
          <th style="padding: 12px; text-align: center;">Rename Field</th>
          <th style="padding: 12px; text-align: center;">Change Type</th>
        </tr>
      </thead>
      <tbody>
        ${schemaEvolutionData.map((row, i) => html`
          <tr style="background: ${i % 2 === 0 ? 'white' : ieeeColors.lightGray};">
            <td style="padding: 12px; font-weight: bold; color: ${formats.find(f => f.name === row.format)?.color || ieeeColors.navy};">${row.format}</td>
            <td style="padding: 12px; text-align: center;">
              <span style="background: ${getImpactColor(row.addField.impact)}20; color: ${getImpactColor(row.addField.impact)}; padding: 4px 8px; border-radius: 4px; font-size: 11px; display: inline-block;">
                ${row.addField.impact}
              </span>
            </td>
            <td style="padding: 12px; text-align: center;">
              <span style="background: ${getImpactColor(row.removeField.impact)}20; color: ${getImpactColor(row.removeField.impact)}; padding: 4px 8px; border-radius: 4px; font-size: 11px; display: inline-block;">
                ${row.removeField.impact}
              </span>
            </td>
            <td style="padding: 12px; text-align: center;">
              <span style="background: ${getImpactColor(row.renameField.impact)}20; color: ${getImpactColor(row.renameField.impact)}; padding: 4px 8px; border-radius: 4px; font-size: 11px; display: inline-block;">
                ${row.renameField.impact}
              </span>
            </td>
            <td style="padding: 12px; text-align: center;">
              <span style="background: ${getImpactColor(row.changeType.impact)}20; color: ${getImpactColor(row.changeType.impact)}; padding: 4px 8px; border-radius: 4px; font-size: 11px; display: inline-block;">
                ${row.changeType.impact}
              </span>
            </td>
          </tr>
        `)}
      </tbody>
    </table>
  </div>

  <div style="margin-top: 15px; display: grid; grid-template-columns: repeat(4, 1fr); gap: 10px; font-size: 11px;">
    <div style="display: flex; align-items: center; gap: 5px;">
      <span style="width: 12px; height: 12px; background: ${ieeeColors.green}; border-radius: 2px;"></span>
      <span>None - Safe change</span>
    </div>
    <div style="display: flex; align-items: center; gap: 5px;">
      <span style="width: 12px; height: 12px; background: ${ieeeColors.teal}; border-radius: 2px;"></span>
      <span>Low - Minor risk</span>
    </div>
    <div style="display: flex; align-items: center; gap: 5px;">
      <span style="width: 12px; height: 12px; background: ${ieeeColors.orange}; border-radius: 2px;"></span>
      <span>Varies - Case dependent</span>
    </div>
    <div style="display: flex; align-items: center; gap: 5px;">
      <span style="width: 12px; height: 12px; background: ${ieeeColors.red}; border-radius: 2px;"></span>
      <span>Breaking - Incompatible</span>
    </div>
  </div>

</div>
`

(a)

(b)

(c)

Figure 1318.4: How different formats handle schema changes over time

1318.8 Memory Footprint Analysis

For constrained embedded devices, the memory footprint of serialization libraries is often as important as the wire format efficiency.

memoryData = formats.map(f => {
  let librarySize, workingMemory, peakMemory;

  switch(f.id) {
    case "json":
      librarySize = 15; // KB
      workingMemory = 4;
      peakMemory = 8;
      break;
    case "msgpack":
      librarySize = 8;
      workingMemory = 2;
      peakMemory = 4;
      break;
    case "protobuf":
      librarySize = 25;
      workingMemory = 1;
      peakMemory = 3;
      break;
    case "cbor":
      librarySize = 6;
      workingMemory = 2;
      peakMemory = 4;
      break;
    case "avro":
      librarySize = 35;
      workingMemory = 5;
      peakMemory = 10;
      break;
    case "binary":
      librarySize = 0;
      workingMemory = 0.5;
      peakMemory = 1;
      break;
  }

  return {
    format: f,
    librarySize,
    workingMemory,
    peakMemory,
    total: librarySize + peakMemory
  };
}).sort((a, b) => a.total - b.total)

html`
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">
  <h3 style="color: ${ieeeColors.navy}; margin-bottom: 15px;">Memory Footprint (Embedded/MCU)</h3>

  <div style="display: grid; grid-template-columns: 1fr 1fr; gap: 20px;">

    <!-- Library Size -->
    <div style="background: white; border: 1px solid ${ieeeColors.gray}30; border-radius: 12px; padding: 20px;">
      <h4 style="margin: 0 0 15px 0; color: ${ieeeColors.navy};">Library Flash Size</h4>
      ${memoryData.map(item => html`
        <div style="display: flex; align-items: center; gap: 10px; margin-bottom: 8px;">
          <div style="width: 80px; font-size: 12px; color: ${item.format.color};">${item.format.name}</div>
          <div style="flex: 1; background: ${ieeeColors.lightGray}; border-radius: 4px; height: 16px; overflow: hidden;">
            <div style="background: ${item.format.color}; height: 100%; width: ${(item.librarySize / 40) * 100}%;"></div>
          </div>
          <div style="width: 50px; text-align: right; font-size: 11px;">${item.librarySize} KB</div>
        </div>
      `)}
    </div>

    <!-- Working Memory -->
    <div style="background: white; border: 1px solid ${ieeeColors.gray}30; border-radius: 12px; padding: 20px;">
      <h4 style="margin: 0 0 15px 0; color: ${ieeeColors.navy};">Peak RAM During Encode</h4>
      ${memoryData.map(item => html`
        <div style="display: flex; align-items: center; gap: 10px; margin-bottom: 8px;">
          <div style="width: 80px; font-size: 12px; color: ${item.format.color};">${item.format.name}</div>
          <div style="flex: 1; background: ${ieeeColors.lightGray}; border-radius: 4px; height: 16px; overflow: hidden;">
            <div style="background: ${item.format.color}; height: 100%; width: ${(item.peakMemory / 12) * 100}%;"></div>
          </div>
          <div style="width: 50px; text-align: right; font-size: 11px;">${item.peakMemory} KB</div>
        </div>
      `)}
    </div>

  </div>

  <div style="margin-top: 15px; background: ${ieeeColors.purple}10; border-left: 4px solid ${ieeeColors.purple}; padding: 15px; border-radius: 0 8px 8px 0;">
    <strong style="color: ${ieeeColors.purple};">Constrained Device Recommendation:</strong>
    <span style="color: ${ieeeColors.darkGray};">
      For devices with less than 32KB RAM, consider ${memoryData[0].format.name} or ${memoryData[1].format.name}.
      CBOR is specifically designed for constrained environments (IETF RFC 8949).
    </span>
  </div>

</div>
`

(a)

(b)

Figure 1318.5: Memory requirements for encoding and decoding each format

1318.9 Summary

NoteKey Takeaways

1. JSON is universal but inefficient - use for debugging and configuration
2. MessagePack offers 40% size reduction with minimal complexity - good default choice
3. Protocol Buffers provide best efficiency when schema management is acceptable
4. CBOR is the IoT-native standard for CoAP and constrained devices
5. Avro excels in data pipeline scenarios with schema evolution needs
6. Raw Binary should only be used when you control both endpoints and format is frozen

For most IoT deployments, starting with MessagePack or CBOR provides an excellent balance of efficiency, flexibility, and ecosystem support. Switch to Protocol Buffers when performance demands justify the schema management overhead.

TipQuick Decision Guide

If you need…	Choose
Maximum compatibility	JSON
Simple size reduction	MessagePack
Best performance	Protocol Buffers
CoAP integration	CBOR
Schema evolution	Avro
Absolute minimum size	Raw Binary

1318.10 What’s Next

Continue exploring serialization and related data topics:

• Serialization Format Basics - Interactive comparator and feature matrix
• IoT Scenario Analysis - Match formats to specific IoT use cases
• Time-Series Compression - Compression algorithms for IoT data
• Data Pipeline Builder - End-to-end data flow design

1318.11 Related Topics

• IoT Data Formats - Overview of data representation in IoT
• MQTT Protocol - Common IoT messaging protocol
• CoAP Protocol - RESTful protocol with CBOR support