AMQP Broker Storage Requirements
Try Set retention period to 90 days and change fsync interval from 200 ms.
Observe Watch Batched fsync Rate and Messages per Batch with message rate at 399 msgs/sec.
Explain Explain why 90 days retention affects Total Storage Needed.
Calculate
Reset
Technical boundaries. Storage is message rate multiplied by average message size and retention period. Batched fsync Rate is calculated from the fsync interval.
Interactive Calculator: AMQP Broker Storage Requirements
Estimate storage and I/O requirements for your AMQP deployment:
viewof msgRate = Inputs. range ([1 , 10000 ], {
value : 399 ,
step : 1 ,
label : "Message Rate (msgs/sec)"
})
viewof avgSize = Inputs. range ([10 , 10000 ], {
value : 150 ,
step : 10 ,
label : "Average Message Size (bytes)"
})
viewof retentionDays = Inputs. range ([1 , 365 ], {
value : 90 ,
step : 1 ,
label : "Retention Period (days)"
})
viewof fsyncInterval = Inputs. range ([50 , 1000 ], {
value : 200 ,
step : 50 ,
label : "fsync Interval (ms)"
})
dailyBytes = msgRate * avgSize * 86400
dailyGB = dailyBytes / (1000 * 1000 * 1000 )
// Calculate total storage
totalGB = dailyGB * retentionDays
// Calculate I/O metrics
batchesPerSecond = 1000 / fsyncInterval
messagesPerBatch = msgRate / batchesPerSecond
iopsReduction = msgRate / batchesPerSecond
// Format numbers
formatBytes = (bytes) => {
if (bytes >= 1e9 ) return ` ${ (bytes / 1e9 ). toFixed (2 )} GB`
if (bytes >= 1e6 ) return ` ${ (bytes / 1e6 ). toFixed (2 )} MB`
if (bytes >= 1e3 ) return ` ${ (bytes / 1e3 ). toFixed (2 )} KB`
return ` ${ bytes. toFixed (0 )} bytes`
}
// Create styled output
html `<div style="background: #f8f9fa; border-left: 4px solid #16A085; padding: 16px; margin: 16px 0; border-radius: 4px;">
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;">
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 16px; margin-bottom: 16px;">
<div style="background: white; padding: 12px; border-radius: 4px; border: 1px solid #e0e0e0;">
<div style="color: #7F8C8D; font-size: 12px; font-weight: 600; text-transform: uppercase; margin-bottom: 4px;">Daily Throughput</div>
<div style="color: #2C3E50; font-size: 24px; font-weight: 700;"> ${ formatBytes (dailyBytes)} </div>
<div style="color: #7F8C8D; font-size: 11px; margin-top: 4px;"> ${ dailyGB. toFixed (2 )} GB/day</div>
</div>
<div style="background: white; padding: 12px; border-radius: 4px; border: 1px solid #e0e0e0;">
<div style="color: #7F8C8D; font-size: 12px; font-weight: 600; text-transform: uppercase; margin-bottom: 4px;">Total Storage Needed</div>
<div style="color: #2C3E50; font-size: 24px; font-weight: 700;"> ${ totalGB. toFixed (0 )} GB</div>
<div style="color: #7F8C8D; font-size: 11px; margin-top: 4px;"> ${ retentionDays} days retention</div>
</div>
<div style="background: white; padding: 12px; border-radius: 4px; border: 1px solid #e0e0e0;">
<div style="color: #7F8C8D; font-size: 12px; font-weight: 600; text-transform: uppercase; margin-bottom: 4px;">Batched fsync Rate</div>
<div style="color: #2C3E50; font-size: 24px; font-weight: 700;"> ${ batchesPerSecond. toFixed (1 )} </div>
<div style="color: #7F8C8D; font-size: 11px; margin-top: 4px;">operations/sec</div>
</div>
<div style="background: white; padding: 12px; border-radius: 4px; border: 1px solid #e0e0e0;">
<div style="color: #7F8C8D; font-size: 12px; font-weight: 600; text-transform: uppercase; margin-bottom: 4px;">Messages per Batch</div>
<div style="color: #2C3E50; font-size: 24px; font-weight: 700;"> ${ messagesPerBatch. toFixed (0 )} </div>
<div style="color: #7F8C8D; font-size: 11px; margin-top: 4px;"> ${ iopsReduction. toFixed (0 )} x IOPS reduction</div>
</div>
</div>
<div style="background: linear-gradient(135deg, #16A085 0%, #2C3E50 100%); color: white; padding: 12px; border-radius: 4px; font-size: 13px;">
<strong>Sizing Insight:</strong> ${
totalGB < 100 ?
"Small deployment - standard SSD sufficient" :
totalGB < 500 ?
"Medium deployment - consider NVMe SSD for better I/O" :
totalGB < 2000 ?
"Large deployment - use high-performance storage with RAID" :
"Enterprise scale - consider distributed storage cluster"
}
</div>
</div>
</div>`
Try adjusting the parameters to see how message rate, size, and retention period affect storage requirements. Notice how batched fsync dramatically reduces disk I/O operations.
Back to the chapter