← Back to the chapter · Previous detail

Technical detail · Big Data Technologies

Calculate the meter recovery window

The broker retains events for 72 hours. During a 9-hour consumer outage, readings continue to arrive at 24,000 events/s. After restart, the consumer’s normal catch-up rate is 30,000 events/s, so its net catch-up rate is 30,000 − 24,000 = 6,000 events/s.

The outage accumulates 24,000 events/s × 9 hours × 3,600 s/hour = 777,600,000 events. Clearing that backlog takes 777,600,000 / 6,000 events/s = 129,600 s, or 36 hours. Total time from outage start to full recovery is 9 hours of outage + 36 hours of catch-up = 45 hours.

The 72-hour broker retention is enough for this case, but only if the consumer really can sustain 30,000 events/s while new data continues to arrive. A slower catch-up rate or a second outage could exceed retention and force data loss or manual backfill from raw object storage.

← Back to the chapter · Previous detail