Analytics & ML · Study deck
Acquisition Timing and Buffer Contracts
Imagine a vibration sensor that records a machine jolt during a network outage.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain why event time, receive time, and clock source must be recorded separately.
- Choose NTP, PTP, or hardware timestamping based on sensor timing tolerance.
- Design bounded buffers and backpressure policies before outages or bursts occur.
- Pass acquisition metadata downstream so analytics can distinguish real events from timing, drift, or queue artifacts.
Major section
Start With the Story
The reading has an event time, an arrival time, and a place in a waiting queue.
- A buffer is temporary storage for data that cannot be handled at once.
- A gateway is a local bridge from devices to another system.
- The system should mark lost or late data instead of quietly changing the story.
Major section
Analytics Inherits Acquisition
The mathematical gist.: A catalog-typical 500 Hz vibration band needs at least 1,000 raw samples per second before it becomes this chapter's 100 Hz summary stream.
- A 12-bit, 3.3 V ADC adds a separate 0.806 mV step and 0.233 mV RMS rounding noise.
Major section
Event Time, Buffers, and Drift
When both are present, a later analytics job can tell whether a point was genuinely late, delayed in transit, or replayed after an outage.
- For a slow temperature trend that may be tolerable.
- For a 100 Hz vibration stream, where samples are 10 ms apart, the same one-hour drift is 18 samples of timing error.
Major section
Event Time, Buffers, and Drift (continued)
Buffers need the same explicit design.
- A 64 kB buffer holds only about 13 seconds of that stream before it must block, drop, or aggregate.
- During a two-minute cellular outage, the raw backlog would be about 576 kB, far beyond that small buffer.
- A weak architecture just overwrites records silently.
Major section
Event Time, Buffers, and Drift (continued)
The contract passed downstream should include the sample rate, source timestamp, clock source, sequence number, gateway receive time, buffer policy, and quality flags.
- These fields are not bureaucratic metadata; they are how a dashboard distinguishes a real plant event from clock drift, link congestion, gateway reboot, or silent packet loss.
- Event time vs arrival time Network and buffering delays make the moment data arrives an unreliable clock.
- Clocks must be re-synced periodically to keep timestamps accurate over a long deployment.
- The acquisition contract Downstream stages assume samples are properly conditioned, sampled at a known rate, and accurately timestamped.
Major section
Event Time, Buffers, and Drift (continued)
Both systems may show a smooth dashboard later, but only the first one tells the analyst which evidence is complete, summarized, or missing.
- So the acquisition architecture is a contract, not a formality: condition the signal, sample at a known rate, timestamp at the source on a synchronised clock, and manage the buffer explicitly.
- Honour it and analytics can trust its inputs; break any clause — especially time — and the most sophisticated model downstream is computing on data that never meant what it claims.
- That distinction matters when a post-incident review asks whether the vibration spike was absent, averaged away, or lost during backpressure.
Deck summary
Key takeaways
The reading has an event time, an arrival time, and a place in a waiting queue.
- The mathematical gist.: A catalog-typical 500 Hz vibration band needs at least 1,000 raw samples per second before it becomes this chapter's 100 Hz summary stream.
- When both are present, a later analytics job can tell whether a point was genuinely late, delayed in transit, or replayed after an outage.
- Buffers need the same explicit design.
- The contract passed downstream should include the sample rate, source timestamp, clock source, sequence number, gateway receive time, buffer policy, and quality flags.
Retrieval practice
Recall check 1 of 3
Data Dora says: answer from memory, then check your reasoning.
Q1Why is the data acquisition pipeline foundational to all downstream analytics?
Show answer
Answer: A Downstream stages inherit acquisition errors; clean acquisition is the foundation.
Retrieval practice
Recall check 2 of 3
Data Dora says: answer from memory, then check your reasoning.
Q2Two sensors' clocks differ by tens of milliseconds. Why does this break multi-sensor fusion of a fast signal?
Show answer
Answer: C A common time base is required; clock skew misaligns samples and corrupts fusion.
Retrieval practice
Recall check 3 of 3
Data Dora says: answer from memory, then check your reasoning.
Q3Why should sensor data be timestamped at the source (event time) rather than when it arrives at the server?
Show answer
Answer: A Event time reflects when data was actually produced, unlike delay-distorted arrival time.
Print reference
Answers
Answer key.
- A · Downstream stages inherit acquisition errors; clean acquisition is the foundation.
- C · A common time base is required; clock skew misaligns samples and corrupts fusion.
- A · Event time reflects when data was actually produced, unlike delay-distorted arrival time.