Time Series Database Write Path

Step through TSDB ingestion, WAL durability, MemTable buffering, flushes, and compaction pressure.

animation
time-series
database
data-management
iot
interactive
Interactive TSDB write-path lab showing IoT batches moving through validation, write-ahead logging, MemTable insert, acknowledgement, SSTable flush, compaction, throughput estimates, durability tradeoffs, and mobile-safe learning support.
TSDB write path WAL durability MemTable flush Compaction debt

Time Series Database Write Path

Follow an IoT batch from sensor arrival to durable acknowledgement and background storage cleanup. Change workload shape, batch size, WAL policy, tag cardinality, and MemTable size to see why fast ingestion still needs careful storage management.

0 points/sIncoming rate
0 msACK estimate
0 minFlush interval
LowCompaction debt
What MovesA batch token travels through validation, WAL, MemTable, ACK, SSTable flush, and compaction.
What ChangesThroughput, batch wait, WAL delay, memory pressure, SSTable count, and storage amplification update together.
What To CompareSmall batches lower wait time but increase write calls; large batches reduce overhead but can raise per-batch latency.
Common TrapA timestamp column alone is not a TSDB design. Series keys, tag cardinality, retention, and compaction policy decide whether it scales.

Batch arrives from IoT devices

A gateway groups timestamped readings so the TSDB can amortize protocol overhead and write the WAL sequentially.

Step 1 of 7 - Batch
TSDB write path diagram A batch token moves through validation, WAL append, MemTable insert, acknowledgement, SSTable flush, and compaction.

Formula Trace

Teaching estimates use the selected workload. They are not vendor benchmarks.


        


        

          

            

              
Pressure Readouts

              
These signals show why a write path can accept data and still build background work.

            

          

          

            

              
MemTable pressure0%

              

            

            

              
WAL bandwidth pressure0%

              

            

            

              
SSTable/read amplification risk0%

              

            

          

          

        

      


      

        

          

            
Storage Outcome

            
SSTables are immutable. Compaction trades background I/O for fewer overlapping files and better read behavior.

          

        

        

          
0 MB/sWAL write bandwidth

          
0 seriesEstimated active series

          
0/hourNew SSTables before compaction

          
0 MB/hEstimated physical storage growth

        

      

    


    
  


  

    

      

        Technical Accuracy Notes
        

          
    
            
  • A durable write path normally appends to a write-ahead log before acknowledging the client. Some systems acknowledge before an immediate disk sync for lower latency, but that changes the failure window.
    • 
            
  • The MemTable is an in-memory sorted or indexed buffer. It improves recent writes and reads, but it must eventually flush to immutable disk files or chunks.
    • 
            
  • SSTable is an LSM-tree term. Real TSDBs may call these files segments, shards, chunks, or blocks, but the idea here is immutable time-organized storage plus background cleanup.
    • 
            
  • Compaction does not make writes free. It shifts work into background I/O so reads scan fewer overlapping files and storage can reclaim duplicate, old, or expired records.
    • 
          

        

      


      

        Formulas And Units
        

          
    
            
  • points_per_sec = devices / sample_interval_s. This assumes one point per device per interval.
    • 
            
  • batches_per_sec = points_per_sec / batch_size. Large batches reduce write calls, but they add average batch-fill wait.
    • 
            
  • ack_latency_ms ~= batch_wait_ms + wal_sync_ms + memtable_insert_ms. Network, replication, and product internals are outside this teaching estimate.
    • 
            
  • flush_interval_s = memtable_limit_MB / memtable_growth_MBps. Small MemTables or high-cardinality metadata create more frequent flushes.
    • 
            
  • physical_MB_per_hour ~= points_per_sec x bytes_per_point x compression_ratio x storage_amp x 3600 / 1e6.
    • 
          

        

      


      

        Design Lessons
        

          
    
            
  • Batch writes when the application can tolerate the added wait. Dashboards often benefit; alarms may need smaller batches or a separate fast path.
    • 
            
  • Control tag cardinality. A unique request id, firmware hash, or free-form location label in the series key can create many series and large indexes.
    • 
            
  • Measure compaction debt during load tests. A system can show low ingest latency while falling behind on background storage work.
    • 
            
  • Separate observed time, receive time, and ingest time when late data or clock drift matters.
    • 
          

        

      


      

        Practice Prompts
        

          
    
            
  1. Select Factory burst and reduce the MemTable to 16 MB. Explain why the SSTable count rises.
    2. 
            
  2. Select Fleet labels drift and switch from high-cardinality labels to controlled tags. Compare active series and read amplification risk.
    3. 
            
  3. Select Delayed backfill and turn compaction off. Identify which metric warns that future range queries will suffer.
    4. 
            
  4. Change WAL sync from group commit to fsync each batch. Explain the durability and latency tradeoff.
    5. 
          

        

      

    


    

      

        

          
Related Resources

          
Use these after the write-path lab to connect ingestion with modeling, queries, and compression.

        

      

      

        Time-Series Databases
        Time-Series Fundamentals
        Time-Series Queries
        Compression Visualizer