← Back to the chapter · Previous detail
Technical detail · CAP Theorem and Database Categories
Move virtual buckets and contain hot keys
Naive routing can make growth dangerous. If ownership is based only on a direct modulo of the current node count, adding or removing nodes may move many keys at once. More controlled approaches use stable ownership maps, token ranges, virtual buckets, explicit routing tables, or managed service mechanisms that move a bounded portion of the key space with verification.
Worked example: a telemetry cluster maps 1,024 virtual buckets onto 8 physical shards, so each shard owns about 128 buckets. If a ninth shard is added, the team can move 64 buckets first, which is 64 ÷ 1,024 = 6.25% of the key space, watch copy lag and read correctness, then continue in bounded waves. The release drill should record which buckets moved, how many device series were copied, how many writes arrived during dual-write or cutover, and what rollback condition would stop the move.
Hot keys need the same evidence. If one tenant sends 18,000 writes per minute while the median tenant sends 600, a tenant-only shard key can overload one owner even though the global average looks safe. The mitigation might be tenant-plus-device sub-buckets, quota enforcement, precomputed summaries, or moving that tenant to a larger shard group. Under the hood, the router must keep those exceptions deterministic so retries, backfills, and restore jobs use the same ownership map.