Last active
April 19, 2026 14:48
-
-
Save HasanMukit/acaf9d8f9f63bf2e75d2e1c2edd7fe4c to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Consistent Hashing — A Visual Guide</title> | |
| <link rel="preconnect" href="https://fonts.googleapis.com"> | |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> | |
| <link href="https://fonts.googleapis.com/css2?family=Fraunces:ital,opsz,wght@0,9..144,300;0,9..144,400;0,9..144,500;0,9..144,700;1,9..144,400&family=JetBrains+Mono:wght@400;500&family=Inter:wght@400;500;600&display=swap" rel="stylesheet"> | |
| <style> | |
| :root { | |
| --ink: #1a1814; | |
| --paper: #f7f3ec; | |
| --paper-warm: #efe9de; | |
| --rule: #c9bfae; | |
| --muted: #6b6558; | |
| --accent: #b8452e; | |
| --accent-soft: #e8d4ca; | |
| --s1: #2d5f8f; | |
| --s2: #3d7d5e; | |
| --s3: #b8792e; | |
| --s4: #9a3e6b; | |
| --s1-soft: #d4e1ed; | |
| --s2-soft: #d6e4dc; | |
| --s3-soft: #ead9c2; | |
| --s4-soft: #e6d0db; | |
| } | |
| @media (prefers-color-scheme: dark) { | |
| :root { | |
| --ink: #e8e2d6; | |
| --paper: #16140f; | |
| --paper-warm: #1e1b14; | |
| --rule: #3a3628; | |
| --muted: #928a78; | |
| --accent: #d97858; | |
| --accent-soft: #3d2a23; | |
| --s1: #6ba5d4; | |
| --s2: #78bfa0; | |
| --s3: #d9a560; | |
| --s4: #cc7ea8; | |
| --s1-soft: #1f3042; | |
| --s2-soft: #1f352a; | |
| --s3-soft: #3a2d1a; | |
| --s4-soft: #3a222e; | |
| } | |
| } | |
| * { margin: 0; padding: 0; box-sizing: border-box; } | |
| html { scroll-behavior: smooth; } | |
| body { | |
| background: var(--paper); | |
| color: var(--ink); | |
| font-family: 'Fraunces', Georgia, serif; | |
| font-weight: 400; | |
| line-height: 1.7; | |
| font-size: 18px; | |
| -webkit-font-smoothing: antialiased; | |
| -moz-osx-font-smoothing: grayscale; | |
| } | |
| .container { | |
| max-width: 720px; | |
| margin: 0 auto; | |
| padding: 80px 32px 120px; | |
| } | |
| header { | |
| border-bottom: 1px solid var(--rule); | |
| padding-bottom: 48px; | |
| margin-bottom: 56px; | |
| position: relative; | |
| } | |
| .eyebrow { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 11px; | |
| font-weight: 600; | |
| letter-spacing: 0.24em; | |
| text-transform: uppercase; | |
| color: var(--accent); | |
| margin-bottom: 24px; | |
| display: flex; | |
| align-items: center; | |
| gap: 12px; | |
| } | |
| .eyebrow::after { | |
| content: ''; | |
| flex: 1; | |
| height: 1px; | |
| background: var(--rule); | |
| max-width: 120px; | |
| } | |
| h1 { | |
| font-family: 'Fraunces', Georgia, serif; | |
| font-weight: 300; | |
| font-size: clamp(44px, 7vw, 72px); | |
| line-height: 1.0; | |
| letter-spacing: -0.03em; | |
| margin-bottom: 28px; | |
| font-variation-settings: "opsz" 144; | |
| } | |
| h1 em { | |
| font-style: italic; | |
| font-weight: 300; | |
| color: var(--accent); | |
| } | |
| .deck { | |
| font-family: 'Fraunces', Georgia, serif; | |
| font-size: 20px; | |
| line-height: 1.5; | |
| color: var(--muted); | |
| max-width: 560px; | |
| font-weight: 400; | |
| font-style: italic; | |
| } | |
| .meta { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 12px; | |
| color: var(--muted); | |
| margin-top: 32px; | |
| display: flex; | |
| gap: 24px; | |
| letter-spacing: 0.05em; | |
| } | |
| .meta span { | |
| display: flex; | |
| align-items: center; | |
| gap: 8px; | |
| } | |
| .meta span::before { | |
| content: ''; | |
| width: 4px; | |
| height: 4px; | |
| background: var(--accent); | |
| border-radius: 50%; | |
| } | |
| h2 { | |
| font-family: 'Fraunces', Georgia, serif; | |
| font-weight: 400; | |
| font-size: 32px; | |
| line-height: 1.2; | |
| letter-spacing: -0.02em; | |
| margin: 72px 0 24px; | |
| font-variation-settings: "opsz" 144; | |
| } | |
| h2 .num { | |
| font-family: 'JetBrains Mono', monospace; | |
| font-size: 13px; | |
| font-weight: 500; | |
| color: var(--accent); | |
| display: block; | |
| margin-bottom: 12px; | |
| letter-spacing: 0.1em; | |
| } | |
| h3 { | |
| font-family: 'Inter', sans-serif; | |
| font-weight: 600; | |
| font-size: 15px; | |
| letter-spacing: 0.02em; | |
| margin: 40px 0 16px; | |
| color: var(--ink); | |
| } | |
| p { | |
| margin-bottom: 20px; | |
| } | |
| p + p { | |
| text-indent: 0; | |
| } | |
| strong { | |
| font-weight: 500; | |
| color: var(--ink); | |
| } | |
| code { | |
| font-family: 'JetBrains Mono', monospace; | |
| font-size: 0.88em; | |
| background: var(--paper-warm); | |
| padding: 2px 6px; | |
| border-radius: 3px; | |
| color: var(--accent); | |
| font-weight: 500; | |
| } | |
| .lede::first-letter { | |
| font-family: 'Fraunces', serif; | |
| font-weight: 300; | |
| font-size: 88px; | |
| float: left; | |
| line-height: 0.85; | |
| margin: 8px 12px -4px 0; | |
| color: var(--accent); | |
| font-variation-settings: "opsz" 144; | |
| } | |
| .pullquote { | |
| margin: 48px 0; | |
| padding: 0 0 0 32px; | |
| border-left: 2px solid var(--accent); | |
| font-family: 'Fraunces', serif; | |
| font-style: italic; | |
| font-size: 24px; | |
| line-height: 1.4; | |
| color: var(--ink); | |
| font-weight: 400; | |
| } | |
| .figure { | |
| margin: 48px -40px; | |
| background: var(--paper-warm); | |
| border: 1px solid var(--rule); | |
| border-radius: 4px; | |
| padding: 32px 24px 24px; | |
| position: relative; | |
| } | |
| .figure-label { | |
| font-family: 'JetBrains Mono', monospace; | |
| font-size: 10px; | |
| font-weight: 500; | |
| letter-spacing: 0.15em; | |
| text-transform: uppercase; | |
| color: var(--accent); | |
| margin-bottom: 8px; | |
| } | |
| .figure-caption { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 13px; | |
| line-height: 1.5; | |
| color: var(--muted); | |
| margin-top: 20px; | |
| padding-top: 16px; | |
| border-top: 1px solid var(--rule); | |
| font-style: normal; | |
| } | |
| .figure svg { | |
| display: block; | |
| width: 100%; | |
| height: auto; | |
| } | |
| .callout { | |
| margin: 32px 0; | |
| padding: 20px 24px; | |
| background: var(--accent-soft); | |
| border-radius: 4px; | |
| font-family: 'Inter', sans-serif; | |
| font-size: 15px; | |
| line-height: 1.6; | |
| } | |
| .callout-label { | |
| font-family: 'Inter', sans-serif; | |
| font-weight: 600; | |
| font-size: 11px; | |
| letter-spacing: 0.1em; | |
| text-transform: uppercase; | |
| color: var(--accent); | |
| margin-bottom: 6px; | |
| } | |
| .two-col { | |
| display: grid; | |
| grid-template-columns: 1fr 1fr; | |
| gap: 32px; | |
| margin: 32px 0; | |
| } | |
| .card { | |
| padding: 20px; | |
| background: var(--paper-warm); | |
| border-radius: 4px; | |
| border: 1px solid var(--rule); | |
| } | |
| .card h4 { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 13px; | |
| font-weight: 600; | |
| margin-bottom: 8px; | |
| color: var(--accent); | |
| letter-spacing: 0.03em; | |
| } | |
| .card p { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 14px; | |
| line-height: 1.55; | |
| color: var(--muted); | |
| margin: 0; | |
| } | |
| .toc { | |
| margin: 48px 0; | |
| padding: 24px 28px; | |
| border: 1px solid var(--rule); | |
| border-radius: 4px; | |
| background: var(--paper-warm); | |
| } | |
| .toc-title { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 11px; | |
| font-weight: 600; | |
| letter-spacing: 0.18em; | |
| text-transform: uppercase; | |
| color: var(--muted); | |
| margin-bottom: 16px; | |
| } | |
| .toc ol { | |
| list-style: none; | |
| counter-reset: toc-counter; | |
| } | |
| .toc li { | |
| counter-increment: toc-counter; | |
| padding: 6px 0; | |
| font-family: 'Fraunces', serif; | |
| font-size: 17px; | |
| display: flex; | |
| align-items: baseline; | |
| gap: 16px; | |
| } | |
| .toc li::before { | |
| content: counter(toc-counter, decimal-leading-zero); | |
| font-family: 'JetBrains Mono', monospace; | |
| font-size: 11px; | |
| color: var(--accent); | |
| font-weight: 500; | |
| flex-shrink: 0; | |
| } | |
| .toc a { | |
| color: var(--ink); | |
| text-decoration: none; | |
| border-bottom: 1px solid transparent; | |
| transition: border-color 0.2s; | |
| } | |
| .toc a:hover { | |
| border-bottom-color: var(--accent); | |
| } | |
| .divider { | |
| display: flex; | |
| justify-content: center; | |
| margin: 64px 0; | |
| gap: 8px; | |
| } | |
| .divider span { | |
| width: 4px; | |
| height: 4px; | |
| background: var(--rule); | |
| border-radius: 50%; | |
| } | |
| .divider span:nth-child(2) { background: var(--accent); } | |
| footer { | |
| margin-top: 96px; | |
| padding-top: 32px; | |
| border-top: 1px solid var(--rule); | |
| font-family: 'Inter', sans-serif; | |
| font-size: 13px; | |
| color: var(--muted); | |
| text-align: center; | |
| line-height: 1.7; | |
| } | |
| .legend { | |
| display: flex; | |
| flex-wrap: wrap; | |
| gap: 16px 24px; | |
| justify-content: center; | |
| font-family: 'Inter', sans-serif; | |
| font-size: 12px; | |
| color: var(--muted); | |
| margin-top: 20px; | |
| } | |
| .legend-item { | |
| display: flex; | |
| align-items: center; | |
| gap: 8px; | |
| } | |
| .legend-dot { | |
| width: 12px; | |
| height: 12px; | |
| border-radius: 50%; | |
| } | |
| @media (max-width: 640px) { | |
| .container { | |
| padding: 48px 20px 80px; | |
| } | |
| .figure { | |
| margin: 32px -20px; | |
| padding: 24px 12px 16px; | |
| } | |
| .two-col { | |
| grid-template-columns: 1fr; | |
| gap: 16px; | |
| } | |
| .pullquote { | |
| font-size: 20px; | |
| margin: 32px 0; | |
| padding-left: 20px; | |
| } | |
| body { font-size: 17px; } | |
| } | |
| .svg-text { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 11px; | |
| fill: var(--ink); | |
| } | |
| .svg-text-small { | |
| font-family: 'JetBrains Mono', monospace; | |
| font-size: 9px; | |
| fill: var(--ink); | |
| } | |
| .svg-text-muted { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 10px; | |
| fill: var(--muted); | |
| } | |
| .svg-text-bold { | |
| font-family: 'Inter', sans-serif; | |
| font-size: 11px; | |
| font-weight: 600; | |
| fill: var(--ink); | |
| } | |
| .ring-line { | |
| fill: none; | |
| stroke: var(--rule); | |
| stroke-width: 1; | |
| stroke-dasharray: 3 3; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="container"> | |
| <header> | |
| <div class="eyebrow">Distributed systems · A primer</div> | |
| <h1>Consistent <em>Hashing</em></h1> | |
| <p class="deck">How distributed databases decide where your data lives — and why adding a server doesn't break everything.</p> | |
| <div class="meta"> | |
| <span>10 minute read</span> | |
| <span>Visual guide</span> | |
| <span>No prior experience</span> | |
| </div> | |
| </header> | |
| <div class="toc"> | |
| <div class="toc-title">What's inside</div> | |
| <ol> | |
| <li><a href="#problem">The problem with simple hashing</a></li> | |
| <li><a href="#ring">The hash ring</a></li> | |
| <li><a href="#magic">Adding a server</a></li> | |
| <li><a href="#vnodes">Virtual nodes, explained</a></li> | |
| <li><a href="#database">Where the data actually lives</a></li> | |
| <li><a href="#replication">Replication and failure</a></li> | |
| <li><a href="#crash">When a server dies</a></li> | |
| <li><a href="#takeaway">The one-line takeaway</a></li> | |
| </ol> | |
| </div> | |
| <p class="lede">Imagine a library with four shelves. You organize books by a simple rule: take each book's ID, divide by four, and the remainder tells you which shelf. Clean, simple, works perfectly — until you buy a fifth shelf. Now the rule becomes "divide by five," and suddenly almost every book needs to move.</p> | |
| <p>This is exactly what happens in distributed databases. Data is split across servers, and the naive way to decide where a record lives is <code>hash(key) mod N</code>, where N is the number of servers. Add one server and nearly all your data has to migrate. Lose one, same story. Millions of records shuffle around, caches go cold, users wait.</p> | |
| <p>Consistent hashing is a beautifully simple fix that has quietly powered most of the internet's data infrastructure since the early 2000s — Amazon's DynamoDB, Apache Cassandra, Discord's message routing, and every major CDN.</p> | |
| <h2 id="problem"><span class="num">01</span>The problem with simple hashing</h2> | |
| <p>With N servers and the <code>hash(key) mod N</code> approach, changing N changes the answer for nearly every key. Go from 4 servers to 5 and roughly 80% of your data relocates. That's unacceptable at scale.</p> | |
| <p>The goal of consistent hashing is simple: <strong>when servers come and go, only a small fraction of data should move.</strong> Ideally, just the slice that actually changed owners.</p> | |
| <h2 id="ring"><span class="num">02</span>The hash ring</h2> | |
| <p>Picture a giant clock face — but instead of 12 hours, it has billions of tiny positions around the edge. This is the hash ring.</p> | |
| <p>Each server gets placed on the ring based on a hash of its name. Each piece of data also gets placed on the ring based on a hash of its key. To find which server owns a given piece of data, you start at the data's position and walk clockwise until you bump into a server. That's the owner.</p> | |
| <div class="figure"> | |
| <div class="figure-label">Figure 01 — The basic ring</div> | |
| <svg viewBox="0 0 640 420" xmlns="http://www.w3.org/2000/svg"> | |
| <circle cx="280" cy="210" r="150" class="ring-line"/> | |
| <path d="M 280 60 A 160 160 0 0 1 380 100" fill="none" stroke="var(--muted)" stroke-width="0.8" opacity="0.5"/> | |
| <polygon points="378,96 386,102 378,108" fill="var(--muted)" opacity="0.5"/> | |
| <text x="345" y="82" class="svg-text-muted">clockwise</text> | |
| <circle cx="280" cy="60" r="20" fill="var(--s1)"/> | |
| <text x="280" y="65" text-anchor="middle" class="svg-text-bold" fill="white">A</text> | |
| <circle cx="410" cy="285" r="20" fill="var(--s2)"/> | |
| <text x="410" y="290" text-anchor="middle" class="svg-text-bold" fill="white">B</text> | |
| <circle cx="150" cy="285" r="20" fill="var(--s3)"/> | |
| <text x="150" y="290" text-anchor="middle" class="svg-text-bold" fill="white">C</text> | |
| <circle cx="205" cy="82" r="5" fill="var(--s1)"/> | |
| <text x="193" y="72" text-anchor="end" class="svg-text">key1</text> | |
| <circle cx="250" cy="63" r="5" fill="var(--s1)"/> | |
| <text x="242" y="50" text-anchor="end" class="svg-text">key2</text> | |
| <circle cx="380" cy="140" r="5" fill="var(--s2)"/> | |
| <text x="395" y="138" class="svg-text">key3</text> | |
| <circle cx="420" cy="222" r="5" fill="var(--s2)"/> | |
| <text x="435" y="226" class="svg-text">key4</text> | |
| <circle cx="280" cy="360" r="5" fill="var(--s3)"/> | |
| <text x="280" y="382" text-anchor="middle" class="svg-text">key5</text> | |
| <g transform="translate(500, 140)"> | |
| <text x="0" y="0" class="svg-text-bold">Owners</text> | |
| <circle cx="6" cy="22" r="5" fill="var(--s1)"/> | |
| <text x="18" y="26" class="svg-text">key1, key2 → A</text> | |
| <circle cx="6" cy="46" r="5" fill="var(--s2)"/> | |
| <text x="18" y="50" class="svg-text">key3, key4 → B</text> | |
| <circle cx="6" cy="70" r="5" fill="var(--s3)"/> | |
| <text x="18" y="74" class="svg-text">key5 → C</text> | |
| </g> | |
| </svg> | |
| <p class="figure-caption">Three servers (A, B, C) and five keys placed around the ring. Each key walks clockwise to find its owner. Key1 and key2 belong to server A; key3 and key4 belong to B; key5 belongs to C.</p> | |
| </div> | |
| <p>That's the entire routing logic. Simple. But the magic shows up when the cluster changes.</p> | |
| <h2 id="magic"><span class="num">03</span>Adding a server</h2> | |
| <p>Say server A is getting overloaded. You add a new server, <strong>D</strong>, between A and B on the ring.</p> | |
| <p>What moves? Only the keys that sit in the arc between A and D. Those keys used to walk clockwise past A to reach B, but now D is closer and catches them first. Every other key on the ring is unaffected — their clockwise walks still end at the same server they always did.</p> | |
| <div class="figure"> | |
| <div class="figure-label">Figure 02 — The zone of change</div> | |
| <svg viewBox="0 0 640 420" xmlns="http://www.w3.org/2000/svg"> | |
| <circle cx="280" cy="210" r="150" class="ring-line"/> | |
| <path d="M 280 60 A 150 150 0 0 1 410 135" fill="none" stroke="var(--accent)" stroke-width="8" opacity="0.25" stroke-linecap="round"/> | |
| <circle cx="280" cy="60" r="20" fill="var(--s1)"/> | |
| <text x="280" y="65" text-anchor="middle" class="svg-text-bold" fill="white">A</text> | |
| <circle cx="410" cy="135" r="22" fill="var(--accent)" stroke="var(--accent)" stroke-width="3"/> | |
| <text x="410" y="140" text-anchor="middle" class="svg-text-bold" fill="white">D</text> | |
| <text x="446" y="115" class="svg-text-bold" fill="var(--accent)">NEW</text> | |
| <circle cx="410" cy="285" r="20" fill="var(--s2)"/> | |
| <text x="410" y="290" text-anchor="middle" class="svg-text-bold" fill="white">B</text> | |
| <circle cx="150" cy="285" r="20" fill="var(--s3)"/> | |
| <text x="150" y="290" text-anchor="middle" class="svg-text-bold" fill="white">C</text> | |
| <circle cx="205" cy="82" r="5" fill="var(--s1)"/> | |
| <text x="193" y="72" text-anchor="end" class="svg-text">key1</text> | |
| <circle cx="250" cy="63" r="5" fill="var(--s1)"/> | |
| <text x="242" y="50" text-anchor="end" class="svg-text">key2</text> | |
| <circle cx="380" cy="140" r="6" fill="var(--accent)"/> | |
| <text x="370" y="172" class="svg-text-bold" fill="var(--accent)">key3 moved</text> | |
| <circle cx="420" cy="222" r="5" fill="var(--s2)"/> | |
| <text x="435" y="226" class="svg-text">key4</text> | |
| <circle cx="280" cy="360" r="5" fill="var(--s3)"/> | |
| <text x="280" y="382" text-anchor="middle" class="svg-text">key5</text> | |
| <g transform="translate(500, 140)"> | |
| <text x="0" y="0" class="svg-text-bold">After adding D</text> | |
| <circle cx="6" cy="22" r="5" fill="var(--s1)"/> | |
| <text x="18" y="26" class="svg-text">key1, key2 → A</text> | |
| <circle cx="6" cy="46" r="5" fill="var(--accent)"/> | |
| <text x="18" y="50" class="svg-text">key3 → D</text> | |
| <circle cx="6" cy="70" r="5" fill="var(--s2)"/> | |
| <text x="18" y="74" class="svg-text">key4 → B</text> | |
| <circle cx="6" cy="94" r="5" fill="var(--s3)"/> | |
| <text x="18" y="98" class="svg-text">key5 → C</text> | |
| </g> | |
| </svg> | |
| <p class="figure-caption">The orange arc shows the "zone of change" — only keys in that slice had to move. Here, just key3 relocates from B to D. Compare to <code>hash mod N</code>, which would shuffle nearly everything.</p> | |
| </div> | |
| <p>Removing a server works the same way in reverse. Delete a server from the ring, and its keys get absorbed by whichever server sits clockwise. Every other key stays exactly where it is.</p> | |
| <h2 id="vnodes"><span class="num">04</span>Virtual nodes, explained</h2> | |
| <p>There's a problem with this basic design. When servers hash to random positions, some end up owning huge slices of the ring while others get tiny slivers. One server might hold 50% of your data; another holds 5%. That creates hotspots.</p> | |
| <p>The fix: instead of each server planting <em>one flag</em> on the ring, it plants <strong>many small flags</strong> — 150 or 256 of them — at positions scattered all over. Each flag is called a <strong>virtual node</strong> or <strong>vnode</strong>.</p> | |
| <div class="callout"> | |
| <div class="callout-label">Important distinction</div> | |
| A vnode is not a storage container. It's just a marker on the ring — a claim that says "this slice belongs to server-1." The actual data lives in the normal storage system on the physical server. One physical server with 150 vnodes still has one disk; the 150 vnodes are just 150 addresses that all route to the same machine. | |
| </div> | |
| <div class="figure"> | |
| <div class="figure-label">Figure 03 — One flag vs. many flags</div> | |
| <svg viewBox="0 0 640 340" xmlns="http://www.w3.org/2000/svg"> | |
| <text x="160" y="30" text-anchor="middle" class="svg-text-bold">1 vnode per server</text> | |
| <text x="160" y="48" text-anchor="middle" class="svg-text-muted">Uneven territories</text> | |
| <circle cx="160" cy="180" r="100" class="ring-line"/> | |
| <path d="M 160 80 A 100 100 0 0 1 260 180 L 160 180 Z" fill="var(--s1-soft)" opacity="0.8"/> | |
| <path d="M 260 180 A 100 100 0 0 1 130 272 L 160 180 Z" fill="var(--s2-soft)" opacity="0.8"/> | |
| <path d="M 130 272 A 100 100 0 0 1 160 80 L 160 180 Z" fill="var(--s3-soft)" opacity="0.8"/> | |
| <circle cx="160" cy="80" r="10" fill="var(--s1)"/> | |
| <text x="160" y="83" text-anchor="middle" class="svg-text-bold" fill="white" font-size="9">A</text> | |
| <circle cx="260" cy="180" r="10" fill="var(--s2)"/> | |
| <text x="260" y="183" text-anchor="middle" class="svg-text-bold" fill="white" font-size="9">B</text> | |
| <circle cx="130" cy="272" r="10" fill="var(--s3)"/> | |
| <text x="130" y="275" text-anchor="middle" class="svg-text-bold" fill="white" font-size="9">C</text> | |
| <text x="480" y="30" text-anchor="middle" class="svg-text-bold">6 vnodes per server</text> | |
| <text x="480" y="48" text-anchor="middle" class="svg-text-muted">Balanced territories</text> | |
| <g> | |
| <path d="M 480 180 L 434.5 91.0 A 100 100 0 0 1 480.0 80.0 Z" fill="var(--s1-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 480.0 80.0 A 100 100 0 0 1 524.7 90.6 Z" fill="var(--s2-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 524.7 90.6 A 100 100 0 0 1 560.2 120.3 Z" fill="var(--s2-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 560.2 120.3 A 100 100 0 0 1 578.0 200.0 Z" fill="var(--s3-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 578.0 200.0 A 100 100 0 0 1 555.3 245.9 Z" fill="var(--s1-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 555.3 245.9 A 100 100 0 0 1 526.6 268.5 Z" fill="var(--s3-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 526.6 268.5 A 100 100 0 0 1 504.7 276.9 Z" fill="var(--s2-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 504.7 276.9 A 100 100 0 0 1 441.5 272.3 Z" fill="var(--s3-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 441.5 272.3 A 100 100 0 0 1 399.2 238.9 Z" fill="var(--s1-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 399.2 238.9 A 100 100 0 0 1 380.0 180.0 Z" fill="var(--s2-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 380.0 180.0 A 100 100 0 0 1 397.6 123.3 Z" fill="var(--s3-soft)" opacity="0.85"/> | |
| <path d="M 480 180 L 397.6 123.3 A 100 100 0 0 1 434.5 91.0 Z" fill="var(--s1-soft)" opacity="0.85"/> | |
| </g> | |
| <circle cx="480" cy="180" r="100" fill="none" stroke="var(--rule)" stroke-width="1"/> | |
| <g> | |
| <circle cx="480" cy="80" r="6" fill="var(--s1)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="524.7" cy="90.6" r="6" fill="var(--s2)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="560.2" cy="120.3" r="6" fill="var(--s2)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="578" cy="200" r="6" fill="var(--s3)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="555.3" cy="245.9" r="6" fill="var(--s1)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="526.6" cy="268.5" r="6" fill="var(--s3)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="504.7" cy="276.9" r="6" fill="var(--s2)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="441.5" cy="272.3" r="6" fill="var(--s3)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="399.2" cy="238.9" r="6" fill="var(--s1)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="380" cy="180" r="6" fill="var(--s2)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="397.6" cy="123.3" r="6" fill="var(--s3)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| <circle cx="434.5" cy="91" r="6" fill="var(--s1)" stroke="var(--paper-warm)" stroke-width="1.5"/> | |
| </g> | |
| <g transform="translate(380, 320)"> | |
| <circle cx="0" cy="0" r="5" fill="var(--s1)"/> | |
| <text x="10" y="4" class="svg-text">server A</text> | |
| <circle cx="80" cy="0" r="5" fill="var(--s2)"/> | |
| <text x="90" y="4" class="svg-text">server B</text> | |
| <circle cx="160" cy="0" r="5" fill="var(--s3)"/> | |
| <text x="170" y="4" class="svg-text">server C</text> | |
| </g> | |
| </svg> | |
| <p class="figure-caption">Left: with one vnode each, territories are wildly uneven. Right: with six vnodes each, the small scattered slices average out — each server ends up owning roughly a third of the ring. Real systems use 150–256 vnodes per server.</p> | |
| </div> | |
| <p>Vnodes serve two purposes at once:</p> | |
| <div class="two-col"> | |
| <div class="card"> | |
| <h4>Balanced load</h4> | |
| <p>Sum enough small random slices and the totals even out. Each server ends up with a roughly fair share, and hotspots disappear.</p> | |
| </div> | |
| <div class="card"> | |
| <h4>Graceful failure</h4> | |
| <p>When a server dies, its 150 small territories hand off to 150 different neighbors — spreading the load across the whole cluster instead of crushing one unlucky server.</p> | |
| </div> | |
| </div> | |
| <h2 id="database"><span class="num">05</span>Where the data actually lives</h2> | |
| <p>It's worth being precise about this because it's where intuition usually breaks down. There are two completely separate things in the database:</p> | |
| <p><strong>The ring</strong> lives in memory on every server. It's a small routing table — maybe a few thousand entries — that says "ring position X is owned by server Y." It's used only to answer the question "which server owns this key?"</p> | |
| <p><strong>The data</strong> lives on disk on individual servers. It's the actual key-value pairs like <code>alice → alice@example.com</code>. This can be billions of records.</p> | |
| <p>Vnodes exist only in the ring. They never hold data. When a key routes to one of server-1's 150 vnodes, the request lands on server-1's physical machine, and server-1 stores the value in its own local database — alongside every other key that server owns.</p> | |
| <p class="pullquote">Vnodes route. Servers store.</p> | |
| <h2 id="replication"><span class="num">06</span>Replication and failure</h2> | |
| <p>A key stored on only one server is a key you'll eventually lose. Real databases store each piece of data on multiple servers — typically three. This is called the <strong>replication factor</strong>.</p> | |
| <p>When a key is written, the database walks clockwise from the key's position and places copies on the next 3 <em>distinct physical servers</em>. "Distinct" matters: without that rule, three of server-1's vnodes might all be the closest flags, and you'd end up with three replicas all on the same machine — defeating the purpose.</p> | |
| <div class="figure"> | |
| <div class="figure-label">Figure 04 — Four servers, twelve vnodes, replication</div> | |
| <svg viewBox="0 0 640 460" xmlns="http://www.w3.org/2000/svg"> | |
| <circle cx="240" cy="230" r="160" class="ring-line"/> | |
| <circle cx="240" cy="70" r="13" fill="var(--s1)"/> | |
| <text x="240" y="73" text-anchor="middle" class="svg-text-small" fill="white">100</text> | |
| <text x="240" y="54" text-anchor="middle" class="svg-text-bold" fill="var(--s1)">S1</text> | |
| <circle cx="320" cy="97" r="13" fill="var(--s2)"/> | |
| <text x="320" y="100" text-anchor="middle" class="svg-text-small" fill="white">250</text> | |
| <text x="345" y="88" class="svg-text-bold" fill="var(--s2)">S2</text> | |
| <circle cx="380" cy="158" r="13" fill="var(--s1)"/> | |
| <text x="380" y="161" text-anchor="middle" class="svg-text-small" fill="white">400</text> | |
| <text x="405" y="158" class="svg-text-bold" fill="var(--s1)">S1</text> | |
| <circle cx="400" cy="230" r="13" fill="var(--s3)"/> | |
| <text x="400" y="233" text-anchor="middle" class="svg-text-small" fill="white">600</text> | |
| <text x="425" y="234" class="svg-text-bold" fill="var(--s3)">S3</text> | |
| <circle cx="380" cy="302" r="13" fill="var(--s2)"/> | |
| <text x="380" y="305" text-anchor="middle" class="svg-text-small" fill="white">800</text> | |
| <text x="405" y="308" class="svg-text-bold" fill="var(--s2)">S2</text> | |
| <circle cx="320" cy="363" r="13" fill="var(--s1)"/> | |
| <text x="320" y="366" text-anchor="middle" class="svg-text-small" fill="white">950</text> | |
| <text x="345" y="378" class="svg-text-bold" fill="var(--s1)">S1</text> | |
| <circle cx="240" cy="390" r="13" fill="var(--s3)"/> | |
| <text x="240" y="393" text-anchor="middle" class="svg-text-small" fill="white">1200</text> | |
| <text x="240" y="418" text-anchor="middle" class="svg-text-bold" fill="var(--s3)">S3</text> | |
| <circle cx="160" cy="363" r="13" fill="var(--s4)"/> | |
| <text x="160" y="366" text-anchor="middle" class="svg-text-small" fill="white">1500</text> | |
| <text x="135" y="378" text-anchor="end" class="svg-text-bold" fill="var(--s4)">S4</text> | |
| <circle cx="100" cy="302" r="13" fill="var(--s2)"/> | |
| <text x="100" y="305" text-anchor="middle" class="svg-text-small" fill="white">1700</text> | |
| <text x="75" y="308" text-anchor="end" class="svg-text-bold" fill="var(--s2)">S2</text> | |
| <circle cx="80" cy="230" r="13" fill="var(--s4)"/> | |
| <text x="80" y="233" text-anchor="middle" class="svg-text-small" fill="white">2000</text> | |
| <text x="55" y="234" text-anchor="end" class="svg-text-bold" fill="var(--s4)">S4</text> | |
| <circle cx="100" cy="158" r="13" fill="var(--s3)"/> | |
| <text x="100" y="161" text-anchor="middle" class="svg-text-small" fill="white">2300</text> | |
| <text x="75" y="158" text-anchor="end" class="svg-text-bold" fill="var(--s3)">S3</text> | |
| <circle cx="160" cy="97" r="13" fill="var(--s4)"/> | |
| <text x="160" y="100" text-anchor="middle" class="svg-text-small" fill="white">2800</text> | |
| <text x="135" y="88" text-anchor="end" class="svg-text-bold" fill="var(--s4)">S4</text> | |
| <circle cx="283" cy="80" r="5" fill="var(--ink)"/> | |
| <text x="283" y="38" text-anchor="middle" class="svg-text-bold">alice</text> | |
| <circle cx="394" cy="197" r="5" fill="var(--ink)"/> | |
| <text x="420" y="194" class="svg-text-bold">bob</text> | |
| <circle cx="205" cy="385" r="5" fill="var(--ink)"/> | |
| <text x="205" y="410" text-anchor="middle" class="svg-text-bold">carol</text> | |
| <circle cx="83" cy="190" r="5" fill="var(--ink)"/> | |
| <text x="58" y="188" text-anchor="end" class="svg-text-bold">dave</text> | |
| <g transform="translate(460, 80)"> | |
| <text x="0" y="0" class="svg-text-bold">Replication (RF=3)</text> | |
| <text x="0" y="16" class="svg-text-muted">Each key → 3 distinct servers</text> | |
| <text x="0" y="46" class="svg-text-bold">alice</text> | |
| <text x="0" y="62" class="svg-text">→ S2, S1, S3</text> | |
| <text x="0" y="88" class="svg-text-bold">bob</text> | |
| <text x="0" y="104" class="svg-text">→ S3, S2, S1</text> | |
| <text x="0" y="130" class="svg-text-bold">carol</text> | |
| <text x="0" y="146" class="svg-text">→ S4, S2, S3</text> | |
| <text x="0" y="172" class="svg-text-bold">dave</text> | |
| <text x="0" y="188" class="svg-text">→ S4, S3, S1</text> | |
| </g> | |
| </svg> | |
| <p class="figure-caption">Four servers, each with three vnodes. Keys walk clockwise and pick the next three distinct physical servers as replicas. Notice carol and dave: they walk past a repeat S4 vnode to find a different physical server for the third replica.</p> | |
| </div> | |
| <div class="legend"> | |
| <div class="legend-item"><span class="legend-dot" style="background: var(--s1)"></span>server-1</div> | |
| <div class="legend-item"><span class="legend-dot" style="background: var(--s2)"></span>server-2</div> | |
| <div class="legend-item"><span class="legend-dot" style="background: var(--s3)"></span>server-3</div> | |
| <div class="legend-item"><span class="legend-dot" style="background: var(--s4)"></span>server-4</div> | |
| </div> | |
| <h2 id="crash"><span class="num">07</span>When a server dies</h2> | |
| <p>Server-2 catches fire at 3 a.m. Its disk is gone. Here's what actually happens:</p> | |
| <h3>Within seconds</h3> | |
| <p>The other servers run a gossip protocol — constant heartbeat pings between peers. When server-2 stops responding, its peers mark it as down and spread the news. Within 10–30 seconds, every surviving server updates its ring view and removes server-2's vnodes.</p> | |
| <p>Now when alice logs in, her lookup walks clockwise from her position, skips past the (now-absent) server-2 vnodes, and lands on server-3's vnode instead. Server-3 already had her as a replica, so it just serves the read. Alice never noticed anything happened.</p> | |
| <h3>Over the next few hours</h3> | |
| <p>The cluster is now <em>under-replicated</em>. Keys that used to have 3 copies are down to 2. A background repair process slowly identifies every affected key and copies it to a new third replica somewhere, restoring the replication factor. This happens gradually to avoid overwhelming the cluster with a sudden flood of traffic.</p> | |
| <div class="callout"> | |
| <div class="callout-label">The beautiful part</div> | |
| The ring lookup logic doesn't change when a server dies. It's still "hash the key, walk clockwise, find the owner." The only difference is that one server's vnodes are no longer on the ring, so walks end at the next survivor — a server that, by design, already has a replica. | |
| </div> | |
| <p>Because server-2's 150 vnodes were scattered around the ring, its load doesn't dump onto a single unlucky neighbor. Each of its 150 small territories hands off to a different next-clockwise server. The dead server's workload distributes evenly across the entire cluster — no cascading failure.</p> | |
| <h2 id="takeaway"><span class="num">08</span>The one-line takeaway</h2> | |
| <p class="pullquote">A ring of servers where every key walks clockwise to find its home — and that simple rule means adding, removing, or losing servers only reshuffles a small slice of your data instead of all of it.</p> | |
| <p>Everything else — vnodes, replication, gossip, repair — is just making that simple rule work at scale, under failure, with balanced load. The elegance of the system is that the same routing logic works in normal operation and during disasters. No special failure paths. Just a circle, a clockwise walk, and data pre-positioned on the replicas that will inevitably become the new primaries.</p> | |
| <div class="divider"><span></span><span></span><span></span></div> | |
| <p>Consistent hashing is one of those rare ideas that looks almost trivial once you see it, yet quietly holds up enormous parts of the modern internet. The next time you load a video on Netflix, send a message on Discord, or query DynamoDB — somewhere underneath, a key is walking clockwise around a ring, finding its server.</p> | |
| <footer> | |
| A primer on consistent hashing<br> | |
| Designed for first-time readers · Based on the Dynamo paper (2007) and modern implementations in Cassandra, DynamoDB, and Riak | |
| </footer> | |
| </div> | |
| </body> | |
| </html> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment