Code snippets — Load Balancing Techniques

snippet-2.conf

upstream backend {
    server 192.168.1.10:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;
}

snippet-2.txt

Client → L7 LB → Routes /api/* to API servers
                → Routes /static/* to CDN
                → Routes /ws/* to WebSocket servers

snippet-3.conf

# NGINX Plus or use Lua in open-source NGINX
upstream backend {
    zone backend 64k;
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;

    health_check interval=5s passes=2 fails=3 uri=/health;
}

snippet-3.txt

Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A (cycle repeats)

snippet-4.conf

http {
    upstream api_backend {
        least_conn;  # Algorithm

        server 10.0.0.10:8080 weight=2;  # Higher capacity
        server 10.0.0.11:8080 weight=1;
        server 10.0.0.12:8080 weight=1;

        keepalive 32;  # Reuse connections to backends
    }

    server {
        listen 443 ssl http2;
        server_name api.example.com;

        ssl_certificate /etc/ssl/cert.pem;
        ssl_certificate_key /etc/ssl/key.pem;

        location /api/ {
            proxy_pass http://api_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_connect_timeout 5s;
            proxy_send_timeout 30s;
            proxy_read_timeout 30s;

            # Retry on errors
            proxy_next_upstream error timeout http_500;
            proxy_next_upstream_tries 3;
        }
    }
}

snippet-4.txt

Server A (weight 3): handles 3x more traffic
Server B (weight 1): baseline

Sequence: A, A, A, B, A, A, A, B...

snippet-5.txt

Server A: 45 connections
Server B: 52 connections
Server C: 38 connections → next request goes here

snippet-6.txt

Server A: 38ms avg, 45 connections → score: 38 × 45 = 1,710
Server B: 22ms avg, 80 connections → score: 22 × 80 = 1,760
Server C: 45ms avg, 30 connections → score: 45 × 30 = 1,350 → winner

snippet-7.txt

hash(client_ip) % num_servers = server_index

snippet-8.txt

Hash ring with virtual nodes:
[ Server A ] [ Server B ] [ Server C ] [ Server A ] [ Server B ] ...
     ↑                                      ↑
  Request 1 (hash → here)             Request 2 (hash → here)

snippet-9.txt

api.example.com →
  US-East: 1.2.3.4 (primary)
  US-West: 5.6.7.8 (failover)
  EU: 9.10.11.12

snippet.conf

upstream backend {
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;

    sticky cookie srv_id expires=1h domain=.example.com path=/;
}

snippet.txt

Client → L4 LB (sees: TCP port 443) → Backend Server
         ↑ Doesn't look inside the packet