Example setup of django-redis with sentinels

django-redis-sentinels.md

Setting Up Django-Redis with Sentinel

This is an example of configuration for Django to use Redis with Sentinel for high availability and failover that worked for me, adjust it to your needs.

---

Interaction Diagram

Below is my understanding of how Django interacts with Sentinel and Redis:

flowchart LR
    django-redis --> | 1 - queries | Sentinels
    Sentinels --> | 2 - gives master's IP to | django-redis
    django-redis --> | 3 - reads/writes to | Master
    Master -.-> | replicates to | Replica
    Sentinels -.-> | monitor | Master
    Sentinels -.-> | monitor | Replica
    Sentinel1 <-.-> | monitor | Sentinel2
    Sentinel2 <-.-> | monitor | Sentinel3

subgraph Sentinels
    direction TB
    Sentinel1
    Sentinel2
    Sentinel3
end

subgraph Redis
    direction LR
    Master
    Replica
end

subgraph Django
    django-redis
end

Loading

Explanation

1. Django Application:

   • Sends cache requests to Redis Sentinel.
   • Uses Sentinel to determine the current primary Redis instance.

2. Redis Sentinel:

   • Monitors the health of the Redis primary and replicas.
   • Handles failover by promoting a replica to primary if the current primary fails.

3. Redis Primary:

   • Handles write operations and replicates data to the replicas.

4. Redis Replicas:

   • Serve as read-only copies of the primary.
   • Can be promoted to primary during failover.

---

Prerequisites

Ensure Redis Sentinel is installed and configured on your servers. Sentinel should monitor your Redis instances and handle failover.

Configured ACLs for default user: Ensure your Redis Sentinel is configured with proper ACLs for authentication.

Example Redis server ACL:

# huey -> master
user admin allchannels +@all ~* on >{{ redis_password }}

# replica -> master
user replica +psync +replconf +ping on >{{ redis_password }}

# sentinels -> master
user sentinel allchannels +multi +slaveof +ping +exec +subscribe +config|rewrite +role +publish +info +client|setname +client|kill +script|kill on >{{ redis_password }}

# django-redis -> master
user default allchannels +@read +@write +@connection +@pubsub +@scripting ~* on >{{ redis_password }}

Example Redis Sentinel ACL:

# sentinels -> sentinels
user admin allchannels +@all on >{{ redis_password }}

# django-redis -> sentinels
user default -@all +auth +client|getname +client|id +client|setname +command +hello +ping +role +sentinel|get-master-addr-by-name +sentinel|master +sentinel|myid +sentinel|replicas +sentinel|sentinels +sentinel|masters on >{{ redis_password }}

---

Django Configuration

Update your Django settings to use django-redis with Sentinel.

Example Configuration

from django.conf import settings

# Sentinel connection details
SENTINELS = [
    ("<Sentinel1_IP>", 26379),
    ("<Sentinel2_IP>", 26379),
    ("<Sentinel3_IP>", 26379),
]

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://<name_of_master>/<db>",  # As known by Sentinel
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.SentinelClient",
            "CONNECTION_POOL_CLASS": "redis.sentinel.SentinelConnectionPool",
            "USERNAME": "default",  # Django to MASTER ACL username (I think it needs to be "default")
            "PASSWORD": <redis_password>,  # Django to MASTER ACL password
            "SENTINELS": SENTINELS,
            "SENTINEL_KWARGS": {
                # DO NOT PUT "username" in this dict! It is not supported yet and will be "default".
                # https://github.com/jazzband/django-redis/issues/656
                "password": <redis_password>,  # Django to SENTINELS ACL password for user "default".
            },
        },
    },
    "compressor": {
        "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
    },
}

---

Sentinel Configuration

Ensure your Sentinel configuration includes the following:

sentinel sentinel-user admin # sentinel -> sentinel
sentinel sentinel-pass <redis_password> # sentinel -> sentinel
sentinel monitor primary <redis_master_ip> 6379 3
sentinel auth-user primary sentinel # sentinel -> master
sentinel auth-pass primary <redis_password> # sentinel -> master
sentinel down-after-milliseconds primary 5000
sentinel failover-timeout primary 60000
sentinel parallel-syncs primary 1

---

Testing the Setup

1. Verify Sentinel: Run the following commands to check the status of your Sentinel setup:

   redis-cli -h <sentinel_ip> -p 26379 --user admin -a <redis_password> SENTINEL masters
   redis-cli -h <sentinel_ip> -p 26379 --user admin -a <redis_password> SENTINEL replicas primary

2. Test Failover: Simulate a failover by stopping the primary Redis instance:

   redis-cli -h <sentinel_ip> -p 26379 --user admin -a <redis_password> SENTINEL FAILOVER primary

3. Check Django: Ensure your Django application continues to function with minimal disruptions during the failover, and no errors after the failover.