How to configure a Keycloak cluster properly (Quarkus edition)

_keycloak-cluster-config.md

Keycloak Cluster Configuration (How to)

This is a short and simple example on how to build a proper Keycloak cluster, using DNS_PING as discovery protocol and an NGINX server as reverse proxy.

If you prefer to use JDBC_PING, see @xgp's example gist here: https://gist.github.com/xgp/768eea11f92806b9c83f95902f7f8f80

---

Please see also my video about Keycloak Clustering: http://www.youtube.com/watch?v=P96VQkBBNxU
NOTE: The video covers JDBC_PING protocol and uses the legacy Keycloak Wildfly distribution!

docker-compose-cluster.yml

version: '3.8'

services:
  postgres:
    image: postgres:latest
    environment:
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: passw0rd
    volumes:
      - pg-data:/var/lib/postgresql/data

  keycloak:
    image: quay.io/keycloak/keycloak:latest
    command: start-dev -Djgroups.dns.query=keycloak
    environment:
      KC_CACHE: ispn
      KC_CACHE_STACK: kubernetes
      KC_DB: postgres
      KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak
      KC_DB_USERNAME: keycloak
      KC_DB_PASSWORD: passw0rd
      KC_PROXY: edge
      KC_HOSTNAME: localhost
      KC_HOSTNAME_PORT: '8000'
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
    deploy:
      replicas: 2
      endpoint_mode: dnsrr

  lb:
    image: nginx:alpine
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - "8000:8000"

volumes:
  pg-data:
    name: keycloak-demo-cluster-data

nginx.con

upstream backend {
    ip_hash;
    server keycloak-1:8080 fail_timeout=2s;
    server keycloak-2:8080 fail_timeout=2s;
}

server {
    listen       8000;
    server_name  localhost;
    access_log   off;

    location / {
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_set_header    X-Forwarded-Host   $host;
        proxy_set_header    X-Forwarded-Port   $server_port;
        
        proxy_pass              http://backend;
        proxy_connect_timeout   2s;

        proxy_buffer_size          128k;
        proxy_buffers              4 256k;
        proxy_busy_buffers_size    256k;
    }
}

I use this in docker compose maybe can help you in the right direction

mysql: image: mysql:5.7 volumes: - ./mysql:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: root MYSQL_DATABASE: keycloak MYSQL_USER: keycloak MYSQL_PASSWORD: password keycloak: image: quay.io/keycloak/keycloak environment: DB_VENDOR: MYSQL DB_ADDR: mysql DB_DATABASE: keycloak DB_USER: keycloak DB_PASSWORD: password KEYCLOAK_ADMIN: admin KEYCLOAK_ADMIN_PASSWORD: Pa55w0rd ports: - 8080:8080 - 9990:9990 depends_on: - mysql command: start-dev

Thanks, I've already make it work on docker with a JDBC cluster.
I was wondering if there is any workaround to use the KC_CACHE_STACK: kubernetes on docker swarm, they don't discover each other, maybe a naming, I can't specify static IP because the replicas can spawn on different nodes, so there is the catch and the KC_CACHE_STACK: kubernetes is not finding the other keycloak replicas :(

Hi,
You can find what you want here:
Work for keycloak 20.0.3
keycloak/keycloak#10210
The main problem for docker swarm is node discovery with multiple ip interface container. Also a problem with healthcheck container but it's possible but harder than kubernetes :)

Hello,
When using Jboss based Keycloak versions, it was possible to query the cluster size using jboss-cli.sh.
Does anyone know if it is possible to query (just to confirm) the cluster size in Quarkus based Keycloak ?

Not by default. You'll have to provide your custom Infinispan XML config file with proper values - or, with placeholders to be replaced by environment variable values during runtime.

@dasniko thanks for reply, but I am not sure if I got it right. Was your answer about the cache configuration and how we can set number of owners for a cache ?

My original question was more about how can we query the total number of Keycloak nodes present in the cluster at runtime. Any clues about that ?

P.S.
With Jboss I used to do it like this: /opt/jboss/keycloak/bin/jboss-cli.sh -c --command="/subsystem=jgroups/channel=ee:read-attribute(name=view)"

@zhandospdf Ok, then I misunderstood your question.
AFAIK there's no ootb way to do this. Perhaps some custom java code could help here, but I don't have any details.

Hello. I am new to this. I tried this demo following your steps but I got a problem: nginx encountered this error and exited
nginx: [emerg] host not found in upstream "keycloak-1:8080" in /etc/nginx/conf.d/default.conf:2
Can you help me with this?

Replace keycloak-1 (and keycloak-2) with the hostname of the Keycloak containers, as they are reachable from within the Docker network. Usually (default) this is <name-of-the-folder>-<servicename>-<running-number-of-replica>

I recently upgraded from Keycloak 18 to version 21.1.0. In my previous setup, Keycloak was running with cache stack as kubernates across multiple pods. However, after the upgrade, I encountered an issue where the jgroups were unable to locate multiple nodes

Hello , anybody know how to share the cache between hosts that runs the same docker-compose like in this example?
i have this config:
Server 1 & 2 : running docker-compose
ExternalDatabse
loadBalancer: switch between the two servers

the cache is shared only on the instances running on the same server, so if you have a solution for this it will be so helpful for me.

Hi @dasniko
The name of the nging conf file should be nginx.conf not nginx.con.

On my Windows computer I created folder mykeycloak1 and put this code inside this folder.
Files docker-compose-cluster.yml and nginx.conf.
After that I run:
docker compose -f docker-compose-cluster.yml up -d
also I modified nginx.conf file:

upstream backend {
    ip_hash;
#    server keycloak-1:8080 fail_timeout=2s;
#    server keycloak-2:8080 fail_timeout=2s;
    server mykeycloak1-keycloak-1:8080 fail_timeout=5s;
    server mykeycloak1-keycloak-2:8080 fail_timeout=5s;
}

server {
    listen       8000;
    server_name  localhost;
    access_log   off;

    location / {
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_set_header    X-Forwarded-Host   $host;
        proxy_set_header    X-Forwarded-Port   $server_port;
        
        proxy_pass              http://backend;
        proxy_connect_timeout   2s;

        proxy_buffer_size          128k;
        proxy_buffers              4 256k;
        proxy_busy_buffers_size    256k;
    }
}

All containers running successfully.

When I try to access URL http://localhost:8000 it redirect my to URL https://localhost/admin/ with the message "Unable to connect"

Current latest version of keycloak (quay.io/keycloak/keycloak:latest) is 26.0.7.
Is this an issue with keycloak version 26.0.7 or this is an issue with nginx configuration?

Thanks in advance,
VladanO