setting-up-openwebui.md

Here's a docker-compose.yml file that will set up Open WebUI with GPU support:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:cuda
    container_name: open-webui
    network_mode: "host"
    volumes:
      - open-webui:/app/backend/data
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped

volumes:
  open-webui:

To use this with your NVIDIA GPU, you'll need:

NVIDIA Container Toolkit: This is essential for Docker to access your GPU.
Docker Compose with GPU support: Make sure you're using a Docker Compose version that supports the GPU deployment options.

Here's how to set up the prerequisites:

Install NVIDIA Driver (if not already installed): Your host system needs the appropriate NVIDIA drivers installed.

Install NVIDIA Container Toolkit:

# Add the NVIDIA repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update and install
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker to use the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

After setting up the prerequisites and creating the docker-compose.yml file:

Start the container with:
```
docker-compose up -d
```
Access Open WebUI at:
```
http://localhost:3000
```

Important notes:

The container uses the :cuda tag which is specifically built for GPU support
The volume mapping ensures your data persists between restarts
The NVIDIA_VISIBLE_DEVICES environment variable exposes all GPUs to the container
The deploy section configures Docker to use the NVIDIA runtime

If you want to connect this to an Ollama instance running elsewhere, add the OLLAMA_BASE_URL environment variable to the docker-compose file.

aaronsb/setting-up-openwebui.md