Skip to content

Instantly share code, notes, and snippets.

@psgganesh
Last active February 20, 2025 06:09
Show Gist options
  • Save psgganesh/09c37dc961678ed1eb7a45e84b137784 to your computer and use it in GitHub Desktop.
Save psgganesh/09c37dc961678ed1eb7a45e84b137784 to your computer and use it in GitHub Desktop.
OpenWebUI
# docker-compose.yaml
services:
litellm:
image: ghcr.io/berriai/litellm:main-latest
environment:
AWS_REGION_NAME: "ap-southeast-2"
AWS_PROFILE: "litellm"
entrypoint: "litellm"
command: ["--port", "1025", "--config", "/app/config.yaml", "--detailed_debug"]
volumes:
- ./litellm_config.yaml:/app/config.yaml:ro
- ~/.aws:/root/.aws:ro
restart: always
webui:
image: ghcr.io/open-webui/open-webui:main
ports:
- 1024:8080
environment:
WEBUI_AUTH: "false"
OPENAI_API_BASE_URL: "http://litellm:1025"
OPENAI_API_KEY: "anything"
volumes:
- open-webui:/app/backend/data
extra_hosts:
- host.docker.internal:host-gateway
restart: always
depends_on:
- litellm
volumes:
open-webui:
model_list:
- model_name: bedrock-claude-3-sonnet
litellm_params:
model: bedrock/anthropic.claude-3-sonnet-20240229-v1:0
- model_name: amazon-titan-text-express-v1
litellm_params:
model: bedrock/amazon.titan-text-express-v1
@psgganesh
Copy link
Author

  • Copy these files and save it with the file name
  • Keep both the files together in the same dir
  • Run docker compose up -d
  • Update ~/.aws profile and replace if needed on line no 7 of the docker-compose.yml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment