Run LiteLLM in Docker

DockerLLM.md

Title: Run LiteLLM in Docker with Localhost Exposure and API Key

Description: A guide to containerize litellm and expose it on localhost:4000 while using an API_KEY environment variable.

Content:

# Dockerfile for LiteLLM
# ----------------------
# Base Python image
FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install necessary tools and dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    && pip install --upgrade pip \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install litellm with proxy extras
RUN pip install 'litellm[proxy]'

# Expose port 4000 for the server
EXPOSE 4000

# Default environment variable for the model
ENV MODEL=github/gpt-4o

# Updated CMD to expand environment variable
CMD ["sh", "-c", "litellm --model github/gpt-4o"]

Steps to Build and Run

1. Build the Docker Image

   docker build -t litellm-server .
   

2. Run the Container

   docker run -d \
     --name litellm-container \
     -p 4000:4000 \
     -e API_KEY="${API_KEY}" \
     litellm-server --model github/gpt-4o
   

3. Create an ALias for Easy Access

   Add the following to your .bashrc or .zshrc file

   # Function to start the LiteLLM container with a specified model
   start-llm() {
     local model=${1:-gpt-4o}  # Default to 'gpt-4o' if no argument is provided
     docker run -d \
       --name litellm-container \
       -p 4000:4000 \
       -e MODEL="github/$model" \
       litellm-server
   }
   
   # Alias to stop and remove the LiteLLM container
   alias stop-llm='docker rm -f litellm-container'
   

   Reload your shell configuration

   source ~/.zshrc
   

4. Usage

   start-llm gpt-4o
   stop-llm
   

5. Access LiteLLM

http://localhost:4000