Setup weights-and-biases using docker compose

setup_wandb.md

Sure! Here's the converted Docker Compose YAML file with a MySQL server as a separate container and a Docker volume for storage:

version: '3'
services:
  wandb-local:
    image: wandb/local
    container_name: wandb-local
    environment:
      - HOST=https://YOUR_DNS_NAME
      - LICENSE=XXXXX
      - BUCKET=s3://${ACCESS_KEY}:${SECRET_KEY}@${HOST}/${BUCKET_NAME}
      - BUCKET_QUEUE=internal://
      - AWS_REGION=us-east1
      - MYSQL=mysql://${USERNAME}:${PASSWORD}@db/${DATABASE}
    ports:
      - 8080:8080
    depends_on:
      - db
    restart: unless-stopped

  db:
    image: mysql:latest
    container_name: wandb-mysql
    environment:
      - MYSQL_DATABASE=${DATABASE}
      - MYSQL_USER=${USERNAME}
      - MYSQL_PASSWORD=${PASSWORD}
      - MYSQL_ROOT_PASSWORD=${ROOT_PASSWORD}
    volumes:
      - db_data:/var/lib/mysql
    restart: unless-stopped

volumes:
  db_data:

In this Docker Compose file:

• The wandb-local service is defined using the wandb/local image. It includes the necessary environment variables, such as HOST, LICENSE, BUCKET, BUCKET_QUEUE, AWS_REGION, and MYSQL. The MYSQL environment variable is updated to reference the db service for the MySQL connection.

• The db service is added to run a MySQL server using the mysql:latest image. It includes environment variables for the database name, username, password, and root password. These values should be provided through environment variables or a separate .env file.

• A named volume db_data is defined to persist the MySQL data. It is mounted to the /var/lib/mysql directory in the db container.

• The wandb-local service depends on the db service, ensuring that the MySQL server starts before the Weights and Biases local instance.

• Both services are configured to restart unless stopped explicitly.

To use this Docker Compose file:

1. Save the content to a file named docker-compose.yml.

2. Create a .env file in the same directory and provide the necessary environment variable values, such as ACCESS_KEY, SECRET_KEY, HOST, BUCKET_NAME, USERNAME, PASSWORD, DATABASE, and ROOT_PASSWORD.

3. Run the following command to start the containers:

   docker-compose up -d
   

This will start the Weights and Biases local instance and the MySQL server as separate containers, with data persistence using a Docker volume.