Migrate Coolify to a new server

migrate.sh

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com"

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@$destinationHost "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh $backupSourceDir 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup
  Do you want to stop Docker? (y/n)"
  read -r answer
  if [ "$answer" != "${answer#[Yy]}" ]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf $backupFileName -C / $backupSourceDir $HOME/.ssh/authorized_keys $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" root@$destinationHost "$remoteCommands" <$backupFileName; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [ "$answer" != "${answer#[Yy]}" ]; then
  if ! rm -f $backupFileName; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

im getting a msg saying i should be logged as azureuser

azureuser@Coolifyserver:~$ sudo ./CoolifyBackup.sh 
✅ Source directory exists 
✅ SSH key file exists 
✅ Docker is installed and running 
Please login as the user "azureuser" rather than the user "root".

❌ SSH connection to failed 
azureuser@Coolifyserver:~$

@rakithat20 it seems that the root user is disabled by default in Azure.[1] you may need to amend the script (i.e. ctrl+f, replace root with azureuser or another user) as well as grant that user passwordless sudo permissions[2]

Hey @AlejandroAkbal I modified your script for those who want to backup Coolify to a GCP Bucket using rclone

#!/bin/bash

# This script will backup your Coolify instance and upload the backup to a GCP bucket

# Configuration - Modify as needed
GCP_BUCKET="gcpbucketalias:bucket-name/backups/coolify/"

# -- Shouldn't need to modify anything below --
BACKUP_SOURCE_DIR="/data/coolify/"
BACKUP_FILE_NAME="coolify_backup_$(date +'%Y-%m-%d').tar.gz"

# Ensure the script is run as root
if [ "$EUID" -ne 0 ]; then
    printf "❌ Please run the script as root\n" >&2
    exit 1
fi

# Check if the source directory exists
if [ ! -d "$BACKUP_SOURCE_DIR" ]; then
    printf "❌ Source directory %s does not exist\n" "$BACKUP_SOURCE_DIR" >&2
    exit 1
fi
printf "✅ Source directory exists\n"

# Check if Docker is installed and running
if ! command -v docker >/dev/null 2>&1; then
    printf "❌ Docker is not installed\n" >&2
    exit 1
fi

if ! systemctl is-active --quiet docker; then
    printf "❌ Docker is not running\n" >&2
    exit 1
fi
printf "✅ Docker is installed and running\n"

# Get the names of all running Docker containers
container_names=$(docker ps --format '{{.Names}}')

# Initialize an array to hold the volume paths
volume_paths=()

# Loop over the container names and get their volumes
for container_name in $container_names; do
    volume_names=$(docker inspect --format '{{range .Mounts}}{{.Name}} {{end}}' "$container_name")
    for volume_name in $volume_names; do
        if [ -n "$volume_name" ]; then
            volume_paths+=("/var/lib/docker/volumes/$volume_name/_data")
        fi
    done
done

# Calculate and print the total size of the volumes and the source directory
total_size=$(du -csh "${volume_paths[@]}" 2>/dev/null | grep total | awk '{print $1}')
printf "✅ Total size of volumes to migrate: %s\n" "${total_size:-0}"

backup_source_dir_size=$(du -csh "$BACKUP_SOURCE_DIR" 2>/dev/null | grep total | awk '{print $1}')
printf "✅ Size of the source directory: %s\n" "${backup_source_dir_size:-0}"

# Create the backup tarball with progress feedback
printf "🚸 It's recommended to stop all Docker containers before creating the backup. Do you want to stop Docker? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    systemctl stop docker && systemctl stop docker.socket
    printf "✅ Docker stopped\n"
else
    printf "🚸 Docker not stopped, continuing with the backup\n"
fi

tar --exclude='*.sock' -Pczf "$BACKUP_FILE_NAME" -C / "$BACKUP_SOURCE_DIR" "$HOME/.ssh/authorized_keys" "${volume_paths[@]}" --checkpoint=.1000
if [ $? -ne 0 ]; then
    printf "❌ Backup file creation failed\n" >&2
    exit 1
fi
printf "✅ Backup file created\n"

# Transfer the backup file to GCP bucket
if ! rclone --gcs-bucket-policy-only copy "$BACKUP_FILE_NAME" "$GCP_BUCKET"; then
    printf "❌ Backup file transfer to GCP bucket failed\n" >&2
    exit 1
fi
printf "✅ Backup file transferred to GCP bucket\n"

# Clean up - Ask the user for confirmation before removing the local backup file
printf "Do you want to remove the local backup file? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! rm -f "$BACKUP_FILE_NAME"; then
        printf "❌ Failed to remove local backup file\n" >&2
        exit 1
    fi
    printf "✅ Local backup file removed\n"
else
    printf "🚸 Local backup file not removed\n"
fi

# Optionally start Docker again
printf "Do you want to start Docker again? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    systemctl start docker && systemctl start docker.socket
    printf "✅ Docker started\n"
else
    printf "🚸 Docker not started\n"
fi```

This seems to be the only way to migrate from one machine to another one. I was capable of backing up and restore it to the same coolify image. I guess, by now, if I'd updated coolify before restore the file.dmp, I wouldn't be able to back this system up again with this file.dmp. I have also tried to change the .env file so this could matchup the origial system. Yet 500 error occored.

Anyways, my only question is how can I test this migrate.sh ?

Awesome script, worked perfectly on first try!

TPGLLC-US

You didn't per chance write another that runs from a clean machine, checks the storage bucket for backups, and upon choosing one restores it?

glad everyone likes the script <3

@dreadedhamish you can probably ask chatgpt to do that no?

Here is the script modified by o1 to handle a ssh key passphrase using ssh-agent:

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com" # destination server IP or domain

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Function to initialize ssh-agent and add the SSH key
initialize_ssh_agent() {
  # Check if ssh-agent is already running
  if [ -z "$SSH_AGENT_PID" ] || ! ps -p "$SSH_AGENT_PID" > /dev/null 2>&1; then
    echo "🔄 Starting ssh-agent..."
    eval "$(ssh-agent -s)"
    if [ $? -ne 0 ]; then
      echo "❌ Failed to start ssh-agent"
      exit 1
    fi
    echo "✅ ssh-agent started"
  else
    echo "✅ ssh-agent is already running"
  fi

  # Add the SSH key to the agent
  echo "🔒 Adding SSH key to ssh-agent"
  ssh-add "$sshKeyPath"
  if [ $? -ne 0 ]; then
    echo "❌ Failed to add SSH key. Please ensure the passphrase is correct."
    exit 1
  fi
  echo "✅ SSH key added to ssh-agent"
}

# Initialize ssh-agent and add the SSH key
initialize_ssh_agent

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@"$destinationHost" "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh "$backupSourceDir" 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup"
  read -rp "Do you want to stop Docker? (y/n): " answer
  if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf "$backupFileName" -C / "$backupSourceDir" "$HOME/.ssh/authorized_keys" $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh root@"$destinationHost" "$remoteCommands" < "$backupFileName"; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
  if ! rm -f "$backupFileName"; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

# Kill ssh-agent if it was started by this script
if [ -n "$SSH_AGENT_PID" ]; then
  echo "🔒 Stopping ssh-agent..."
  eval "$(ssh-agent -k)"
  echo "✅ ssh-agent stopped"
fi

Thanks everyone for helping with this.

Thanks a lot , Great work

Thanks very much for everyone's input. Yesterday, I had to migrate a live self-hosted admin and spent the better part of a day trying different approaches. In the end, I found the variation provided by @AlejandroAkbal to be spot-on for my needs, and it worked on the first run.

does this script works for v3? i am using v3 coolify and i want to migrate it to new vps. i guess moving it to v4 isn't possible automatically. so does it work for v3 ?