Migrate Coolify to a new server

migrate.sh

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com"

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@$destinationHost "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh $backupSourceDir 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup
  Do you want to stop Docker? (y/n)"
  read -r answer
  if [ "$answer" != "${answer#[Yy]}" ]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf $backupFileName -C / $backupSourceDir $HOME/.ssh/authorized_keys $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" root@$destinationHost "$remoteCommands" <$backupFileName; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [ "$answer" != "${answer#[Yy]}" ]; then
  if ! rm -f $backupFileName; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

Thanks a lot for the script

I tried something very similar, but must have missed something, as I spent an entire day trying to figure out some errors in coolify. Thank you sincerely for posting this. As far as I can tell, everything works exactly as it did on my original VPS and it took literally seconds to do.

Here is an updated version with support for multiple volumes and some improvements

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server,
# including Docker volumes, Coolify database, and SSH keys.

# Configuration - Modify as needed
sshKeyPath="/home/user/.ssh/key" # Key to the destination server
destinationHost="192.168.1.1"
sshPort=22 # SSH port for the destination server

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Ensure the script is run as root
if [ "$EUID" -ne 0 ]; then
    echo "❌ Please run the script as root"
    exit 1
fi

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
    echo "❌ Source directory $backupSourceDir does not exist"
    exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
    echo "❌ SSH key file $sshKeyPath does not exist"
    exit 1
fi
echo "✅ SSH key file exists"

# Check if Docker is installed and running
if ! command -v docker >/dev/null 2>&1; then
    echo "❌ Docker is not installed"
    exit 1
fi

if ! systemctl is-active --quiet docker; then
    echo "❌ Docker is not running"
    exit 1
fi
echo "✅ Docker is installed and running"

# Check if we can SSH to the destination server
if ! ssh -p "$sshPort" -i "$sshKeyPath" -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@"$destinationHost" "exit"; then
    echo "❌ SSH connection to $destinationHost failed"
    exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an array to hold the volume paths
volumePaths=()

# Loop over the container names and get their volumes
for containerName in $containerNames; do
    volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}} {{end}}' "$containerName")
    for volumeName in $volumeNames; do
        if [ -n "$volumeName" ]; then
            volumePaths+=("/var/lib/docker/volumes/$volumeName/_data")
        fi
    done
done

# Calculate and print the total size of the volumes and the source directory
totalSize=$(du -csh "${volumePaths[@]}" 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Total size of volumes to migrate: ${totalSize:-0}"

backupSourceDirSize=$(du -csh "$backupSourceDir" 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: ${backupSourceDirSize:-0}"

# Check if the backup file already exists and create it if it does not
if [ ! -f "$backupFileName" ]; then
    echo "🚸 Backup file does not exist, creating..."

    # Optionally stop Docker before creating the backup
    echo "🚸 It's recommended to stop all Docker containers before creating the backup. Do you want to stop Docker? (y/n)"
    read -rp "Answer: " answer
    if [[ "$answer" =~ ^[Yy]$ ]]; then
        systemctl stop docker && systemctl stop docker.socket
        echo "✅ Docker stopped"
    else
        echo "🚸 Docker not stopped, continuing with the backup"
    fi

    # Create the backup tarball with progress feedback
    tar --exclude='*.sock' -Pczf "$backupFileName" -C / "$backupSourceDir" "$HOME/.ssh/authorized_keys" "${volumePaths[@]}" --checkpoint=.1000
    if [ $? -ne 0 ]; then
        echo "❌ Backup file creation failed"
        exit 1
    fi
    echo "✅ Backup file created"
else
    echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
    if systemctl is-active --quiet docker; then
        if ! systemctl stop docker; then
            echo '❌ Docker stop failed';
            exit 1;
        fi
        echo '✅ Docker stopped';
    else
        echo 'ℹ️ Docker is not a service, skipping stop command';
    fi

    cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;
    if ! tar -Pxzf - -C /; then
        echo '❌ Backup file extraction failed';
        exit 1;
    fi
    echo '✅ Backup file extracted';

    cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
    mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
    chmod 600 ~/.ssh/authorized_keys;
    echo '✅ Authorized keys merged';

    if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
        echo '❌ Coolify installation failed';
        exit 1;
    fi
    echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh -p "$sshPort" -i "$sshKeyPath" -o "StrictHostKeyChecking no" root@"$destinationHost" "$remoteCommands" <"$backupFileName"; then
    echo "❌ Remote commands execution or Docker restart failed"
    exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! rm -f "$backupFileName"; then
        echo "❌ Failed to remove local backup file"
        exit 1
    fi
    echo "✅ Local backup file removed"
else
    echo "🚸 Local backup file not removed"
fi

im getting a msg saying i should be logged as azureuser `
azureuser@Coolifyserver:~$ sudo ./CoolifyBackup.sh
✅ Source directory exists
✅ SSH key file exists
✅ Docker is installed and running
Please login as the user "azureuser" rather than the user "root".

❌ SSH connection to failed
azureuser@Coolifyserver:~$`

im getting a msg saying i should be logged as azureuser

azureuser@Coolifyserver:~$ sudo ./CoolifyBackup.sh 
✅ Source directory exists 
✅ SSH key file exists 
✅ Docker is installed and running 
Please login as the user "azureuser" rather than the user "root".

❌ SSH connection to failed 
azureuser@Coolifyserver:~$

@rakithat20 it seems that the root user is disabled by default in Azure.[1] you may need to amend the script (i.e. ctrl+f, replace root with azureuser or another user) as well as grant that user passwordless sudo permissions[2]

Hey @AlejandroAkbal I modified your script for those who want to backup Coolify to a GCP Bucket using rclone

#!/bin/bash

# This script will backup your Coolify instance and upload the backup to a GCP bucket

# Configuration - Modify as needed
GCP_BUCKET="gcpbucketalias:bucket-name/backups/coolify/"

# -- Shouldn't need to modify anything below --
BACKUP_SOURCE_DIR="/data/coolify/"
BACKUP_FILE_NAME="coolify_backup_$(date +'%Y-%m-%d').tar.gz"

# Ensure the script is run as root
if [ "$EUID" -ne 0 ]; then
    printf "❌ Please run the script as root\n" >&2
    exit 1
fi

# Check if the source directory exists
if [ ! -d "$BACKUP_SOURCE_DIR" ]; then
    printf "❌ Source directory %s does not exist\n" "$BACKUP_SOURCE_DIR" >&2
    exit 1
fi
printf "✅ Source directory exists\n"

# Check if Docker is installed and running
if ! command -v docker >/dev/null 2>&1; then
    printf "❌ Docker is not installed\n" >&2
    exit 1
fi

if ! systemctl is-active --quiet docker; then
    printf "❌ Docker is not running\n" >&2
    exit 1
fi
printf "✅ Docker is installed and running\n"

# Get the names of all running Docker containers
container_names=$(docker ps --format '{{.Names}}')

# Initialize an array to hold the volume paths
volume_paths=()

# Loop over the container names and get their volumes
for container_name in $container_names; do
    volume_names=$(docker inspect --format '{{range .Mounts}}{{.Name}} {{end}}' "$container_name")
    for volume_name in $volume_names; do
        if [ -n "$volume_name" ]; then
            volume_paths+=("/var/lib/docker/volumes/$volume_name/_data")
        fi
    done
done

# Calculate and print the total size of the volumes and the source directory
total_size=$(du -csh "${volume_paths[@]}" 2>/dev/null | grep total | awk '{print $1}')
printf "✅ Total size of volumes to migrate: %s\n" "${total_size:-0}"

backup_source_dir_size=$(du -csh "$BACKUP_SOURCE_DIR" 2>/dev/null | grep total | awk '{print $1}')
printf "✅ Size of the source directory: %s\n" "${backup_source_dir_size:-0}"

# Create the backup tarball with progress feedback
printf "🚸 It's recommended to stop all Docker containers before creating the backup. Do you want to stop Docker? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    systemctl stop docker && systemctl stop docker.socket
    printf "✅ Docker stopped\n"
else
    printf "🚸 Docker not stopped, continuing with the backup\n"
fi

tar --exclude='*.sock' -Pczf "$BACKUP_FILE_NAME" -C / "$BACKUP_SOURCE_DIR" "$HOME/.ssh/authorized_keys" "${volume_paths[@]}" --checkpoint=.1000
if [ $? -ne 0 ]; then
    printf "❌ Backup file creation failed\n" >&2
    exit 1
fi
printf "✅ Backup file created\n"

# Transfer the backup file to GCP bucket
if ! rclone --gcs-bucket-policy-only copy "$BACKUP_FILE_NAME" "$GCP_BUCKET"; then
    printf "❌ Backup file transfer to GCP bucket failed\n" >&2
    exit 1
fi
printf "✅ Backup file transferred to GCP bucket\n"

# Clean up - Ask the user for confirmation before removing the local backup file
printf "Do you want to remove the local backup file? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! rm -f "$BACKUP_FILE_NAME"; then
        printf "❌ Failed to remove local backup file\n" >&2
        exit 1
    fi
    printf "✅ Local backup file removed\n"
else
    printf "🚸 Local backup file not removed\n"
fi

# Optionally start Docker again
printf "Do you want to start Docker again? (y/n)\n"
read -rp "Answer: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
    systemctl start docker && systemctl start docker.socket
    printf "✅ Docker started\n"
else
    printf "🚸 Docker not started\n"
fi```

This seems to be the only way to migrate from one machine to another one. I was capable of backing up and restore it to the same coolify image. I guess, by now, if I'd updated coolify before restore the file.dmp, I wouldn't be able to back this system up again with this file.dmp. I have also tried to change the .env file so this could matchup the origial system. Yet 500 error occored.

Anyways, my only question is how can I test this migrate.sh ?

Awesome script, worked perfectly on first try!

TPGLLC-US

You didn't per chance write another that runs from a clean machine, checks the storage bucket for backups, and upon choosing one restores it?

glad everyone likes the script <3

@dreadedhamish you can probably ask chatgpt to do that no?

Here is the script modified by o1 to handle a ssh key passphrase using ssh-agent:

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com" # destination server IP or domain

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Function to initialize ssh-agent and add the SSH key
initialize_ssh_agent() {
  # Check if ssh-agent is already running
  if [ -z "$SSH_AGENT_PID" ] || ! ps -p "$SSH_AGENT_PID" > /dev/null 2>&1; then
    echo "🔄 Starting ssh-agent..."
    eval "$(ssh-agent -s)"
    if [ $? -ne 0 ]; then
      echo "❌ Failed to start ssh-agent"
      exit 1
    fi
    echo "✅ ssh-agent started"
  else
    echo "✅ ssh-agent is already running"
  fi

  # Add the SSH key to the agent
  echo "🔒 Adding SSH key to ssh-agent"
  ssh-add "$sshKeyPath"
  if [ $? -ne 0 ]; then
    echo "❌ Failed to add SSH key. Please ensure the passphrase is correct."
    exit 1
  fi
  echo "✅ SSH key added to ssh-agent"
}

# Initialize ssh-agent and add the SSH key
initialize_ssh_agent

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@"$destinationHost" "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh "$backupSourceDir" 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup"
  read -rp "Do you want to stop Docker? (y/n): " answer
  if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf "$backupFileName" -C / "$backupSourceDir" "$HOME/.ssh/authorized_keys" $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh root@"$destinationHost" "$remoteCommands" < "$backupFileName"; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
  if ! rm -f "$backupFileName"; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

# Kill ssh-agent if it was started by this script
if [ -n "$SSH_AGENT_PID" ]; then
  echo "🔒 Stopping ssh-agent..."
  eval "$(ssh-agent -k)"
  echo "✅ ssh-agent stopped"
fi

Thanks everyone for helping with this.

Thanks a lot , Great work

Thanks very much for everyone's input. Yesterday, I had to migrate a live self-hosted admin and spent the better part of a day trying different approaches. In the end, I found the variation provided by @AlejandroAkbal to be spot-on for my needs, and it worked on the first run.

does this script works for v3? i am using v3 coolify and i want to migrate it to new vps. i guess moving it to v4 isn't possible automatically. so does it work for v3 ?