Migrate Coolify to a new server

migrate.sh

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com"

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@$destinationHost "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh $backupSourceDir 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup
  Do you want to stop Docker? (y/n)"
  read -r answer
  if [ "$answer" != "${answer#[Yy]}" ]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf $backupFileName -C / $backupSourceDir $HOME/.ssh/authorized_keys $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh -i "$sshKeyPath" -o "StrictHostKeyChecking no" root@$destinationHost "$remoteCommands" <$backupFileName; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [ "$answer" != "${answer#[Yy]}" ]; then
  if ! rm -f $backupFileName; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

This seems to be the only way to migrate from one machine to another one. I was capable of backing up and restore it to the same coolify image. I guess, by now, if I'd updated coolify before restore the file.dmp, I wouldn't be able to back this system up again with this file.dmp. I have also tried to change the .env file so this could matchup the origial system. Yet 500 error occored.

Anyways, my only question is how can I test this migrate.sh ?

Awesome script, worked perfectly on first try!

TPGLLC-US

You didn't per chance write another that runs from a clean machine, checks the storage bucket for backups, and upon choosing one restores it?

glad everyone likes the script <3

@dreadedhamish you can probably ask chatgpt to do that no?

Here is the script modified by o1 to handle a ssh key passphrase using ssh-agent:

#!/bin/bash

# This script will backup your Coolify instance and move everything to a new server. Docker volumes, Coolify database, and ssh keys

# 1. Script must run on the source server
# 2. Have all the containers running that you want to migrate

# Configuration - Modify as needed
sshKeyPath="$HOME/.ssh/your_private_key" # Key to destination server
destinationHost="server.example.com" # destination server IP or domain

# -- Shouldn't need to modify anything below --
backupSourceDir="/data/coolify/"
backupFileName="coolify_backup.tar.gz"

# Function to initialize ssh-agent and add the SSH key
initialize_ssh_agent() {
  # Check if ssh-agent is already running
  if [ -z "$SSH_AGENT_PID" ] || ! ps -p "$SSH_AGENT_PID" > /dev/null 2>&1; then
    echo "🔄 Starting ssh-agent..."
    eval "$(ssh-agent -s)"
    if [ $? -ne 0 ]; then
      echo "❌ Failed to start ssh-agent"
      exit 1
    fi
    echo "✅ ssh-agent started"
  else
    echo "✅ ssh-agent is already running"
  fi

  # Add the SSH key to the agent
  echo "🔒 Adding SSH key to ssh-agent"
  ssh-add "$sshKeyPath"
  if [ $? -ne 0 ]; then
    echo "❌ Failed to add SSH key. Please ensure the passphrase is correct."
    exit 1
  fi
  echo "✅ SSH key added to ssh-agent"
}

# Initialize ssh-agent and add the SSH key
initialize_ssh_agent

# Check if the source directory exists
if [ ! -d "$backupSourceDir" ]; then
  echo "❌ Source directory $backupSourceDir does not exist"
  exit 1
fi
echo "✅ Source directory exists"

# Check if the SSH key file exists
if [ ! -f "$sshKeyPath" ]; then
  echo "❌ SSH key file $sshKeyPath does not exist"
  exit 1
fi
echo "✅ SSH key file exists"

# Check if we can SSH to the destination server, ignore "The authenticity of host can't be established." errors
if ! ssh -o "StrictHostKeyChecking no" -o "ConnectTimeout=5" root@"$destinationHost" "exit"; then
  echo "❌ SSH connection to $destinationHost failed"
  exit 1
fi
echo "✅ SSH connection successful"

# Get the names of all running Docker containers
containerNames=$(docker ps --format '{{.Names}}')

# Initialize an empty string to hold the volume paths
volumePaths=""

# Loop over the container names
for containerName in $containerNames; do
  # Get the volumes for the current container
  volumeNames=$(docker inspect --format '{{range .Mounts}}{{.Name}}{{end}}' "$containerName")

  # Loop over the volume names
  for volumeName in $volumeNames; do
    # Check if the volume name is not empty
    if [ -n "$volumeName" ]; then
      # Add the volume path to the volume paths string
      volumePaths+=" /var/lib/docker/volumes/$volumeName"
    fi
  done
done

# Calculate the total size of the volumes
# shellcheck disable=SC2086
totalSize=$(du -csh $volumePaths 2>/dev/null | grep total | awk '{print $1}')

# Print the total size of the volumes
echo "✅ Total size of volumes to migrate: $totalSize"

# Print size of backupSourceDir
backupSourceDirSize=$(du -csh "$backupSourceDir" 2>/dev/null | grep total | awk '{print $1}')
echo "✅ Size of the source directory: $backupSourceDirSize"

# Check if the backup file already exists
if [ ! -f "$backupFileName" ]; then
  echo "🚸 Backup file does not exist, creating"

  # Recommend stopping docker before creating the backup
  echo "🚸 It's recommended to stop all Docker containers before creating the backup"
  read -rp "Do you want to stop Docker? (y/n): " answer
  if [[ "$answer" =~ ^[Yy]$ ]]; then
    if ! systemctl stop docker; then
      echo "❌ Docker stop failed"
      exit 1
    fi
    echo "✅ Docker stopped"
  else
    echo "🚸 Docker not stopped, continuing with the backup"
  fi

  # shellcheck disable=SC2086
  if ! tar --exclude='*.sock' -Pczf "$backupFileName" -C / "$backupSourceDir" "$HOME/.ssh/authorized_keys" $volumePaths; then
    echo "❌ Backup file creation failed"
    exit 1
  fi
  echo "✅ Backup file created"
else
  echo "🚸 Backup file already exists, skipping creation"
fi

# Define the remote commands to be executed
remoteCommands="
  # Check if Docker is a service
  if systemctl is-active --quiet docker; then
    # Stop Docker if it's a service
    if ! systemctl stop docker; then
      echo '❌ Docker stop failed';
      exit 1;
    fi
    echo '✅ Docker stopped';
  else
    echo 'ℹ️ Docker is not a service, skipping stop command';
  fi

  echo '🚸 Saving existing authorized keys...';
  cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_backup;

  echo '🚸 Extracting backup file...';
  if ! tar -Pxzf - -C /; then
    echo '❌ Backup file extraction failed';
    exit 1;
  fi
  echo '✅ Backup file extracted';

  echo '🚸 Merging authorized keys...';
  cat ~/.ssh/authorized_keys_backup ~/.ssh/authorized_keys | sort | uniq > ~/.ssh/authorized_keys_temp;
  mv ~/.ssh/authorized_keys_temp ~/.ssh/authorized_keys;
  chmod 600 ~/.ssh/authorized_keys;
  echo '✅ Authorized keys merged';

  if ! curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash; then
    echo '❌ Coolify installation failed';
    exit 1;
  fi
  echo '✅ Coolify installed';
"

# SSH to the destination server, execute the remote commands
if ! ssh root@"$destinationHost" "$remoteCommands" < "$backupFileName"; then
  echo "❌ Remote commands execution or Docker restart failed"
  exit 1
fi
echo "✅ Remote commands executed successfully"

# Clean up - Ask the user for confirmation before removing the local backup file
echo "Do you want to remove the local backup file? (y/n)"
read -r answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
  if ! rm -f "$backupFileName"; then
    echo "❌ Failed to remove local backup file"
    exit 1
  fi
  echo "✅ Local backup file removed"
else
  echo "🚸 Local backup file not removed"
fi

# Kill ssh-agent if it was started by this script
if [ -n "$SSH_AGENT_PID" ]; then
  echo "🔒 Stopping ssh-agent..."
  eval "$(ssh-agent -k)"
  echo "✅ ssh-agent stopped"
fi

Thanks everyone for helping with this.

Thanks a lot , Great work

Thanks very much for everyone's input. Yesterday, I had to migrate a live self-hosted admin and spent the better part of a day trying different approaches. In the end, I found the variation provided by @AlejandroAkbal to be spot-on for my needs, and it worked on the first run.

does this script works for v3? i am using v3 coolify and i want to migrate it to new vps. i guess moving it to v4 isn't possible automatically. so does it work for v3 ?