PeerTube admin tools for docker

README.md

Prerequisites

• Domain Name
• Main Dedicated Server
• Private FTP Backups Server

General budget: from ~17€ to ~100€ per month

Domain Name

Obviously you need a domain name for 15/20€ per year pointing to the IP of the Main Dedicated server ;-)

Be sure to configure DKIM TXT DNS record after installing PeerTube Stack.

Main dedicated server

Minimum hardware

1. ~10€ per month for small channels with few and short content
2. ~45€ per month for channels with average and moderate content and maybe one channel with lots of content
3. ~100€ per month for several channels with very regular and big content

• CPU X86_64:

  1. Intel Avoton / Atom with at least 2 cores but slow for transcoding
  2. Intel Core i3, Intel Xeon
  3. Intel Core i5, Intel Xeon

• RAM:

  1. >=2Go
  2. >=8Go
  3. >=32Go

• HDD:

  1. <= 500Go
  2. >= 2To
  3. > 4To

• Bandwith:

  1. >= 150Mbps / Unlimited
  2. >= 250Mbps / Unlimited
  3. >= 500Mbps / Unlimited

Source: https://github.com/Chocobozzz/PeerTube/blob/develop/FAQ.md#should-i-have-a-big-server-to-run-peertube

Operating System

• Linux distribution like Debian Buster, Ubuntu server...
• Docker installed https://docs.docker.com/install/

PeerTube Docker Stack

Installing with one shell-command: https://github.com/kimsible/install-peertube/

Private FTP server for daily backup

At least 100Go, generally free and provided with the Main Dedicated Server, used to backup all the Docker data-volume and setup including database with videos files.

PeerTube Backup Script

Creation and usage: As root or sudoer user

• Install curl
• Copy peertube-backup content to /usr/bin/peertube-backup
• Define SERVER_URL=user:[email protected] according to your needs
• Make /usr/bin/peertube-backup executable with chmod +x /usr/bin/peertube-backup
• Proceed backup by running peertube-backup (will overwrite existing daily backup on ftp server)
• The backup will be avalaible on backup ftp server

PeerTube Daily Backups

Creation and usage: As root or sudoer user

• Edit crontab with crontab -u root -e
• Add this line to run it as docker user every day at 5:25am :

25 5  * * * /usr/bin/peertube-backup

PeerTube Restore Script

Creation and usage: As root or sudoer user

• Install curl, lftp
• Copy peertube-restore content to /usr/bin/peertube-restore
• Define SERVER_URL=user:[email protected] according to your needs
• Make /usr/bin/peertube-restore executable with chmod +x /usr/bin/peertube-restore
• Proceed restore by running peertube-restore peertube-backup_YYYY-MM-DD.tar.bz2

Migrate old server to local new server

Creation and usage: As root or sudoer user

// Basic transfert
scp -r -p username@old-server:/var/peertube /var/peertube

// Optimized transfert
rsync -raP username@old-server:/var/peertube /var/peertube

PeerTube CLI Server Tools scripts

Creation: As root or sudoer user

Usage: As non-root user with docker group

• Copy peertube-cli content to /usr/bin/peertube
• Make /usr/bin/peertube executable with chmod +x /usr/bin/peertube
• Run any Server Tool script with peertube SERVER_TOOL

Server Tools List: https://docs.joinpeertube.org/#/maintain-tools?id=server-tools

PeerTube Assets Update - DEPRECATED

This feature is included in PeerTube with client-overrides since v2.3.0: https://github.com/Chocobozzz/PeerTube/releases/tag/v2.3.0

Creation and usage: As root or sudoer user

• Copy peertube-assets content to /usr/bin/peertube-assets
• Make /usr/bin/peertube-asssets executable with chmod +x /usr/bin/peertube-assets
• Copy peertube-assets.service content to /etc/systemd/system/peertube-assets.service
• Run systemctl daemon-reload and systemctl enable peertube-assets
• Put your assets in /var/peertube/docker-volume/data/client-overrides/assets with scp -r -p /local-assets-dir username@main-server:/var/peertube/docker-volume/data/assets
• Run systemctl restart peertube

peertube-assets

#!/usr/bin/env sh

# Constants
COMPOSE=/usr/local/bin/docker-compose

# Go to workir
cd /var/peertube

# Create assets volume if non-exists
mkdir -p ./docker-volume/data/client-overrides/assets

if [ ! -f docker-volume/data/client-overrides/assets/images/logo.svg ] && [ ! -f docker-volume/data/client-overrides/assets/images/favicon.png ] && [ ! -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
  echo 'No assets to update'
  exit 1
fi

# Do not update until server is up
echo -n "Wait until PeerTube server is up ... "
sleep 50s &
while [ -z "`$COMPOSE logs --tail=2 peertube | grep -o 'Server listening on'`" ]; do
  # Break if any stack errors occur
  # Displays errors and exit
  stack_errors=`$COMPOSE logs --tail=40 peertube | grep -i 'error'`
  if [ ! -z "$stack_errors" ]; then
    echo $stack_errors
    exit 1
  fi
  # Break after 50s / until pid of "sleep 50s" is destroyed
  # Display logs and exit
  if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 50s'`" ]; then
    $COMPOSE logs --tail=40 peertube
    exit 1
  fi
done
echo "done"

# logo.svg
if [ -f docker-volume/data/client-overrides/assets/images/logo.svg ]; then
  echo -n 'Update images/logo.svg ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/logo.svg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/logo.svg
  echo 'done'
fi

# favicon.png
if [ -f docker-volume/data/client-overrides/assets/images/favicon.png ]; then
  echo -n 'Update images/favicon.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/favicon.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/favicon.png
  echo 'done'
fi

# default-playlist.jpg
if [ -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
  echo -n 'Update images/default-playlist.jpg ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/default-playlist.jpg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/default-playlist.jpg
  echo 'done'
fi

# icons/icon-36x36.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png ]; then
  echo -n 'Update images/icons/icon-36x36.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-36x36.png
  echo 'done'
fi

# icons/icon-48x48.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png ]; then
  echo -n 'Update images/icons/icon-48x48.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-48x48.png
  echo 'done'
fi

# icons/icon-72x72.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png ]; then
  echo -n 'Update images/icons/icon-72x72.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-72x72.png
  echo 'done'
fi

# icons/icon-96x96.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png ]; then
  echo -n 'Update images/icons/icon-96x96.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-96x96.png
  echo 'done'
fi

# icons/icon-144x144.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png ]; then
  echo -n 'Update images/icons/icon-144x144.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-144x144.png
  echo 'done'
fi

# icons/icon-192x192.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png ]; then
  echo -n 'Update images/icons/icon-192x192.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-192x192.png
  echo 'done'
fi

# icons/icon-512x512.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png ]; then
  echo -n 'Update images/icons/icon-512x512.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-512x512.png
  echo 'done'
fi

peertube-assets.service

[Unit]
Description=PeerTube Assets Update Script
Requires=peertube.service
After=peertube.service
PartOf=peertube.service

[Service]
User=docker
Group=docker
WorkingDirectory=/var/peertube
ExecStart=/usr/bin/peertube-assets

[Install]
WantedBy=peertube.service

peertube-backup

#!/usr/bin/env sh

#########################################################
# DEEPENDENCIES                                         #
# curl                                                  #
#########################################################

#########################################################
# ENVIRONMENT VARIABLES                                 #
SERVER_URL=user:[email protected]
SERVER_SSL=false
#########################################################

# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
  echo "ERROR: this script must be run as root or as a sudoer user with sudo"
  exit 1
fi

# Constants
COMPOSE=/usr/local/bin/docker-compose
DB_TAR=db.tar
CURRENT_DATE=$(date '+%Y-%m-%d_%H-%M-%S')
TRANSCODED_TAR=peertube-backup_$CURRENT_DATE-transcoded.tar
BACKUP_BZIP=peertube-backup_$CURRENT_DATE.tar.bz2

if [ "$SERVER_SSL" = true ]; then
  SSL_OPTION⁼'--ssl '
else
  SSL_OPTION=''
fi

# Go to workir
cd /var/peertube

# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

# Do not backup if PostgreSQL container is down
if [ -z "$($COMPOSE ps -q $PEERTUBE_DB_HOSTNAME)" ]; then
  echo -n "ERROR: PostgreSQL docker container is down, can't backup"
  exit 1
fi

# Get videoFile table size in PostgreSQL database
echo -n "Get number of rows of videoFile table in PostgreSQL database ... "
TRANSCODED_COUNT=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT count(*) FROM "videoFile"'`
echo "done"

# Get videoFile table last id in PostgreSQL database
echo -n "Get last id of videoFile table in PostgreSQL database ... "
TRANSCODED_LAST=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT max(id) FROM "videoFile"'`
echo "done"

# Dump PostgreSQL database
echo -n "Dump existing PostgreSQL database ... "
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_dump -U $POSTGRES_USER -Ft $POSTGRES_DB > docker-volume/$DB_TAR
echo "done"

# Archive and Compress DB Dump, Docker volume and Compose files
# Do not include mounted db though db.tar has been created
# Do not include transcoded video files in /data/videos and /data/streaming-playlists
# Send the compressed file to ftp
echo "Archive, compress and send to ftp existing database dump, Docker volume and Compose files ... "
tar --exclude=docker-volume/db --exclude=docker-volume/data/videos --exclude=docker-volume/data/streaming-playlists -cjf /tmp/peertube-backup.tar.bz2 docker-volume .env docker-compose.yml
curl $SSL_OPTION-aT /tmp/peertube-backup.tar.bz2 ftp://$SERVER_URL/$BACKUP_BZIP
rm -f /tmp/peertube-backup.tar.bz2
echo "done"

# Clean db.tar
echo -n "Clean archived database dump ... "
rm -f docker-volume/$DB_TAR
echo "done"

# Init markers if needed
if [ ! -f docker-volume/data/.videocount ] || [ ! -f docker-volume/data/.videolast ]; then
  echo -n "Init current number of transcoded video files and last index markers... "
  echo 0 > docker-volume/data/.videocount
  echo 0 > docker-volume/data/.videolast
  echo "done"
fi

# Send transcoded videos files to ftp if needed
if [ "$TRANSCODED_COUNT" -ne "`cat docker-volume/data/.videocount`" ] || [ "$TRANSCODED_LAST" -ne "`cat docker-volume/data/.videolast`" ]; then
  echo "Archive and send transcoded video files ... "
  tar -cf /tmp/peertube-backup-transcoded.tar docker-volume/data/videos docker-volume/data/streaming-playlists
  curl $SSL_OPTION-aT /tmp/peertube-backup-transcoded.tar ftp://$SERVER_URL/$TRANSCODED_TAR
  rm -f /tmp/peertube-backup-transcoded.tar
  echo "done"
fi

# Set current number of transcoded videos files and the last index in db to backup only if needed
echo -n "Set current number of transcoded video files and last index ... "
echo $TRANSCODED_COUNT > docker-volume/data/.videocount
echo $TRANSCODED_LAST > docker-volume/data/.videolast
echo "done"

peertube-cli

#!/usr/bin/env sh

# Constants
COMPOSE=/usr/local/bin/docker-compose
SERVER_TOOLS="
parse-log
create-transcoding-job
create-import-video-file-job
prune-storage
optimize-old-videos
update-host
reset-password
plugin:install
plugin:uninstall
purge-old-account
psql
"

# Limit command to Server Tools List
if [ -z $1 ] || [ -z "`echo $SERVER_TOOLS | grep -o -w $1`" ]; then
  echo "\nUsage:	peertube SERVER_TOOL\n"
  echo "Server Tools:\n$SERVER_TOOLS"
  exit 1
fi

# Go to workir
cd /var/peertube

# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

if [ "$1" = "purge-old-account" ]; then
  username=$2
  if [ -z "$username" ]; then
    echo "\nUsage:	peertube purge-old-account USERNAME\n"
    exit 1
  fi
  count=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c "SELECT count(*) FROM account WHERE name = '$username';" | xargs`
  if [ "$count" -gt 0 ]; then
    echo "ERROR: can't purge an existing account"
    exit 1
  fi
  # Run psql delete command
  channelname=$username"_channel"
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$username';"
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$channelname';"
elif [ "$1" = "psql" ]; then
  # Run psql command
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -U $POSTGRES_USER $POSTGRES_DB -c "$2"
else
  # Run ServerTools command
  $COMPOSE exec -u peertube -e NODE_CONFIG_DIR=/config -e NODE_ENV=production peertube npm run $@
fi

peertube-restore

#!/usr/bin/env sh

#########################################################
# DEEPENDENCIES                                         #
# lftp                                                  #
# curl                                                  #
#########################################################

#########################################################
# ENVIRONMENT VARIABLES                                 #
SERVER_CREDENTIALS=user:password
SERVER_ADDRESS=address.tld
SERVER_SSL=false
#########################################################

# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
  echo "ERROR: this script must be run as root or as a sudoer user with sudo"
  exit 1
fi

# Constants
BACKUP=$1
COMPOSE=/usr/local/bin/docker-compose

if [ "$SERVER_SSL" = true ]; then
  SCHEME⁼'ftps'
  SSL_CMD=''
  SSL_OPTION='--ssl '
else
  SCHEME='ftp'
  SSL_CMD='set ssl:verify-certificate false;'
  SSL_OPTION=''
fi

# Backup name argument must exist and must match ^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$
# Otherwise, display usage and all backups from ftp
match=`echo $BACKUP | grep -E -o "^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$"`
if [ -z $BACKUP ] || [ -z $match ]; then
  echo "\nUsage:	peertube-restore peertube-backup_YYYY-MM-DD(-transcoded).tar(.bz2)\n"
  lftp $SCHEME://$SERVER_CREDENTIALS@$SERVER_ADDRESS -e "$SSL_CMD ls; quit"
  exit 1
fi

# Get tar options
bzip=`echo $BACKUP | grep -E -o "bz2$"`
if [ -z $bzip ]; then
  tar_option='xf'
else
  tar_option='xjf'
fi

# Stop service to not conflict restore
echo -n "Stop systemd service to not conflict restore ... "
systemctl stop peertube
echo "done"

# Get backup to restore from ftp into buffer and extract it
echo "Get $BACKUP from ftp backup server and extract it to /var/peertube ... "
curl -L $SSL_OPTION-u $SERVER_CREDENTIALS ftp://$SERVER_ADDRESS/$BACKUP -o /tmp/$BACKUP
tar -$tar_option /tmp/$BACKUP -C /var/peertube
rm -f /tmp/$BACKUP
echo "done"

# Go to workdir
cd /var/peertube

# Restore db.tar if exists
if [ -f ./docker-volume/db.tar ] && [ -f .env ] && [ -f .docker-compose.yml ]; then
  # Get postgres user
  POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

  # Get postgres db
  POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

  # Get postgres service name
  PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

  # Remove existing mounted db files
  echo -n "Remove existing mounted /var/docker-volume/db ... "
  rm -rf ./docker-volume/db
  sleep 1s
  echo "done"

  # Up postgres service
  $COMPOSE up -d $PEERTUBE_DB_HOSTNAME
  echo -n "Wait until PosgreSQL database is up ... "
  sleep 10s &
  while [ -z "`$COMPOSE logs --tail=2 $PEERTUBE_DB_HOSTNAME | grep -o 'database system is ready to accept connections'`" ]; do
    # Break if any database errors occur
    # Displays errors and exit
    db_errors=`$COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME | grep -i 'error'`
    if [ ! -z "$db_errors" ]; then
      echo $db_errors
      exit 1
    fi
    # Break after 10s / until pid of "sleep 10s" is destroyed
    # Display logs and exit
    if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 10s'`" ]; then
      $COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME
      exit 1
    fi
  done
  echo "done"

  # Run restore command
  echo -n "Restore existing dump /var/peertube/docker-volume/db.tar ... "
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_restore -U $POSTGRES_USER -d $POSTGRES_DB < ./docker-volume/db.tar
  echo "done"

  # Clean db.tar
  echo -n "Clean existing dump /var/peertube/docker-volume/db.tar ... "
  rm -f ./docker-volume/db.tar
  echo "done"
fi

# Re-start service
echo -n "Re-start stopped systemd service ... "
systemctl start peertube
echo "done"

You might want to limit your SQL query of videoFiles to local videos.

Thanks, you're right, didn't think about it.