PeerTube admin tools for docker

README.md

Prerequisites

• Domain Name
• Main Dedicated Server
• Private FTP Backups Server

General budget: from ~17€ to ~100€ per month

Domain Name

Obviously you need a domain name for 15/20€ per year pointing to the IP of the Main Dedicated server ;-)

Be sure to configure DKIM TXT DNS record after installing PeerTube Stack.

Main dedicated server

Minimum hardware

1. ~10€ per month for small channels with few and short content
2. ~45€ per month for channels with average and moderate content and maybe one channel with lots of content
3. ~100€ per month for several channels with very regular and big content

• CPU X86_64:

  1. Intel Avoton / Atom with at least 2 cores but slow for transcoding
  2. Intel Core i3, Intel Xeon
  3. Intel Core i5, Intel Xeon

• RAM:

  1. >=2Go
  2. >=8Go
  3. >=32Go

• HDD:

  1. <= 500Go
  2. >= 2To
  3. > 4To

• Bandwith:

  1. >= 150Mbps / Unlimited
  2. >= 250Mbps / Unlimited
  3. >= 500Mbps / Unlimited

Source: https://github.com/Chocobozzz/PeerTube/blob/develop/FAQ.md#should-i-have-a-big-server-to-run-peertube

Operating System

• Linux distribution like Debian Buster, Ubuntu server...
• Docker installed https://docs.docker.com/install/

PeerTube Docker Stack

Installing with one shell-command: https://github.com/kimsible/install-peertube/

Private FTP server for daily backup

At least 100Go, generally free and provided with the Main Dedicated Server, used to backup all the Docker data-volume and setup including database with videos files.

PeerTube Backup Script

Creation and usage: As root or sudoer user

• Install curl
• Copy peertube-backup content to /usr/bin/peertube-backup
• Define SERVER_URL=user:[email protected] according to your needs
• Make /usr/bin/peertube-backup executable with chmod +x /usr/bin/peertube-backup
• Proceed backup by running peertube-backup (will overwrite existing daily backup on ftp server)
• The backup will be avalaible on backup ftp server

PeerTube Daily Backups

Creation and usage: As root or sudoer user

• Edit crontab with crontab -u root -e
• Add this line to run it as docker user every day at 5:25am :

25 5  * * * /usr/bin/peertube-backup

PeerTube Restore Script

Creation and usage: As root or sudoer user

• Install curl, lftp
• Copy peertube-restore content to /usr/bin/peertube-restore
• Define SERVER_URL=user:[email protected] according to your needs
• Make /usr/bin/peertube-restore executable with chmod +x /usr/bin/peertube-restore
• Proceed restore by running peertube-restore peertube-backup_YYYY-MM-DD.tar.bz2

Migrate old server to local new server

Creation and usage: As root or sudoer user

// Basic transfert
scp -r -p username@old-server:/var/peertube /var/peertube

// Optimized transfert
rsync -raP username@old-server:/var/peertube /var/peertube

PeerTube CLI Server Tools scripts

Creation: As root or sudoer user

Usage: As non-root user with docker group

• Copy peertube-cli content to /usr/bin/peertube
• Make /usr/bin/peertube executable with chmod +x /usr/bin/peertube
• Run any Server Tool script with peertube SERVER_TOOL

Server Tools List: https://docs.joinpeertube.org/#/maintain-tools?id=server-tools

PeerTube Assets Update - DEPRECATED

This feature is included in PeerTube with client-overrides since v2.3.0: https://github.com/Chocobozzz/PeerTube/releases/tag/v2.3.0

Creation and usage: As root or sudoer user

• Copy peertube-assets content to /usr/bin/peertube-assets
• Make /usr/bin/peertube-asssets executable with chmod +x /usr/bin/peertube-assets
• Copy peertube-assets.service content to /etc/systemd/system/peertube-assets.service
• Run systemctl daemon-reload and systemctl enable peertube-assets
• Put your assets in /var/peertube/docker-volume/data/client-overrides/assets with scp -r -p /local-assets-dir username@main-server:/var/peertube/docker-volume/data/assets
• Run systemctl restart peertube

peertube-assets

#!/usr/bin/env sh

# Constants
COMPOSE=/usr/local/bin/docker-compose

# Go to workir
cd /var/peertube

# Create assets volume if non-exists
mkdir -p ./docker-volume/data/client-overrides/assets

if [ ! -f docker-volume/data/client-overrides/assets/images/logo.svg ] && [ ! -f docker-volume/data/client-overrides/assets/images/favicon.png ] && [ ! -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
  echo 'No assets to update'
  exit 1
fi

# Do not update until server is up
echo -n "Wait until PeerTube server is up ... "
sleep 50s &
while [ -z "`$COMPOSE logs --tail=2 peertube | grep -o 'Server listening on'`" ]; do
  # Break if any stack errors occur
  # Displays errors and exit
  stack_errors=`$COMPOSE logs --tail=40 peertube | grep -i 'error'`
  if [ ! -z "$stack_errors" ]; then
    echo $stack_errors
    exit 1
  fi
  # Break after 50s / until pid of "sleep 50s" is destroyed
  # Display logs and exit
  if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 50s'`" ]; then
    $COMPOSE logs --tail=40 peertube
    exit 1
  fi
done
echo "done"

# logo.svg
if [ -f docker-volume/data/client-overrides/assets/images/logo.svg ]; then
  echo -n 'Update images/logo.svg ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/logo.svg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/logo.svg
  echo 'done'
fi

# favicon.png
if [ -f docker-volume/data/client-overrides/assets/images/favicon.png ]; then
  echo -n 'Update images/favicon.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/favicon.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/favicon.png
  echo 'done'
fi

# default-playlist.jpg
if [ -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
  echo -n 'Update images/default-playlist.jpg ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/default-playlist.jpg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/default-playlist.jpg
  echo 'done'
fi

# icons/icon-36x36.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png ]; then
  echo -n 'Update images/icons/icon-36x36.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-36x36.png
  echo 'done'
fi

# icons/icon-48x48.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png ]; then
  echo -n 'Update images/icons/icon-48x48.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-48x48.png
  echo 'done'
fi

# icons/icon-72x72.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png ]; then
  echo -n 'Update images/icons/icon-72x72.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-72x72.png
  echo 'done'
fi

# icons/icon-96x96.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png ]; then
  echo -n 'Update images/icons/icon-96x96.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-96x96.png
  echo 'done'
fi

# icons/icon-144x144.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png ]; then
  echo -n 'Update images/icons/icon-144x144.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-144x144.png
  echo 'done'
fi

# icons/icon-192x192.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png ]; then
  echo -n 'Update images/icons/icon-192x192.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-192x192.png
  echo 'done'
fi

# icons/icon-512x512.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png ]; then
  echo -n 'Update images/icons/icon-512x512.png ... '
  docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-512x512.png
  echo 'done'
fi

peertube-assets.service

[Unit]
Description=PeerTube Assets Update Script
Requires=peertube.service
After=peertube.service
PartOf=peertube.service

[Service]
User=docker
Group=docker
WorkingDirectory=/var/peertube
ExecStart=/usr/bin/peertube-assets

[Install]
WantedBy=peertube.service

peertube-backup

#!/usr/bin/env sh

#########################################################
# DEEPENDENCIES                                         #
# curl                                                  #
#########################################################

#########################################################
# ENVIRONMENT VARIABLES                                 #
SERVER_URL=user:[email protected]
SERVER_SSL=false
#########################################################

# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
  echo "ERROR: this script must be run as root or as a sudoer user with sudo"
  exit 1
fi

# Constants
COMPOSE=/usr/local/bin/docker-compose
DB_TAR=db.tar
CURRENT_DATE=$(date '+%Y-%m-%d_%H-%M-%S')
TRANSCODED_TAR=peertube-backup_$CURRENT_DATE-transcoded.tar
BACKUP_BZIP=peertube-backup_$CURRENT_DATE.tar.bz2

if [ "$SERVER_SSL" = true ]; then
  SSL_OPTION⁼'--ssl '
else
  SSL_OPTION=''
fi

# Go to workir
cd /var/peertube

# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

# Do not backup if PostgreSQL container is down
if [ -z "$($COMPOSE ps -q $PEERTUBE_DB_HOSTNAME)" ]; then
  echo -n "ERROR: PostgreSQL docker container is down, can't backup"
  exit 1
fi

# Get videoFile table size in PostgreSQL database
echo -n "Get number of rows of videoFile table in PostgreSQL database ... "
TRANSCODED_COUNT=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT count(*) FROM "videoFile"'`
echo "done"

# Get videoFile table last id in PostgreSQL database
echo -n "Get last id of videoFile table in PostgreSQL database ... "
TRANSCODED_LAST=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT max(id) FROM "videoFile"'`
echo "done"

# Dump PostgreSQL database
echo -n "Dump existing PostgreSQL database ... "
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_dump -U $POSTGRES_USER -Ft $POSTGRES_DB > docker-volume/$DB_TAR
echo "done"

# Archive and Compress DB Dump, Docker volume and Compose files
# Do not include mounted db though db.tar has been created
# Do not include transcoded video files in /data/videos and /data/streaming-playlists
# Send the compressed file to ftp
echo "Archive, compress and send to ftp existing database dump, Docker volume and Compose files ... "
tar --exclude=docker-volume/db --exclude=docker-volume/data/videos --exclude=docker-volume/data/streaming-playlists -cjf /tmp/peertube-backup.tar.bz2 docker-volume .env docker-compose.yml
curl $SSL_OPTION-aT /tmp/peertube-backup.tar.bz2 ftp://$SERVER_URL/$BACKUP_BZIP
rm -f /tmp/peertube-backup.tar.bz2
echo "done"

# Clean db.tar
echo -n "Clean archived database dump ... "
rm -f docker-volume/$DB_TAR
echo "done"

# Init markers if needed
if [ ! -f docker-volume/data/.videocount ] || [ ! -f docker-volume/data/.videolast ]; then
  echo -n "Init current number of transcoded video files and last index markers... "
  echo 0 > docker-volume/data/.videocount
  echo 0 > docker-volume/data/.videolast
  echo "done"
fi

# Send transcoded videos files to ftp if needed
if [ "$TRANSCODED_COUNT" -ne "`cat docker-volume/data/.videocount`" ] || [ "$TRANSCODED_LAST" -ne "`cat docker-volume/data/.videolast`" ]; then
  echo "Archive and send transcoded video files ... "
  tar -cf /tmp/peertube-backup-transcoded.tar docker-volume/data/videos docker-volume/data/streaming-playlists
  curl $SSL_OPTION-aT /tmp/peertube-backup-transcoded.tar ftp://$SERVER_URL/$TRANSCODED_TAR
  rm -f /tmp/peertube-backup-transcoded.tar
  echo "done"
fi

# Set current number of transcoded videos files and the last index in db to backup only if needed
echo -n "Set current number of transcoded video files and last index ... "
echo $TRANSCODED_COUNT > docker-volume/data/.videocount
echo $TRANSCODED_LAST > docker-volume/data/.videolast
echo "done"

peertube-cli

#!/usr/bin/env sh

# Constants
COMPOSE=/usr/local/bin/docker-compose
SERVER_TOOLS="
parse-log
create-transcoding-job
create-import-video-file-job
prune-storage
optimize-old-videos
update-host
reset-password
plugin:install
plugin:uninstall
purge-old-account
psql
"

# Limit command to Server Tools List
if [ -z $1 ] || [ -z "`echo $SERVER_TOOLS | grep -o -w $1`" ]; then
  echo "\nUsage:	peertube SERVER_TOOL\n"
  echo "Server Tools:\n$SERVER_TOOLS"
  exit 1
fi

# Go to workir
cd /var/peertube

# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

if [ "$1" = "purge-old-account" ]; then
  username=$2
  if [ -z "$username" ]; then
    echo "\nUsage:	peertube purge-old-account USERNAME\n"
    exit 1
  fi
  count=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c "SELECT count(*) FROM account WHERE name = '$username';" | xargs`
  if [ "$count" -gt 0 ]; then
    echo "ERROR: can't purge an existing account"
    exit 1
  fi
  # Run psql delete command
  channelname=$username"_channel"
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$username';"
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$channelname';"
elif [ "$1" = "psql" ]; then
  # Run psql command
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -U $POSTGRES_USER $POSTGRES_DB -c "$2"
else
  # Run ServerTools command
  $COMPOSE exec -u peertube -e NODE_CONFIG_DIR=/config -e NODE_ENV=production peertube npm run $@
fi

peertube-restore

#!/usr/bin/env sh

#########################################################
# DEEPENDENCIES                                         #
# lftp                                                  #
# curl                                                  #
#########################################################

#########################################################
# ENVIRONMENT VARIABLES                                 #
SERVER_CREDENTIALS=user:password
SERVER_ADDRESS=address.tld
SERVER_SSL=false
#########################################################

# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
  echo "ERROR: this script must be run as root or as a sudoer user with sudo"
  exit 1
fi

# Constants
BACKUP=$1
COMPOSE=/usr/local/bin/docker-compose

if [ "$SERVER_SSL" = true ]; then
  SCHEME⁼'ftps'
  SSL_CMD=''
  SSL_OPTION='--ssl '
else
  SCHEME='ftp'
  SSL_CMD='set ssl:verify-certificate false;'
  SSL_OPTION=''
fi

# Backup name argument must exist and must match ^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$
# Otherwise, display usage and all backups from ftp
match=`echo $BACKUP | grep -E -o "^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$"`
if [ -z $BACKUP ] || [ -z $match ]; then
  echo "\nUsage:	peertube-restore peertube-backup_YYYY-MM-DD(-transcoded).tar(.bz2)\n"
  lftp $SCHEME://$SERVER_CREDENTIALS@$SERVER_ADDRESS -e "$SSL_CMD ls; quit"
  exit 1
fi

# Get tar options
bzip=`echo $BACKUP | grep -E -o "bz2$"`
if [ -z $bzip ]; then
  tar_option='xf'
else
  tar_option='xjf'
fi

# Stop service to not conflict restore
echo -n "Stop systemd service to not conflict restore ... "
systemctl stop peertube
echo "done"

# Get backup to restore from ftp into buffer and extract it
echo "Get $BACKUP from ftp backup server and extract it to /var/peertube ... "
curl -L $SSL_OPTION-u $SERVER_CREDENTIALS ftp://$SERVER_ADDRESS/$BACKUP -o /tmp/$BACKUP
tar -$tar_option /tmp/$BACKUP -C /var/peertube
rm -f /tmp/$BACKUP
echo "done"

# Go to workdir
cd /var/peertube

# Restore db.tar if exists
if [ -f ./docker-volume/db.tar ] && [ -f .env ] && [ -f .docker-compose.yml ]; then
  # Get postgres user
  POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"

  # Get postgres db
  POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"

  # Get postgres service name
  PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"

  # Remove existing mounted db files
  echo -n "Remove existing mounted /var/docker-volume/db ... "
  rm -rf ./docker-volume/db
  sleep 1s
  echo "done"

  # Up postgres service
  $COMPOSE up -d $PEERTUBE_DB_HOSTNAME
  echo -n "Wait until PosgreSQL database is up ... "
  sleep 10s &
  while [ -z "`$COMPOSE logs --tail=2 $PEERTUBE_DB_HOSTNAME | grep -o 'database system is ready to accept connections'`" ]; do
    # Break if any database errors occur
    # Displays errors and exit
    db_errors=`$COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME | grep -i 'error'`
    if [ ! -z "$db_errors" ]; then
      echo $db_errors
      exit 1
    fi
    # Break after 10s / until pid of "sleep 10s" is destroyed
    # Display logs and exit
    if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 10s'`" ]; then
      $COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME
      exit 1
    fi
  done
  echo "done"

  # Run restore command
  echo -n "Restore existing dump /var/peertube/docker-volume/db.tar ... "
  $COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_restore -U $POSTGRES_USER -d $POSTGRES_DB < ./docker-volume/db.tar
  echo "done"

  # Clean db.tar
  echo -n "Clean existing dump /var/peertube/docker-volume/db.tar ... "
  rm -f ./docker-volume/db.tar
  echo "done"
fi

# Re-start service
echo -n "Re-start stopped systemd service ... "
systemctl start peertube
echo "done"

You might want to limit your SQL query of videoFiles to local videos.

You might want to limit your SQL query of videoFiles to local videos.

Thanks, you're right, didn't think about it.