Bash scripts to create VOD HLS stream with ffmpeg using GPU

create-vod-hls-gpu.sh

#!/usr/bin/env bash
START_TIME=$SECONDS
set -e

echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls-gpu.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1

# comment/add lines here to control which renditions would be created
renditions=(
# resolution  bitrate  audio-rate
  "426x240    400k    128k"
  "640x360    800k     192k"
  "842x480    1400k    256k"
  "1280x720   2800k    256k"
  "1920x1080  5000k    320k"
)

segment_target_duration=10       # try to create a new segment every X seconds
max_bitrate_ratio=1.07          # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5   # maximum buffer size between bitrate conformance checks

#########################################################################

source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
  target="${source##*/}" # leave only last component of path
  target="${target%.*}"  # strip extension
fi
mkdir -p ${target}

# ----CUSTOM----
sourceResolution="$(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 ${source})"
# echo ${sourceResolution}
arrIN=(${sourceResolution//x/ })
sourceWidth="${arrIN[0]}"
sourceHeight="${arrIN[1]}"

echo ${sourceWidth}
echo ${sourceHeight}

sourceAudioBitRate="$(ffprobe -v error -select_streams a:0 -show_entries stream=bit_rate -of csv=s=x:p=0 ${source})"
sourceAudioBitRateFormatted=$((sourceAudioBitRate / 1000))
# ----END CUSTOM----

key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer

# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -profile:v main -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"

# misc params
misc_params="-hide_banner -y -hwaccel cuvid -vsync 0"

master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
resolutionValid=0
prevHeight=0
for rendition in "${renditions[@]}"; do
  # drop extraneous spaces
  rendition="${rendition/[[:space:]]+/ }"

  # rendition fields
  resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
  bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
  audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"

  audioBitRateFormatted=${audiorate%?} # remove "k" at the last index

  # take highest possible audio bit rate
  if [ $audioBitRateFormatted -gt $sourceAudioBitRateFormatted ]; then
      audiorate=${sourceAudioBitRateFormatted}k
  fi

  # calculated fields
  width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
  height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
  maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
  bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
  bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
  name="${height}p"
  
  if [ $sourceHeight -le $prevHeight ]; then
      echo "video source has height smaller than output height (${height})"
      break
  fi

  widthParam=0
  heightParam=0

  if [ $(((width / sourceWidth) * sourceHeight)) -gt $height ]; then
    widthParam=-2
    heightParam=$height
  else
    widthParam=$width
    heightParam=-2
  fi
  
  cmd+=" ${static_params} -vf hwupload_cuda,scale_npp=w=${widthParam}:h=${heightParam}"
  cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
  cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
  
  # add rendition entry in the master playlist
  master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"

  resolutionValid=1
  prevHeight=${height}
done

if [ $resolutionValid -eq 1 ]; then
  # start conversion
  echo -e "Executing command:\n/content/ffmpeg ${misc_params} -i ${source} ${cmd}\n"
  /content/ffmpeg ${misc_params} -i ${source} ${cmd}
  # create master playlist file
  echo -e "${master_playlist}" > ${target}/playlist.m3u8
  echo "Done - encoded HLS is at ${target}/"
else
  echo "Video source is too small"
  exit 1
fi

ELAPSED_TIME=$(($SECONDS - $START_TIME))

echo "Elapsed time: ${ELAPSED_TIME}"
echo "-----FINISH GENERATING HLS STREAM-----"

This is GPU-version that has same features with my previous script (non-GPU)

What's in this version?

This script run ffmpeg that uses NVIDIA GPU to reduce significant time to generate HLS stream.

Prerequisite

• Make sure you're having NVIDIA GPU card and driver installed (run nvidia-smito check)
• FFMPEG build from source that supports GPU (see my instructions on how to build below)
• sudo privilege

Build FFmpeg from source with GPU support

This instructions is for Ubuntu, other Linux distributions should be same

First we need to install NVIDIA API Header (required before building ffmpeg).

Change to some directory before proceed

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers && sudo make install

Next install required dependencies:

#install dependencies
sudo apt-get update -qq && sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libgnutls28-dev \
  libtool \
  libvorbis-dev \
  pkg-config \
  texinfo \
  wget \
  yasm \
  zlib1g-dev \
  bc

sudo apt-get install nasm libx264-dev libx265-dev libnuma-dev libvpx-dev libfdk-aac-dev libmp3lame-dev libopus-dev

Finally clone and build FFmpeg from source. Change to some directory before proceed:

# clone ffmpeg
git clone https://git.ffmpeg.org/ffmpeg.git

# checkout to version 3.4. DO NOT use latest version at the moment (4.0) because it's incompatible with many NVIDIA drivers
cd ffmpeg && git checkout 567c20f78109588620c57696ab39ff314fe12d6c

./configure \
  --enable-gpl \
  --enable-gnutls \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-cuda-sdk \
  --enable-cuvid \
  --enable-nvenc \
  --enable-nonfree \
  --enable-libnpp \
  --extra-cflags=-I/usr/local/cuda/include \
  --extra-ldflags=-L/usr/local/cuda/lib64 \
  --nvccflags="-gencode arch=compute_75,code=sm_75 -O2"

make -j 10

After finishing, you'll see an ffmpeg binary generated in your current working directory (NOTICE this). Now you're ready to go with FFMPEG with GPU support.

How to run

Look back my script on line 120:

# Change this line
/content/ffmpeg ${misc_params} -i ${source} ${cmd}

# To this
<path_to_the_ffmpeg_binary> ${misc_params} -i ${source} ${cmd}

Save it and now run this command:

bash create-vod-hls-gpu.sh video.mp4

How fast is it?

Tested in environment:

• Ubuntu 18.04
• 16GB RAM + 2.2GHz CPU
• One Tesla K80 graphics card (11.5GB Memory-----NVIDIA-SMI 450.51.05-----Driver Version: 418.67-----CUDA Version: 10.1 )

Test video:

• Duration: 120 mins
• Resolution: 1080p (1920x800)

Total time to process reduced from 14 hours down to just 1,5 hours 😄😄

Working example

https://colab.research.google.com/drive/1nME0I9AHvJu-tRnT3c1SOa40wgs9RIqD?usp=sharing

Can you add AMD GPU support?

Hi @xubmuajkub,

For support with AMD GPU, you may refer to this thread, basically the solution is updating my script with the following:

• change hwaccel to auto
• chang -c:v to h264_amf

I haven't tested yet ('cuz I don't have any PC that has AMD 😅😅), but searching for awhile and most them have same solution as described above.

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

It would be good if we can have Windows .bat file

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

• Make sure you have installed nv-codec-headers (see the beginning part of my instructions)
• Note that latest version of ffmpeg may not work for all graphics card, for me I use version 3.4, you may want to choose another version (I think from 3.0-3.4 is good)

This issue is described here.

I don't have time at the moment to convert my script to .bat, you're free to create your own.

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I only run this script on Ubuntu so I'm not sure it works same in other platforms.

Thanks for your report