Bash scripts to create VOD HLS stream with ffmpeg using GPU

create-vod-hls-gpu.sh

#!/usr/bin/env bash
START_TIME=$SECONDS
set -e

echo "-----START GENERATING HLS STREAM-----"
# Usage create-vod-hls-gpu.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1

# comment/add lines here to control which renditions would be created
renditions=(
# resolution  bitrate  audio-rate
  "426x240    400k    128k"
  "640x360    800k     192k"
  "842x480    1400k    256k"
  "1280x720   2800k    256k"
  "1920x1080  5000k    320k"
)

segment_target_duration=10       # try to create a new segment every X seconds
max_bitrate_ratio=1.07          # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5   # maximum buffer size between bitrate conformance checks

#########################################################################

source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
  target="${source##*/}" # leave only last component of path
  target="${target%.*}"  # strip extension
fi
mkdir -p ${target}

# ----CUSTOM----
sourceResolution="$(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 ${source})"
# echo ${sourceResolution}
arrIN=(${sourceResolution//x/ })
sourceWidth="${arrIN[0]}"
sourceHeight="${arrIN[1]}"

echo ${sourceWidth}
echo ${sourceHeight}

sourceAudioBitRate="$(ffprobe -v error -select_streams a:0 -show_entries stream=bit_rate -of csv=s=x:p=0 ${source})"
sourceAudioBitRateFormatted=$((sourceAudioBitRate / 1000))
# ----END CUSTOM----

key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer

# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -profile:v main -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"

# misc params
misc_params="-hide_banner -y -hwaccel cuvid -vsync 0"

master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
resolutionValid=0
prevHeight=0
for rendition in "${renditions[@]}"; do
  # drop extraneous spaces
  rendition="${rendition/[[:space:]]+/ }"

  # rendition fields
  resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
  bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
  audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"

  audioBitRateFormatted=${audiorate%?} # remove "k" at the last index

  # take highest possible audio bit rate
  if [ $audioBitRateFormatted -gt $sourceAudioBitRateFormatted ]; then
      audiorate=${sourceAudioBitRateFormatted}k
  fi

  # calculated fields
  width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
  height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
  maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
  bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
  bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
  name="${height}p"
  
  if [ $sourceHeight -le $prevHeight ]; then
      echo "video source has height smaller than output height (${height})"
      break
  fi

  widthParam=0
  heightParam=0

  if [ $(((width / sourceWidth) * sourceHeight)) -gt $height ]; then
    widthParam=-2
    heightParam=$height
  else
    widthParam=$width
    heightParam=-2
  fi
  
  cmd+=" ${static_params} -vf hwupload_cuda,scale_npp=w=${widthParam}:h=${heightParam}"
  cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
  cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
  
  # add rendition entry in the master playlist
  master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"

  resolutionValid=1
  prevHeight=${height}
done

if [ $resolutionValid -eq 1 ]; then
  # start conversion
  echo -e "Executing command:\n/content/ffmpeg ${misc_params} -i ${source} ${cmd}\n"
  /content/ffmpeg ${misc_params} -i ${source} ${cmd}
  # create master playlist file
  echo -e "${master_playlist}" > ${target}/playlist.m3u8
  echo "Done - encoded HLS is at ${target}/"
else
  echo "Video source is too small"
  exit 1
fi

ELAPSED_TIME=$(($SECONDS - $START_TIME))

echo "Elapsed time: ${ELAPSED_TIME}"
echo "-----FINISH GENERATING HLS STREAM-----"

Can you add AMD GPU support?

Hi @xubmuajkub,

For support with AMD GPU, you may refer to this thread, basically the solution is updating my script with the following:

• change hwaccel to auto
• chang -c:v to h264_amf

I haven't tested yet ('cuz I don't have any PC that has AMD 😅😅), but searching for awhile and most them have same solution as described above.

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

It would be good if we can have Windows .bat file

Do you have anyway to fix this issue?

[AVHWDeviceContext @ 0x7ffff3582f40] Could not initialize the CUDA driver API
[AVFilterGraph @ 0x7ffff340b4e0] Error initializing filter 'hwupload_cuda'
Error reinitializing filters!
Failed to inject frame into filter network: Unknown error occurred

Additional Information:
I use GTX 1060 on WSL on Windows 10

• Make sure you have installed nv-codec-headers (see the beginning part of my instructions)
• Note that latest version of ffmpeg may not work for all graphics card, for me I use version 3.4, you may want to choose another version (I think from 3.0-3.4 is good)

This issue is described here.

I don't have time at the moment to convert my script to .bat, you're free to create your own.

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I've got this one almost running correctly on OSX with an Radeon Pro 560X.

I made two tweaks as suggested here.
static_params="-c:a aac -ar 48000 -c:v h264_videotoolbox -profile:v main -crf 19 -sc_threshold 0"
misc_params="-hide_banner -y -hwaccel auto -vsync 0"

Now it only breaks breaks on this parameter '-vf scale=w=${widthParam}:h=${heightParam}'.

When I remove it from the command it starts rendering but of course the resulting video is in the wrong scale.

I only run this script on Ubuntu so I'm not sure it works same in other platforms.

Thanks for your report