bash scripts to create VOD HLS stream with ffmpeg almighty (tested on Linux and OS X)

README.md

running:

bash create-vod-hls.sh beach.mkv

will produce:

    beach/
      |- playlist.m3u8
      |- 360p.m3u8
      |- 360p_001.ts
      |- 360p_002.ts
      |- 480p.m3u8
      |- 480p_001.ts
      |- 480p_002.ts
      |- 720p.m3u8
      |- 720p_001.ts
      |- 720p_002.ts
      |- 1080p.m3u8
      |- 1080p_001.ts
      |- 1080p_002.ts  

origin: http://docs.peer5.com/guides/production-ready-hls-vod/

create-vod-hls.sh

#!/usr/bin/env bash

set -e

# Usage create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]
[[ ! "${1}" ]] && echo "Usage: create-vod-hls.sh SOURCE_FILE [OUTPUT_NAME]" && exit 1

# comment/add lines here to control which renditions would be created
renditions=(
# resolution  bitrate  audio-rate
#  "426x240    400k    64k"
  "640x360    800k     96k"
  "842x480    1400k    128k"
  "1280x720   2800k    128k"
  "1920x1080  5000k    192k"
)

segment_target_duration=4       # try to create a new segment every X seconds
max_bitrate_ratio=1.07          # maximum accepted bitrate fluctuations
rate_monitor_buffer_ratio=1.5   # maximum buffer size between bitrate conformance checks

#########################################################################

source="${1}"
target="${2}"
if [[ ! "${target}" ]]; then
  target="${source##*/}" # leave only last component of path
  target="${target%.*}"  # strip extension
fi
mkdir -p ${target}


key_frames_interval="$(echo `ffprobe ${source} 2>&1 | grep -oE '[[:digit:]]+(.[[:digit:]]+)? fps' | grep -oE '[[:digit:]]+(.[[:digit:]]+)?'`*2 | bc || echo '')"
key_frames_interval=${key_frames_interval:-50}
key_frames_interval=$(echo `printf "%.1f\n" $(bc -l <<<"$key_frames_interval/10")`*10 | bc) # round
key_frames_interval=${key_frames_interval%.*} # truncate to integer

# static parameters that are similar for all renditions
static_params="-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0"
static_params+=" -g ${key_frames_interval} -keyint_min ${key_frames_interval} -hls_time ${segment_target_duration}"
static_params+=" -hls_playlist_type vod"

# misc params
misc_params="-hide_banner -y"

master_playlist="#EXTM3U
#EXT-X-VERSION:3
"
cmd=""
for rendition in "${renditions[@]}"; do
  # drop extraneous spaces
  rendition="${rendition/[[:space:]]+/ }"

  # rendition fields
  resolution="$(echo ${rendition} | cut -d ' ' -f 1)"
  bitrate="$(echo ${rendition} | cut -d ' ' -f 2)"
  audiorate="$(echo ${rendition} | cut -d ' ' -f 3)"

  # calculated fields
  width="$(echo ${resolution} | grep -oE '^[[:digit:]]+')"
  height="$(echo ${resolution} | grep -oE '[[:digit:]]+$')"
  maxrate="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${max_bitrate_ratio}" | bc)"
  bufsize="$(echo "`echo ${bitrate} | grep -oE '[[:digit:]]+'`*${rate_monitor_buffer_ratio}" | bc)"
  bandwidth="$(echo ${bitrate} | grep -oE '[[:digit:]]+')000"
  name="${height}p"
  
  cmd+=" ${static_params} -vf scale=w=${width}:h=${height}:force_original_aspect_ratio=decrease"
  cmd+=" -b:v ${bitrate} -maxrate ${maxrate%.*}k -bufsize ${bufsize%.*}k -b:a ${audiorate}"
  cmd+=" -hls_segment_filename ${target}/${name}_%03d.ts ${target}/${name}.m3u8"
  
  # add rendition entry in the master playlist
  master_playlist+="#EXT-X-STREAM-INF:BANDWIDTH=${bandwidth},RESOLUTION=${resolution}\n${name}.m3u8\n"
done

# start conversion
echo -e "Executing command:\nffmpeg ${misc_params} -i ${source} ${cmd}"
ffmpeg ${misc_params} -i ${source} ${cmd}

# create master playlist file
echo -e "${master_playlist}" > ${target}/playlist.m3u8

echo "Done - encoded HLS is at ${target}/"

when i try to convert the video it gives me error
bash create-vod-hls.sh vid.mp4 Executing command: ffmpeg -hide_banner -y -i vid.mp4 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=640:h=360:force_original_aspect_ratio=decrease -b:v 800k -maxrate 856k -bufsize 1200k -b:a 96k -hls_segment_filename vid/360p_%03d.ts vid/360p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=842:h=480:force_original_aspect_ratio=decrease -b:v 1400k -maxrate 1498k -bufsize 2100k -b:a 128k -hls_segment_filename vid/480p_%03d.ts vid/480p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k -hls_segment_filename vid/720p_%03d.ts vid/720p.m3u8 -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 60 -keyint_min 60 -hls_time 4 -hls_playlist_type vod -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease -b:v 5000k -maxrate 5350k -bufsize 7500k -b:a 192k -hls_segment_filename vid/1080p_%03d.ts vid/1080p.m3u8 Unrecognized option 'hls_playlist_type'. Error splitting the argument list: Option not found

Thanks for the script. But when I run this, here's my error message.

./create-vod-hls.sh 022505043294641312470435.mp4

[libx264 @ 0x55a878bcb800] [Eval @ 0x7fff9d53be00] Undefined constant or missing '(' in 'keyint_min'
[libx264 @ 0x55a878bcb800] Unable to parse option value "-keyint_min"
[libx264 @ 0x55a878bcb800] Error setting option g to value -keyint_min.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a878bca840] Qavg: 119.779
[aac @ 0x55a878bca840] 2 frames left in the queue on closing
[aac @ 0x55a878b9c080] Qavg: 119.779
[aac @ 0x55a878b9c080] 2 frames left in the queue on closing
[aac @ 0x55a878bcf800] Qavg: 119.779
[aac @ 0x55a878bcf800] 2 frames left in the queue on closing
[aac @ 0x55a878ba8e40] Qavg: 119.779
[aac @ 0x55a878ba8e40] 2 frames left in the queue on closing

did you solve this? I have the same error

hi, is there any way to include codec info as well in manifest?
Currently it is like:

#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080```

What i want is :

#EXT-X-STREAM-INF:BANDWIDTH=767744,AVERAGE-BANDWIDTH=720351,CODECS="avc1.4d401f,mp4a.40.5",RESOLUTION=640x480,FRAME-RATE=29.970

Is there a way to get CODECS in the manifest? If I use the following command:

ffmpeg -y -i ~/work/sample-videos/flowers.mp4  \
  -preset slow -g 48 -sc_threshold 0 \
  -map 0:0 -map 0:1 -map 0:0 -map 0:1 \
  -s:v:0 640x360 -c:v:0 libx264 -b:v:0 365k \
  -s:v:1 960x540 -c:v:1 libx264 -b:v:1 2000k  \
  -c:a copy \
  -var_stream_map "v:0,a:0 v:1,a:1" \
  -master_pl_name master.m3u8 \
  -f hls -hls_time 6 -hls_list_size 0 \
  -hls_segment_filename "v%v/fileSequence%d.ts" \
  v%v/prog_index.m3u8

CODECS are auto included for h264(for above command) but the above command does not support resolution based naming of segments.

Thanks

I tried to get all the segments for 10 seconds except for the first one which will be of 5 seconds by passing-hls_init_time 5 -hls_time 10 -hls_playlist_type VOD but seems like hls_time is being ignored all together, i.e. all the segments are of 5 seconds instead of 10. Am I doing something wrong? Can someone help?

Hi @mrbar42. Thank you for great script! Why resolution is 842x480 but by standard it is 854x480? from here https://support.google.com/youtube/answer/6375112?co=GENIE.Platform%3DDesktop&hl=en

Hi @mrbar42. Thank you for great script! Why resolution is 842x480 but by standard it is 854x480? from here https://support.google.com/youtube/answer/6375112?co=GENIE.Platform%3DDesktop&hl=en

There are many iterations of 480p; use the one relevant to your most prevalent client device.
https://en.wikipedia.org/wiki/480p#Resolutions

Thanks for the script. But when I run this, here's my error message.
./create-vod-hls.sh 022505043294641312470435.mp4
[libx264 @ 0x55a878bcb800] [Eval @ 0x7fff9d53be00] Undefined constant or missing '(' in 'keyint_min'
[libx264 @ 0x55a878bcb800] Unable to parse option value "-keyint_min"
[libx264 @ 0x55a878bcb800] Error setting option g to value -keyint_min.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x55a878bca840] Qavg: 119.779
[aac @ 0x55a878bca840] 2 frames left in the queue on closing
[aac @ 0x55a878b9c080] Qavg: 119.779
[aac @ 0x55a878b9c080] 2 frames left in the queue on closing
[aac @ 0x55a878bcf800] Qavg: 119.779
[aac @ 0x55a878bcf800] 2 frames left in the queue on closing
[aac @ 0x55a878ba8e40] Qavg: 119.779
[aac @ 0x55a878ba8e40] 2 frames left in the queue on closing

did you solve this? I have the same error

I change L:35 to awk

key_frames_interval="$(awk  "BEGIN { rounded = sprintf(\"%.0f\", ${key_frames_interval}); print rounded }")"

check this single comment multi output => https://github.com/masterofthesith/ffmpeg

How about having differntial segments accoding to the duration of the original video. For example, having shorter segments for the first 10% duration of video and then longer segments for the rest 90%?

I was looking for a code to convert video files in a directory where it has multiple subfolders for each video file. I want to run a batch code so it can convert the video files and save them where the original file was there.

Example:

: tree movies/
movies/
├── subfolder1
│ └── movie1.mp4
├── subfolder2
│ └── movie2.mp4
└── subfolder3
└── movie3.mp4

And so on.

I want the output to be the same as my main structure but in .m3u8 format.

I have used a code where it can convert each movie manually but for some reason, I can use the .m3u8 file or link only in that server I can't make it work all over the internet.

for i in .mp4; do
ffmpeg -i "$i" -c copy -map 0 -hls_time 3 -hls_list_size 0 -f hls "${i%.}.hls"
done

Thanks

@DravitLochan i am also thinking like you to initial hls video size size is small like first few are divided in 2 sec chunk and then after 10 sec we make it to 5 sec chunk.
so have you achieved this solution ?
so something we can do 5 chunk are in increasing order like 1s,2s,3s,4s,5s and then all remain are 5s/10sec chunk

Isn't it easier to create a master playlist file using FFMPEG itself?
Using -master_pl_name param

Ref: https://stackoverflow.com/a/71985380/6513289

Great script to get started, doesn't play in Safari though, might be the codec this script is using.