Some snippets you can quickly adapt for use with FFmpeg and GNU Parallel for use for standard tasks.

ffmpeg-gnu-parallel-snippets.md

Useful Examples of ffmpeg and GNU parallel on the command-line:

Transcoding FLAC music to Opus:

ffmpeg is a highly useful application for converting music and videos. However, audio transcoding is limited to a a single core. If you have a large FLAC archive and you wanted to compress it into the efficient Opus codec, it would take forever with the fastest processor to complete, unless you were to take advantage of all cores in your CPU.

parallel 'ffmpeg -v 0 -i "{}" -c:a libopus -b:a 128k "{.}.opus"' ::: $(find -type f -name '*.flac')

Transcoding Videos to VP9:

libvp9 has one glaring flaw in regards to encoding: it can only use about three cores at any given point in time. If you have an eight core processor and a dozen or more episodes of a TV series to transcode, you can use the parallel program to run several encode jobs on the host at the same time, provided you also have enough memory for that.

vp9_params="-c:v libvpx-vp9 -tile-columns 6 -frame-parallel 1 -rc_lookahead 25 -threads 4 -speed 1 -b:v 0 -crf 18"
opus_params="-c:a libopus -b:a 128k"
parallel -j 4 'ffmpeg -v 0 -i "{}" $vp9_params $opus_params -f webm "{.}.webm"' ::: $(find -type f -name '*.mkv')

Converting to H.264 with ffmpeg's VAAPI-based encoder:

Unlike NVENC, ffmpeg's VAAPI-based encoder is not limited to a maximum number of maximum simultaneous encodes based on the GPU's SKUs (As is the case at the present, consumer-grade NVIDIA GPUs can only do 2 simultaneous encodes whereas the Quadro and Tesla series have an unlimited number, dependent on available resources). In this case, you can take advantage of VAAPI as shown with GNU Parallel: (Note that you don't need to use -j$(nproc) here as its' the default if -j is undefined)

vaapi_params="-hwaccel vaapi -vaapi_device /dev/dri/renderD128 -c:v h264_vaapi -qp 19 -bf 2 -vf 'format=nv12,hwupload'"
opus_params="-c:a libopus -b:a 128k"
parallel -j 4 'ffmpeg -v 0 -i "{}" $vaapi_params $opus_params -f mkv "{.}.mkv"' ::: $(find -type f -name '*.mp4')

Converting to H.264 with FFmpeg's NVENC encoders:

The example below will encode all videos in the current directory from MKV to MP4 by launching two concurrent processes such that it will work on consumer-grade Nvidia GPUs that are limited to two simultaneous NVENC encode sessions only:

Note that the example below re-encodes a bunch of MKV files to MP4's with NPP scaling enabled and with high quality Dolby Digital surround sound with Dolby Pro-Logic II down-mixing enabled, with extensions for Dolby Pro-logic IIz and the Dolby headphone profile mode enabled. Adjust as needed. (My original files were encoded in DTS, with 5.1 channel mapping layout, and their audio tracks were massive, had to re-encode the audio stream to AC3 to cut down on the size).

parallel -j 2 --verbose 'ffmpeg -loglevel debug -i "{}" -c:a ac3 -b:a 448k -ac 6 -dsur_mode on -dmix_mode dplii -dsurex_mode dpliiz -dheadphone_mode on -copyright 1 \
  -filter:v hwupload_cuda,scale_npp=w=1920:h=800:format=nv12:interp_algo=lanczos,hwdownload,format=nv12 \
  -c:v h264_nvenc -preset:v slow -profile:v high -b:v 2250k -rc:v vbr_2pass -rc-lookahead:v 40 -surfaces 40 -bf 4 -refs 4 \
  -f mp4 "{.}.mp4"' ::: $(find -type f -name '*.mkv')

What of two-pass encodes based on the example above?

parallel -j 2 --verbose 'ffmpeg -loglevel debug -threads 4 -i "{}" -pass 1 -c:a ac3 -b:a 448k -ac 6 -dsur_mode on -dmix_mode dplii -dsurex_mode dpliiz -dheadphone_mode on -copyright 1 \
      -c:v h264_nvenc -preset:v slow -profile:v high -b:v 2250k -2pass 1 -rc:v vbr_2pass -rc-lookahead:v 40 -surfaces 40 -bf 4 -refs 4 \
      -f mp4 -y "/dev/null" && ffmpeg -loglevel debug -threads 4 -i "{}" -pass 2 -c:a ac3 -b:a 448k -ac 6 -dsur_mode on -dmix_mode dplii -dsurex_mode dpliiz -dheadphone_mode on -copyright 1 \
      -c:v h264_nvenc -preset:v slow -profile:v high -b:v 2250k -2pass 1 -rc:v vbr_2pass -rc-lookahead:v 40 -surfaces 40 -bf 4 -refs 4 -f mp4 -y "{.}.mp4"' ::: $(find -type f -name '*.avi')

The example above re-encodes a bunch of AVI files to MP4's with two-pass encode mode and with high quality Dolby Digital surround sound with Dolby Pro-Logic II down-mixing enabled, with extensions for Dolby Pro-logic IIz and the Dolby headphone profile mode enabled. Adjust as needed. (Original files were encoded in DTS, with 5.1 channel mapping layout).

Adjust the parallel -j value to a higher number if you own a Quadro or Tesla GPU, as these SKUs don't have the imposed limit mentioned above.

Dear friend,

Why do you prefer to use parallel in video encoder side as you can run multiple ffmpeg at the same time ? Do GNU parallel use more effective threading ?

Hello there,

For some codecs (namely the software based VP9 encoder), their threading is somewhat limited.
Sure, you could run multiple FFmpeg sessions for the same task, however, on clusters with shared resources, I find GNU Parallel to be better at resource constraints than relying on ffmpeg's -threads option.

The construct: parallel ... ::: $(find ...) limits the number of arguments to what can fit on a single line. Instead use find ... | parallel ... (or find ... -print0 | parallel -0 ... in case you have file names containing \n). This way you will not be limited by the number of arguments.

{} is automatically quoted by GNU Parallel and should not be quoted, because quoting it again can lead to problems:
parallel "touch '{}'" ::: don\'t fails, whereas parallel "touch {}" ::: don\'t works. Unless you use special shell characters (which you don't in your examples) you can even write: parallel touch {} ::: don\'t

Thanks for the insight, @ole-tange.

Is it OK if I reach out to you for more on GNU Parallel? I've found it to be incredibly useful, but daunting to use.

Thanks to Ole for figuring the problem with the snippet, removing "s fixed it.

I am currently running an encode using a adjusted version of your snippet for parallel 2 pass encoding, but I'm seeing very strange behavior.
Instead of running 3 Workers that work on one command until its done it seems like all 12 are running and just getting put to sleep and woken up again in short succession (at least that's what htop is telling me). This is also backed up by a massive increase in both RAM usage as well as disk IO compared to running three enoders in parallel the manual way (by using screen and commands containing the individual filenames).
This is the exact command i use:
parallel -j 3 'ffmpeg -i "{}" -c:v libaom-av1 -pix_fmt yuv420p -cpu-used 1 -b:v 600k -tune psnr -an -pass 1 -passlogfile "{}.log" -f webm -y "/dev/null" && ffmpeg -i "{}" -c:v libaom-av1 -pix_fmt yuv420p -cpu-used 1 -b:v 600k -tune psnr -an -pass 2 -passlogfile "{}.log" "{.}.webm"' ::: $(find -type f -name '*.avi')

Is this expected behavior or am i missing something? From what i understand parallel shouldn't run all possible workers at once and just altering between sleep and active process states right?

ole-tange, being the motherfuckin' man as usual.