This gist will show you how to launch multiple ffmpeg instances with xargs, very useful for NVIDIA NVENC based encoding where standard GPUs limit the maximum simultaneous encode sessions to two.

ffmpeg-multi-instances-xargs.md

Spawning multiple ffmpeg processes with xargs:

On standard NVIDIA GPUs (Not the Quadros and Tesla lines), NVENC encodes are limited to two simultaneous sessions. The sample below illustrates how to pass a list of AVI files to ffmpeg and encode them to HEVC on two encode sessions:

$ find Videos/ -type f -name \*.avi -print | sed 's/.avi$//' |\
  xargs -n 1 -I@ -P 2 ffmpeg -i "@.avi" -c:a aac -c:v hevc_nvenc "@.mp4"

This will find all files with the ending .avi in the directory Videos/ and transcode them into HEVC/H265+AAC files with the ending .mp4. The noteworthy part here is the -P 2 to xargs, which starts up to two processes in parallel.

You may modify the command above to specify any other file wildcards to suit your case.

Note that you can also combine the encode process with CUDA's accelerated NPP for faster image scaling on top of it. So instead of doing this:

ffmpeg -i input.avi \
      -c:a aac -b:a 128k \
      -s hd480 \
      -c:v hevc_nvenc -b:v 1024k \
      -y output.mp4

You can now replace the "-s hd480" with a (somewhat complicated) filter and do:

ffmpeg -i input.avi \
      -c:a aac -b:a 128k \
      -filter:v hwupload_cuda,scale_npp=w=852:h=480:format=nv12:interp_algo=lanczos,hwdownload,format=nv12 \
      -c:v hevc_nvenc -b:v 1024k \
      -y output.mp4

This uploads each frame to the CUDA engine where it then scales these with the lanczos algorithm to a size of 852x480 pixels (same as hd480). The format "nv12" as well as "yupv420p" can be used here and needs to be specified in order for it to work. "nv12" seems to run a bit faster than "yuv420p" but both formats are recognized by the NVENC encoder.

Now, running two ffmpeg processes in parallel and using CUDA based NPP (Nvidia Performance Primitives extensions, see this for the build guide) for scaling, you can rapidly encode video(s) using an NVENC capable GPU.

find Videos/ -type f -name \*.avi -print0 | sed 's/.avi$//' | xargs -0 -n 1 -I@ -P 2 \
   ffmpeg -i "@.avi" \
      -c:a aac -b:a 128k \
      -filter:v hwupload_cuda,scale_npp=w=852:h=480:format=nv12:interp_algo=lanczos,hwdownload,format=nv12 \
      -c:v hevc_nvenc -b:v 1024k \
      -y "@.mp4"

Of course, if you have a Tesla or Quadro GPU, it's possible to launch more than two simultaneous encode sessions, as your need may arise. Merely modify the -P $(n) argument passed to xargs, where $(n) is the number of simultaneous ffmpeg processes.

Have fun.

Hello @Tinkery,

You can reach me through email (it's on my Github profile) for further discussion.

Hello
Brainiarc7, this concept would work with live streams?

I do have a GTX1070 nvidia card.

Yes, it would.