FFmpeg cheat sheet for 360 video

ffmpeg-cheatsheet.md

FFmpeg Cheat Sheet for 360º video

Brought to you by Headjack

 
FFmpeg is one of the most powerful tools for video transcoding and manipulation, but it's fairly complex and confusing to use. That's why I decided to create this cheat sheet which shows some of the most often used commands.

 
Let's start with some basics:

• ffmpeg calls the FFmpeg application in the command line window, could also be the full path to the FFmpeg binary or .exe file
• -i is follwed by the path to the input video
• -c:v sets the video codec you want to use
  Options include libx264 for H.264, libx265 for H.265/HEVC, libvpx-vp9 for VP9, and copy if you want to preserve the video codec of the input video
• -b:v sets the video bitrate, use a number followed by M to set value in Mbit/s, or K to set value in Kbit/s
• -c:a sets the audio codec you want to use Options include aac for use in combination with H.264 and H.265/HEVC, libvorbis for VP9, and copy if you want to preserve the audio codec of the input video
• -b:a sets the audio bitrate of the output video
• -vf sets so called video filters, which allow you to apply transformations on a video like scale for changing the resolution and setdar for setting an aspect ratio
• -r sets the frame rate of the output video
• -pix_fmt sets the pixel format of the output video, required for some input files and so recommended to always use and set to yuv420p for playback
• -map allows you to specify streams inside a file
• -ss seeks to the given timestamp in the format HH:MM:SS
• -t sets the time or duration of the output

 
Get video info

ffmpeg -i input.mp4

 
Transcode video

The simplest example to transcode an input video to H.264:

ffmpeg -i input.mp4 -c:v libx264 output.mp4

However, a more reasonable example, which includes setting an audio codec, setting the pixel format and both a video and audio bitrate, would be:

ffmpeg -i input.mp4 -c:v libx264 -b:v 30M -pix_fmt yuv420p -c:a aac -b:a 192K output.mp4

To tanscode to H.265/HEVC instead, all we do is change libx264 to libx265:

ffmpeg -i input.mp4 -c:v libx265 -b:v 15M -pix_fmt yuv420p -c:a aac -b:a 192K output.mp4

iOS 11 and OSX 11 now support HEVC playback, but you have to make sure you use FFmpeg 3.4 or higher, and then add -tag:v hvc1 to your encode, or else you won't be able to play the video on your Apple device.

For VP9 we have to change both the video and the audio codec, as well as the file extension of the ouput video. We also added -threads 16 to make sure FFmpeg uses multi-threaded rendering to speed things up significantly:

ffmpeg -i input.mp4 -threads 16 -c:v libvpx-vp9 -b:v 15M -pix_fmt yuv420p -c:a libvorbis -b:a 192K output.webm

You may have noticed we also halved the video bitrate from 30M for H.264 to 15M for H.265/HEVC and VP9. This is because the latter ones are advanced codecs which output the same visual quality video at about half the bitrate of H.264. Sweet huh! They do take way longer to encode though and are not as widely supported as H.264 yet.

 
Hardware accelerated encoding

We just saw how to encode to H.264 using the libx264 codec, but the latest Zeranoe FFmpeg builds for Windows now support hardware accelerated encoding on machines with Nvidia GPUs (even older ones), which significantly speeds up the encoding process. You use this powerful feature by changing the libx264 codec to h264_nvenc:

ffmpeg -i input.mp4 -c:v h264_nvenc output.mp4

To use hardware acceleration for H.265/HEVC, use hevc_nvenc instead:

ffmpeg -i input.mp4 -c:v hevc_nvenc output.mp4

If you get any error messages, either your FFmpeg version or your GPU does not support hardware acceleration, or you are using an unsupported -pix_fmt. There is unfortunately no hardware acceleration support in FFmpeg for the VP9 codec.

We noticed one strange artefact when using h264_nvenc and hevc_nvenc in combination with scaling. For example, when we scaled a 4096x4096 video down to 3840x2160 pixels, the height of the output video showed correctly as 2160 pixels, but the stored_height was 2176 pixels for some reason, which causes issues when trying to play it back on Android 360º video players.

 
Resize video to UHD@30fps

At the moment, the most common playback resolution for 360º video is the UHD resolution of 3840x2160 at 30 frames per second. The commands we have to add for this are:

-vf scale=3840x2160,setdar=16:9 -r 30

Which results in something like this:

ffmpeg -i input.mp4 -vf scale=3840x2160,setdar=16:9 -r 30 -c:v libx265 -b:v 15M -pix_fmt yuv420p -c:a aac -b:a 192K output.mp4

 
Add, remove, extract or replace audio

Add an audio stream to a video without re-encoding:

ffmpeg -i input.mp4 -i audio.aac -c copy output.mp4

However, in most cases you will have to re-encode the audio to fit your video container:

ffmpeg -i input.mp4 -i audio.wav -c:v copy -c:a aac output.mp4

Remove an audio stream from the input video using the -an command:

ffmpeg -i input.mp4 -c:v copy -an output.mp4

Extract an audio stream from the input video using the -vn command:

ffmpeg -i input.mp4 -vn -c:a copy output.aac

Replace an audio stream in a video using the -map command:

ffmpeg -i input.mp4 -i audio.wav -map 0:0 -map 1:0 -c:v copy -c:a aac output.mp4

You could add the -shortest command to force the output video to take the length of the shortest input file if the input audio file and the input video file are not exactly the same length

 
Sequence to video

Many high-end video pipelines work with DPX, EXR or TIFF sequences. To transform these sequences into video files, the easiest way is to specify the first file in the sequence as the input and then use -framerate to set the input frame rate and -r to set the output frame rate:

ffmpeg -i input_0001.dpx -framerate 59.94 -c:v libx264 -b:v 30M -r 29.97 -an output.mp4

 
Stereo to mono

We can use video filters to cut the bottom half of a stereoscopic top-bottom video to turn it into a monoscopic video:

ffmpeg -i input.mp4 -vf crop=h=in_h/2:y=0 -c:a copy output.mp4

 
Cut a piece out of a video

Use -ss to set the start time in the video and -t to set the duration of the segment you want to cut

ffmpeg -ss 00:01:32 -i input.mp4 -c:v copy -c:a copy -t 00:00:10 output.mp4

The above command seeks to 1.32 minutes in the video, and then outputs the next 10 seconds. As you can see, -ss is placed before the -i command, which results in way faster (but slightly less accurate) seeking.

 
Concatenate two videos

Concatenation is not possible with all video formats, but it works fine for MP4 files for example. There are a couple of ways to concatenate video files, but I will only describe the way that worked for me here, which requires you to create a txt file with the paths to the files you want to concatenate.

Only if the files you want to concatenate have the exact same encoding settings can you concatenate without re-encoding:

ffmpeg -f concat -i files.txt -c copy output.mp4

In the files.txt file, place urls to the files you want to concatenate:

file '/path/to/video1.mp4'
file '/path/to/video2.mp4'
file '/path/to/video3.mp4'

You can add -safe 0 if you are using absolute paths. If you miss some frames after concatenation, keep in mind that the concatenation happens on I-frames, so if you don't cut at exactly the right frame, FFmpeg will discard all frames up to the nearest I-frame before concatenating.

Thats a great cheat sheet thank you!
I am looking for a command which would allow me to take a .360 video (2 camera sensors - GoPro Max) and split it into 2 - to get raw data (video frames or even videos) - to have data from each sensor separately. I have been looking around, tried using qscale, but so far unsuccessful.
Any advice would be highly appreciated, thanks!
MZ

Great cheat sheet, thanks!

I'm wondering, does anyone know how we can create a non-VR (i.e. flattened) thumbnail from a VR video?

Great cheat sheet, thanks!

I'm wondering, does anyone know how we can create a non-VR (i.e. flattened) thumbnail from a VR video?

Use jacobmartinez' solution to cut the video

ffmpeg -i <left_input> -i -filter_complex "[0:v]scale=iw:ih/2[top]; [1:v]scale=iw:ih/2[bottom]; [top][bottom]vstack[ou]" -map "[ou]" -c:v libx264 -b:v 30M -pix_fmt yuv420p

Then take a screenshot of it, and use fisheye (this is for diagonal):

ffmpeg -i [input] -vf v360=fisheye:equirect:ih_fov=360:iv_fov=360 [output]

Just an idea, didn't actually test it.

Figured it out, @cooperdk's comments were a great help.

I used a -vf filter of
v360=fisheye:flat:in_stereo=sbs:ih_fov=180:iv_fov=180

to flatten out a sbs 180 video.

Hey @nickkraakman (or anyone) do you know how to convert an EAC video from youtube to regular rectangular? I keep seeing people posting about using ffmpeg to do it, but no one says how. I work with with students so really new to the world of ffmpeg. Any help would be much appreciated.

Hi @sultanreflects, I assume that you want a regular (no-3D no-VR) video, here is the command:
ffmpeg -i input.mp4 -vf v360=eac:rectilinear output.mp4

He @oldclock thanks for the response. When you say no-3D no-VR you mean it won't be stereoscopic like TB or LR? And is there a way to retain the stereoscopic in the conversion? I'm trying to load 3D videos on our headsets but the schools wifi doesn't let you get on quest through the Quest so trying to put a library of history videos together that are already downloaded to the device and not streamed from youtube.

Hi @sultanreflects, it seems that you just want to watch YT 360 videos offline, right ?
The easiest way is to download them by correct command then you will get equirectangle format (which is almost supported by any VR player) instead of EAC format.
For example, by passing --user-agent "" to youtube-dl.
For more information please see the discussion here.

And one real nice generator here: http://www.rodrigopolo.com/ffmpeg/

thanks.
The tool is now here :
https://v1.rodrigopolo.com/ffmpeg/

is there a way to use FFMPEG with the new GoPro MAX? their app blows

You can use a of this forks
https://github.com/gmat/goproMax-ffmpeg-v5
https://github.com/gmat/goproMax-FFmpeg-v4
based of this work https://github.com/eltorio/FFmpeg and this https://www.trekview.org/blog/reverse-engineering-gopro-360-file-format-part-4/ (link updated)
the use of the filter should be like this :
./ffmpeg -hwaccel opencl -v verbose -filter_complex '[0:0]format=yuv420p,hwupload[a] , [0:5]format=yuv420p,hwupload[b], [a][b]gopromax_opencl, hwdownload,format=yuv420p' -i IN.360 -c:v libx264 -map_metadata 0 -map 0:3 OUT.mp4

I am using FFmpeg to convert dual fisheye 360 video recorded on a Rylo camera into equirectangular format. My goal is to stitch videos in bulk on the PC instead of using the Android app and dealing with large video files on my phone. I’ve got it stitching at an acceptable quality level. What I’m losing in the conversion is the "always upright" characteristic that is preserved when the Rylo Android app processes the same raw dual fisheye source file. I’m pretty sure the dual fisheye source .mp4 file must contain some metadata that the Android app is able to read. How can I make sure that data is processed when FFmpeg stitches that same source file?

Here's a video of the issue: https://youtu.be/l1Evob6MQvA

I posted this question in StackExchange: https://superuser.com/q/1716892/261107

Youtube - half sideways (EAC) to equirectangular

If you come across the videos where the top is right-side up and the bottom is rotated clockwise 90 degrees, and you need equirectangular, you are probably after:

ffmpeg -i input.mp4 -vf "v360=eac:e" output.mp4

I found the answer on this awesome wiki

Is there a way to change the point of view using ffmpeg within a 360 video?

Is there a way to change the point of view using ffmpeg within a 360 video?

You can do it in Premiere Pro using the built in 360 editor. https://www.youtube.com/watch?v=YeKS9cXr9Rk

Thanks, but I need to change the POV at certain times in the video using ffmpeg, programmatically.

Replace hevc_nvenc with hevc_videotoolbox for Apple Silicon Macs

I am trying to split large file (to upload Insta360 to OpenStreetMap mapillary) and I am doing ffmpeg -i input.mp4 -c copy -map 0:0 -segment_time 00:05:00 -f segment -reset_timestamps 1 out%03d.mp4 but I cannot get side data to be added (I tried -strict experimental). Anyone have idea how to just simply split 360 video into multiple (but with same projection) 360 videos with side data?

edit: managed to add it without ffmpeg, in later step using exiftool -api LargeFileSupport=1 -tagsFromFile input.mp4 -all:all out000.mp4

I'm wondering if anyone knows the right command to speedup the 360 video speed. The command below generates an invalid MP4 for some reason.

ffmpeg -i input.360 -filter_complex "[0:v]setpts=0.25*PTS[v];[0:a]atempo=4[a]" -map "[v]" -map "[a]" -c:v hevc_videotoolbox fast.mp4