Creates a montage grid of frames from a video file with frame and time offset labels to help give a basic summary of the video file contents.

extract-grid-overview.py

#!/usr/bin/env python3
'''
Creates a montage grid of frames from a video file with frame and
time offset labels to help give a basic summary of the video file contents.
'''

# ----------------------------------------------------------------------------
# REQUIRES: This script requires `ffmpeg`` (including `ffprobe``) and
# imagemagick (with `convert` and `montage`) to be available on the path.
#
# WARNING: This is an extremely kludgy tool that makes a real mess but tries to
# clean up afterward. If you stop it mid-process you might have a bunch of .png
# files to clean up. This is seriously janky and will probably be unhappy with
# small files with large grid sizes.
#
# There are ABSOLUTELY better ways to do this, but it works "good enough"
# for my purposes. (You know how that goes.)
#
# 2022-06-19, Fortyseven (Network47.org)
# LICENSE: Steal me.
# ----------------------------------------------------------------------------

import os
import sys
from os.path import exists

CMD_FFPROBE_FRAME_COUNT = "ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 "
CMD_FFPROBE_FRAMERATE_COUNT = "ffprobe -v error -select_streams v:0 -of default=noprint_wrappers=1:nokey=1 -show_entries stream=r_frame_rate "
FFMPEG_SHARP = "unsharp=lx=5:ly=5:la=0.5"

DEFAULT_COLS = 8
DEFAULT_ROWS = 8
INTERFRAME_OFFSET = 30  # used to avoid the "black frame 0" problem

MONTAGE_OUTPUT_FILE = "montage.png"

# too big and `montage` will run out of memory -- yes, it seems to load each frame into memory when building... 8x8x1920x1080x4
EXTRACTED_FRAME_WIDTH = 256

LABEL_FONTSIZE = 10

# We'll either take just the input and default to 8x8, or you'll need to
# provide BOTH column and row counts. Otherwise: outta here.

if not len(sys.argv) in [2, 4, 5]:
    print(f"usage: {sys.argv[0]} video_file [col_count row_count] [outfile]")
    sys.exit(-1)

input_video_file = sys.argv[1]

grid_columns = DEFAULT_COLS
grid_rows = DEFAULT_ROWS

if len(sys.argv) >= 4:
    grid_columns = int(sys.argv[2])
    grid_rows = int(sys.argv[3])

if len(sys.argv) == 5:
    MONTAGE_OUTPUT_FILE = sys.argv[4]

try:
    # Query ffprobe for the frame count
    h = os.popen(CMD_FFPROBE_FRAME_COUNT + input_video_file)
    vid_frames_total = int(h.read().strip())

    # Query ffprobe for the framerate
    h2 = os.popen(CMD_FFPROBE_FRAMERATE_COUNT + input_video_file)
    vid_fps = round(eval(h2.read().strip()), 3)

    # Get an estimate of how long the video is, in minutes
    vid_minutes = round((vid_frames_total / vid_fps)/60, 3)
    grid_total_frames = grid_columns * grid_rows

    print(
        f"* There are {vid_frames_total} frames at {vid_fps}fps for ~{vid_minutes} mins.")

    print(
        f"* Grid of size {grid_columns}x{grid_rows} will have {grid_total_frames} frames.")

    frame_index_roster = []  # array of frame indexes into the video
    select_elements = []     # array for building the ffmpeg extraction call
    frame_data = []          # array holding the label under each frame
    label_fnames = []        # filenames for the resulting labeled frames
    # gather a list of candidate frame numbers to extract
    for grid_frame_index in range(grid_columns * grid_rows):
        frame_index_roster.append(
            round(vid_frames_total / grid_total_frames) * grid_frame_index + INTERFRAME_OFFSET)

    # builds list of frame index specifiers to pass to ffmpeg, also sets up later vars
    i = 1
    for row in range(grid_rows):
        for col in range(grid_columns):
            frame_index = frame_index_roster[grid_columns * row + col]
            top = "eq(n\\,"
            bot = top + f"{frame_index})"
            select_elements.append(bot)

            pct_done = (frame_index / vid_frames_total)
            min_offs = vid_minutes * pct_done

            label_text = f"Frame {frame_index} @ {round(min_offs,2)} min ({round(pct_done * 100,2)}%)"
            frame_data.append(label_text)

            label_fnames.append(f"montage_labeled_{i}.png")
            i += 1

    # build the ffmpeg extraction call
    vfsel = f"-vf \"select='{'+'.join(select_elements)}', scale={EXTRACTED_FRAME_WIDTH}\:-1, {FFMPEG_SHARP}\""
    cmd_extract = f"ffmpeg -hide_banner -loglevel error -i {input_video_file} {vfsel} "
    cmd_extract += f" -vsync 0 montage_frame_%d.png"

    print("* Extracting frames (this will take some time depending on size of grid)...")
    if os.system(cmd_extract):
        os._exit(-1)

    # add labels to all the exported frames
    print("* Building labeled versions...")
    for frame_id in range(grid_columns*grid_rows):
        cmd_label = f"convert montage_frame_{1+frame_id}.png -background black -fill white -pointsize {LABEL_FONTSIZE} label:\"{frame_data[frame_id]}\" -gravity center -append montage_labeled_{1+frame_id}.png"
        # print(frame_data[frame_id])
        if os.system(cmd_label):
            os._exit(-1)

    print(f"* Building montage to `{MONTAGE_OUTPUT_FILE}`...")

    cmd_montage = f"montage -density {EXTRACTED_FRAME_WIDTH} -tile {grid_columns}x{grid_rows} -geometry +0+0 -border 0 "
    cmd_montage += " ".join(label_fnames)
    cmd_montage += f" {MONTAGE_OUTPUT_FILE}"

    if os.system(cmd_montage):
        os._exit(-1)

except Exception as e:
    print("Some horrible thing happened: ")
    print(e)

finally:
    # try to delete all our temporary files
    print("* Removing temporary files...")
    for i in range(grid_columns * grid_rows):
        f = f"montage_frame_{1+i}.png"
        if exists(f):
            os.remove(f)

        f = f"montage_labeled_{1+i}.png"
        if exists(f):
            os.remove(f)

I had a couple of issues running it--
First, it seems to have issues with filenames containing spaces. Even if I surround the filename with quotes it interprets it as multiple arguments, In a test run, I got the error "Argument 'tpy.mp4' provided as input filename, but 'tpy' was already specified."
Second, I then renamed the file so it didn't contain spaces, but in my run I got the error "Invalid parameter - -background" after the Building labeled versions step. I'm using Windows Powershell and the current versions of both this code and ffmpeg as of 8/15/2022.

I believe I noticed similar issues now and then, come to think of it. I'll migrate this over to use argparse for argument handling sometime in the next day or so. Maybe I should make a proper repo while I'm at it... either way, I'll ping you with an update when that happens. 👍

Gotcha, thanks much! I can see using this script frequently to make thumbnails when I link to videos on our gaming blog.

Sweet.

I went the repo route and migrated over to argparse. Seemed to work alright while I was generating example images for the readme, so: 🤞. :)

Usage should go like extract-grid-overview.py -c 12 -r 12 input.mp4 output.png, and of course -h for a reminder. I'll probably add flags for other settings at some point (like the label styles, etc).

https://github.com/Fortyseven/extract-grid-overview

Direct link to the updated .py -> https://raw.githubusercontent.com/Fortyseven/extract-grid-overview/master/extract-grid-overview.py