Creates a video from sequential training progress images. Skips more and more frames as iterations increase.

render_training_video.py

from glob import glob

# Creates a video from a folder containing images of training progress
# Allows sampling the image less often as iterations increase
# (Requires images be sortable by filename for now. Could use modification date too)
def render_video(output_path="/content/taming-transformers/output.mp4", src_path="/content/taming-transformers/", file_glob="*.png", step_increase_every=100):
  tmp_path = "/tmp/latentvisions_tmp"

  files = glob(src_path+file_glob)
  files.sort()

  if len(files) == 0:
    print(f"No files found in '{src_path}{file_glob}'. Returning...")
    return

  !rm -rf $tmp_path
  !mkdir -p $tmp_path

  for i, filepath in enumerate(files):

    # this is a bit hacky. we should be adding a float to get
    # smooth increase in steps.
    save_step = i % (i//step_increase_every + 1) == 0
    
    if save_step:
      !cp -v $filepath $tmp_path
    else:
      !mv $filepath /tmp

  !ffmpeg -y -r 20 -pattern_type glob -i "{tmp_path+"/"+file_glob}" -vcodec libx264 -crf 15 -pix_fmt yuv420p {output_path} -hide_banner -loglevel error

render_video()

Thanks for this. On stylegan2 skipping steps traversing the latent where loss worsened or went the wrong direction seemed to help smooth a lot of the animation for me, but this is great for quick experiments while i'm reimplementing my VQGAN related implementation. I need to run metrics on each frame generation and record the hyperparameters as I think that basic heuristic and secondary ones for momentum can be extended into something better if I analyze or find some more pointers on how people are trying to smooth and create coherent latent animations now.

I updated it to make it render an mp4 that is understood by more players... I didn't test it for a while so let me know if there are any bugs