Creates a video from sequential training progress images. Skips more and more frames as iterations increase.

render_training_video.py

from glob import glob

# Creates a video from a folder containing images of training progress
# Allows sampling the image less often as iterations increase
# (Requires images be sortable by filename for now. Could use modification date too)
def render_video(output_path="/content/taming-transformers/output.mp4", src_path="/content/taming-transformers/", file_glob="*.png", step_increase_every=100):
  tmp_path = "/tmp/latentvisions_tmp"

  files = glob(src_path+file_glob)
  files.sort()

  if len(files) == 0:
    print(f"No files found in '{src_path}{file_glob}'. Returning...")
    return

  !rm -rf $tmp_path
  !mkdir -p $tmp_path

  for i, filepath in enumerate(files):

    # this is a bit hacky. we should be adding a float to get
    # smooth increase in steps.
    save_step = i % (i//step_increase_every + 1) == 0
    
    if save_step:
      !cp -v $filepath $tmp_path
    else:
      !mv $filepath /tmp

  !ffmpeg -y -r 20 -pattern_type glob -i "{tmp_path+"/"+file_glob}" -vcodec libx264 -crf 15 -pix_fmt yuv420p {output_path} -hide_banner -loglevel error

render_video()

Important: For this to make sense one should save an image on every training iteration

Thanks for this. On stylegan2 skipping steps traversing the latent where loss worsened or went the wrong direction seemed to help smooth a lot of the animation for me, but this is great for quick experiments while i'm reimplementing my VQGAN related implementation. I need to run metrics on each frame generation and record the hyperparameters as I think that basic heuristic and secondary ones for momentum can be extended into something better if I analyze or find some more pointers on how people are trying to smooth and create coherent latent animations now.

I updated it to make it render an mp4 that is understood by more players... I didn't test it for a while so let me know if there are any bugs