create stylegan2 psi-translation video

stylegan2_psi_translation_video.md

Make psi-translation videos

Short file on how to make psi-transition videos with StyleGan2. This assumes you are already able to run StyleGan2.

Edit generator

Edit generate_images() in run_generator.py to iterate through psi values instead of seeds.

Replace the for-loop with the following code (primarily only the loop iterator is changed):

    seed = 12345
    start = 0
    end = 2.01
    step = '0.01'
    for psi in list(drange(start, end, step)):
        print('Generating psi = %f image for seed %d ...' % (psi, seed))
        Gs_kwargs.truncation_psi = psi
        rnd = np.random.RandomState(seed)
        z = rnd.randn(1, *Gs.input_shape[1:]) # [minibatch, component]
        tflib.set_vars({var: rnd.randn(*var.shape.as_list()) for var in noise_vars}) # [height, width]
        images = Gs.run(z, None, **Gs_kwargs) # [minibatch, height, width, channel]
        PIL.Image.fromarray(images[0], 'RGB').save(dnnlib.make_run_dir_path('psi_%f_seed%04d.png' % (psi, seed)))

Add this function to help us iterate through floats:

import decimal
def drange(x, y, jump):
      while x < y:
          yield float(x)
          x += decimal.Decimal(jump)

You may edit the values start, end, and step according to your preferences and sample. Most samples will look the best between psi values of 0.3 and 1.5

This code will run slowly as it is performing only one inference per batch. This could be changed in theory but I have not attempted it, as generally a psi value is constant throughout a batch.

Produce video with ffmpeg

All of your images may be combined with this ffmpeg command:

cat $(ls ./results/input_dir/*.png | sort --numeric-sort) | ffmpeg -framerate 25 -y -i - -r 25 -c:v libx264 -pix_fmt yuv420p -crf 33 -preset veryslow -tune animation output.mp4

Set input_dir and optionally modify the framerate via -framerate and -r

Format ffmpeg video

Lastly, you may want to format your video to have nicer transitions:

1. To have the video 'pause' at a given psi value, I simply copied the image many times, which is a lazy but effective way to attain this (e.g. copy the psi1.0 image 24 times, and then it will 'pause' at that image for 1 second as it is the same frame 25 times in a row with an fps of 25)

2. To reverse the video or otherwises mix/match/combine them, use the following commands:

   To reverse a video:

   ffmpeg -i input.mp4 -vf reverse output_reversed.mp4
   

   To concatenate several files together (for example, a normal+reversed video):

   ffmpeg -f concat -safe 0 -i list.txt -c copy video_concat.mp4
   

   where list.txt has one file per line in the format file 'file_name.mp4'

   Lastly, you may find it useful to crop sections/durations from your videos:

   ffmpeg -ss starting_time -i test_reverse.mp4 -t output_duation crop.mp4
   

   Where starting_time is the starting time stamp and output_duration (optional) is the duration of your output

   You could also run waifu2x on images prior to producing a video in order to get a much nicer 1024x1024 output video.