sdv-SD2-1-example.ipynb

sdv-sd2-1-example.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPn/ls1tZkFLvKoDOMLPVyF",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "gpuClass": "premium"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/nateraw/abaf9f6359ca90e3c95f2e1006011bd6/sdv-sd2-1-example.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "! pip install git+https://github.com/nateraw/stable-diffusion-videos"
      ],
      "metadata": {
        "id": "rTl3OkX88Or0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ru1c_UQ88Lvh"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "\n",
        "from stable_diffusion_videos import StableDiffusionWalkPipeline\n",
        "from diffusers import DPMSolverMultistepScheduler\n",
        "\n",
        "device = \"mps\" if torch.backends.mps.is_available() else \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "torch_dtype = torch.float16 if device == \"cuda\" else torch.float32\n",
        "pipe = StableDiffusionWalkPipeline.from_pretrained(\n",
        "    \"stabilityai/stable-diffusion-2-1\",\n",
        "    torch_dtype=torch_dtype,\n",
        ").to(device)\n",
        "pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pipe.walk(\n",
        "    prompts=['a cat', 'a dog'],\n",
        "    seeds=[1234, 4321],\n",
        "    num_interpolation_steps=5,\n",
        "    num_inference_steps=50,\n",
        "    fps=5,\n",
        ")"
      ],
      "metadata": {
        "id": "o_C6Cn2n8P_B"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}

Love this!
Question how would I make my video's longer?

Kind regards!!

Hey there, glad ya like it :)

There are a number of ways to make the video longer. Your video is generated with num_frames = num_interpolation_steps * (len(prompts) - 1) frames, and the video duration is num_frames / fps seconds long.

So...

• can add more prompts/seeds
• can increase num_interpolation steps
• can decrease FPS (this is really just making the video slower but longer in duration as its same number of frames

For example, the following would produce num_frames = 30 * 4 (120) frames at 30fps, so 4 seconds long with decent quality.

prompts=['a cat', 'a dog', 'a bird', 'a horse', 'a camel']
seeds = [0, 1, 2, 3, 4]  # Set to anything you want, or random numbers. must be same len as prompts
pipe.walk(prompts=prompts, seeds=seeds, num_interpolation_steps=30, num_inference_steps=50, fps=30)

Super clear information! I'm just not completely sure I understand the concept of the seeds. I need x amount of seeds for x amount of prompts in one pipe.walk, correct? But what are the random numbers? Kind regards!

…

On Tue, 21 Feb 2023 at 19:35, Nathan Raw ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Hey there, glad ya like it :) There are a number of ways to make the video longer. Your video is generated with num_frames = num_interpolation_steps * (len(prompts) - 1) frames, and the video duration is num_frames / fps seconds long. So... - can add more prompts/seeds - can increase num_interpolation steps - can decrease FPS (this is really just making the video slower but longer in duration as its same number of frames For example, the following would produce num_frames = 30 * 4 (120) frames at 30fps, so 4 seconds long with decent quality. prompts=['a cat', 'a dog', 'a bird', 'a horse', 'a camel']seeds = [0, 1, 2, 3, 4] # Set to anything you want, or random numbers. must be same len as promptspipe.walk(prompts=prompts, seeds=seeds, num_interpolation_steps=30, num_inference_steps=50, fps=30) — Reply to this email directly, view it on GitHub <https://gist.github.com/abaf9f6359ca90e3c95f2e1006011bd6#gistcomment-4478529> or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOARUOTE3YELLF3KFPNF2LWYUDGRBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTEMBZHAZTAMRSU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

-- www.dewildequinten.be