Stable Diffusion on AMD GPUs on Windows using DirectML

Stable_Diffusion.md

Stable Diffusion for AMD GPUs on Windows using DirectML

UPDATE: A faster (20x) approach for running Stable Diffusion using MLIR/Vulkan/IREE is available on Windows:

https://github.com/nod-ai/SHARK/blob/main/shark/examples/shark_inference/stable_diffusion/stable_diffusion_amd.md

Install 🤗 diffusers

conda create --name sd39 python=3.9 -y
conda activate sd39
pip install diffusers==0.3.0
pip install transformers
pip install onnxruntime
pip install onnx

Install DirectML latest release

You can download the nightly onnxruntime-directml release from the link below

• https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-directml/overview/1.13.0.dev20220908001

Run python --version to find out, which whl file to download.

• If you are on Python3.7, download the file that ends with **-cp37-cp37m-win_amd64.whl.
• If you are on Python3.8, download the file that ends with **-cp38-cp38m-win_amd64.whl
• and likewise

pip install ort_nightly_directml-1.13.0.dev20220908001-cp39-cp39-win_amd64.whl --force-reinstall

Convert Stable Diffusion model to ONNX format

This apporach is faster than downloading the onnx models files.

• Download diffusers/scripts/convert_stable_diffusion_checkpoint_to_onnx.py to your working directory. You can try the command below to download the script.

wget https://raw.githubusercontent.com/huggingface/diffusers/main/scripts/convert_stable_diffusion_checkpoint_to_onnx.py

• Run huggingface-cli.exe login and provide huggingface access token.
• Convert the model using the command below. Models are stored in stable_diffusion_onnx folder.

python convert_stable_diffusion_checkpoint_to_onnx.py --model_path="CompVis/stable-diffusion-v1-4" --output_path="./stable_diffusion_onnx"

Run Stable Diffusion on AMD GPUs

Here is an example python code for stable diffusion pipeline using huggingface diffusers.

from diffusers import StableDiffusionOnnxPipeline
pipe = StableDiffusionOnnxPipeline.from_pretrained("./stable_diffusion_onnx", provider="DmlExecutionProvider")
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0] 
image.save("astronaut_rides_horse.png")

Question @harishanand95 , is there a way yo use the Image to Image (img2img) feature with this AMD method?

I think for now there is no way to use img2img with AMD, I hope soon we can use it. Also if I'm wrong I want to know too :)

Ok thanks, someone said it is possible in Windows something about Torsh..
I have ANOTHER QUESTION:

What line of code should I add to get the SEED of an image I am generating?

Thanks

@Stable777 @harishanand95 to use a seed it needs to be turned into latents using torch.randn before the pipe is called to generate the image.

import torch
import numpy as np
from diffusers import StableDiffusionOnnxPipeline
pipe = StableDiffusionOnnxPipeline.from_pretrained("./stable_diffusion_onnx", provider="DmlExecutionProvider", torch_dtype=torch.float16,)
prompt = "a photo of an astronaut riding a horse on mars"
height = 512
width = 512
num_inference_steps = 75
guidance_scale = 7.5
eta = 0.0
seed = 239571688563800
generator = torch.Generator()
#seed = generator.seed()
generator = generator.manual_seed(seed)
latents = torch.randn((1, 4, height // 8, width // 8),generator = generator)
image = pipe(prompt, height, width, num_inference_steps, guidance_scale, eta, latents = latents, execution_provider="DmlExecutionProvider").images[0]
image.save("astronaut_rides_horse.png")

Prompt: a photo of an astronaut riding a horse on mars - Seed: 239571688563800

Below is an example script for generating an image using a random seed + some logging and getting the prompt via console user input.

Stable Diffusion Onnx DirectML Text to Img:

• User is prompted in console for "Prompt Text"
• Image is generated using a random seed + Prompt Text
• Date/Time, Prompt Text, Seed & Completion Time is logged in a Txt File "prompts.txt"
• Image is saved, named date-time.png (date-time = time image generation was started)
• User is asked for another prompt or q to quit.

import os
import sys
import time
import torch
import numpy as np
from diffusers import StableDiffusionOnnxPipeline

height = 512
width = 512
num_inference_steps = 75
guidance_scale = 7.5
eta = 0.0
prompt=""
variations=""
pipe = StableDiffusionOnnxPipeline.from_pretrained("./stable_diffusion_onnx", provider="DmlExecutionProvider", torch_dtype=torch.float16,)

def txt_to_img(prompt):
    generator = torch.Generator()
    seed = generator.seed()
    generator = generator.manual_seed(seed)
    latents = torch.randn(
        (1, 4, height // 8, width // 8),
        generator = generator
    )
    gen_time = time.strftime("%m%d%Y-%H%M%S")
    start_time = time.time()
    image = pipe(prompt, height, width, num_inference_steps, guidance_scale, eta, latents = latents, execution_provider="DmlExecutionProvider").images[0] 
    image.save("./" + gen_time + ".png")
    log_info = "\n" + gen_time + " - Prompt: " + prompt + " - Seed: " + str(seed) + " (" + str(time.time() - start_time) + "s)"
    with open('./prompts.txt', 'a+', encoding="utf-8") as f:
        f.write(log_info)
    image = None

os.system('cls')
print('Stable Diffusion Onnx DirectML\nText to Img\n')
while prompt != "q":
    while prompt == "":
        prompt = input('Please Enter Prompt (or q to quit): ')
    if prompt != "q":
        while variations == "":
            variations = input('How many image variations?: ')
            if variations.isnumeric() == False:
                variations = ""
        for i in range(int(variations)):
            txt_to_img(prompt)
        prompt = ""
        variations = ""
pipe = None
os.system('cls')
sys.exit("Quit Called, Script Ended")

Prompt: a photo of an astronaut riding a horse on mars - Seed: 239155044343300

Are negative prompts possible at all with this?

The Stable Diffusion 1.5 weights and Onnx Files have been released. You will need to go to: https://huggingface.co/runwayml/stable-diffusion-v1-5 and review and accept the usage/download agreement.

You can download the SD v1.5 Onnx files using the following command
git clone https://huggingface.co/runwayml/stable-diffusion-v1-5 --branch onnx --single-branch stable_diffusion_onnx

I've put up some updated instructions for the install process:
https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674

Are negative prompts possible at all with this?

Yes

from diffusers import StableDiffusionOnnxPipeline
height=512
width=512
num_inference_steps=50
guidance_scale=7.5
eta=0.0
prompt = "a photo of an astronaut riding a horse on mars"
negative_prompt="purple"
pipe = StableDiffusionOnnxPipeline.from_pretrained("./stable_diffusion_onnx", provider="DmlExecutionProvider")
mage = pipe(prompt, height, width, num_inference_steps, guidance_scale, negative_prompt, eta).images[0] 
image.save("astronaut_rides_horse.png")

@harishanand95 looks like the process works up to diffusers==0.5.0 after that StableDiffusionOnnxPipeline is changed to OnnxStableDiffusionPipeline

I think for now there is no way to use img2img with AMD, I hope soon we can use it. Also if I'm wrong I want to know too :)

@lordzerg @Stable777 An Onnx Img2Img Pipeline has been added in Diffusers 0.6.0
huggingface/diffusers#552
https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion

change scheduler. There is astype(np.int64) in scheduling_pndm.py line 168 but not in other schedulers That's why change to PNDMScheduler can fix this. Or modify the other schedulers yourself. have a look here for a more verbose explanation: https://www.travelneil.com/stable-diffusion-updates.html#the-first-thing

If anyone is wondering how to change to PNDMScheduler for your specific model that is not working (Such as the trinart or wifu models). Open the model_index.json file (Located in the model folder you are trying to use) and edit the scheduler option.

I'm having a bit of an issue trying to convert the model. Every time I try to run the command under python 3.10.8 it fails referring to line 24 of the .py file. Anything that I could be doing wrong?

i'm now trying to convert other models i've already downloaded and the conversion script is yelling at me about invalid repo id's. but i'm not trying to use a repo? does anyone know how to point the convert_stable_diffusion_checkpoint_to_onnx.py at a downloaded model?

Unfortunately I don't have time to update the instructions, please follow @averad 's instructions for diffusers>=0.6.0 Thanks! https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674

Unfortunately I don't have time to update the instructions, please follow @averad 's instructions for diffusers>=0.6.0 Thanks! https://gist.github.com/averad/256c507baa3dcc9464203dc14610d674

Thank you @harishanand95 for all you and your team at AMD are doing!

FYI, @harishanand95 is documenting how to use IREE (https://iree-org.github.io/iree/) through the Vulkan API to run StableDiffusion text->image. We expect to release the instructions next week. In our tests, this alternative toolchain runs >10X faster than ONNX RT->DirectML for text->image, and Nod.ai is also working to support img->img soon... we think the performance difference is in part explained by MLIR and IREE being a compiler toolchain, compared to ORT that's more of an interpreter. If you're interested in learning more and supporting this new code path, please email me at claforte at my employer's domain, or send me a Discord friend invite at claforte (my number is #7115). BTW I'm also trying to get the authorization to reward the most helpful open-source developers with a few Navi2 and Navi3 GPUs (soon after they are officially released). :-)

Thank you @claforte @harishanand95 for your efforts at making Stable Diffusion more accessible. I run an RX 580, GFX803 which seems to have lost AMD ROCM support long ago. Still, the internet is full of workarounds that do not work in my experience. Looking forward to your guy's hard work to get us to use the open-source API method.

The main issue here is the windows route. If you use linux you can even use the goto stable diffusion UI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs
Still, I would love to see windows support through the Vulkan API.
If I understand it correctly we need to convert the SD model to SPIR-V using iree-compiler?
There is an example using SHARK: https://github.com/nod-ai/SHARK/blob/b448770ec26d8b8b0cf332f752915ac39b02d935/shark/examples/shark_inference/stable_diff.py

@cpietsch It doesn't work for Linux very well. The Linux-exclusive ROCM only properly support their workstation GPUs and support for consumer GPUs is lagging. You'd have to follow weird workarounds to get them working on the recent cards. And for slightly older cards like GFX803, it turns out to be impossible.

Oh sorry about that. It worked out of the box for my Radeon VII and I thought that that this was the same for the rest.

Hello everyone. As Christian mentioned, we have added a new pipeline for AMD GPUs using MLIR/IREE. This approach significantly boosts the performance of running Stable Diffusion in Windows and avoids the current ONNX/DirectML approach.

Instructions: https://github.com/nod-ai/SHARK/blob/main/shark/examples/shark_inference/stable_diffusion/stable_diffusion_amd.md

Please reach out to us on the discord link on the instructions page or create GitHub issues if something does not work for you.

Thanks!

@averad, Could you please give it a try and update your instructions too? You can reach us on the discord channel if you have any questions, Thanks!

@harishanand95 I will give it a try and update the Instructions.

@harishanand95 I wasn't able to test the process as IREE doesn't have support for RX 500 series cards - GCNv3

I've suggested adding def VK_TTA_RGCNv3 : I32EnumAttrCase<"AMD_RGCNv3", 103, "rgcn3">; and am working on compiling IREE with my suggested changes for testing.

I am getting 3.85 it/s on my 6900xt on SHARK (vulkan), that is 13 seconds for 50 iterations

hi, the exe doesnt work for me following your little guide (using the MLIR driver on 6900XT), im getting errors:

ill try the manual guide

For me the Advanced Installation worked

Time generation ?