I wrote these instructions as part of "installing PyTorch with CUDA 12.1.1".
I extracted them into this separate gist, because I realised there's a much easier way to install magma for CUDA 12.1.1:
https://anaconda.org/pytorch/magma-cuda121
There's a conda package!
conda install -c pytorch magma-cuda121
CUDA 12.1.1 toolkit is gonna offer to install Nvidia driver 530 for us. It's from New Feature branch. It's likely to be newer than the default Nvidia driver you would've installed via apt-get (apt would prefer to give you 525, i.e. Production Branch).
If you're confident that you already have a new enough Nvidia driver for CUDA 12.1.1, and you'd like to keep your driver: feel free to skip this "uninstall driver" step.
But if you're not sure, or you know your driver is too old: let's uninstall it. CUDA will install a new driver for us later.
| float_width=2 # float16 | |
| cond_count=2 # uncond and cond for 1 sample | |
| attn_heads=8 # SD1.5 isn't optimized for flash attn, so all layers have 8 heads, lol | |
| vae_scale_factor=8 | |
| px_height=px_width=768 | |
| latent_height=px_height/vae_scale_factor | |
| latent_width=px_width/vae_scale_factor | |
| q_proj_tokens=k_proj_tokens=latent_height*latent_width | |
| qk_bytes = cond_count*attn_heads*float_width*q_proj_tokens*k_proj_tokens | |
| qk_mb = qk_bytes/1024**2 |
| from torch import FloatTensor, load, baddbmm, zeros | |
| from dataclasses import dataclass | |
| import torch | |
| from os.path import join | |
| @dataclass | |
| class Fixtures: | |
| q_proj: FloatTensor | |
| k_proj: FloatTensor |
| from torch import FloatTensor | |
| vae_scale_factor = 8 | |
| typical_self_attn_key_length = (512/vae_scale_factor) * (512/vae_scale_factor) | |
| desired_self_attn_key_length = (768/vae_scale_factor) * (768/vae_scale_factor) | |
| key_length_factor=desired_self_attn_key_length/typical_self_attn_key_length if is_self_attn else 1. | |
| def softmax(x: FloatTensor, dim=-1) -> FloatTensor: | |
| maxes = x.max(dim, keepdim=True).values |
| from torch import FloatTensor | |
| vae_scale_factor = 8 | |
| typical_self_attn_key_length = (512/vae_scale_factor) * (512/vae_scale_factor) | |
| desired_self_attn_key_length = (200/vae_scale_factor) * (200/vae_scale_factor) | |
| key_length_factor=desired_self_attn_key_length/typical_self_attn_key_length if is_self_attn else 1. | |
| def softmax(x: FloatTensor, dim=-1) -> FloatTensor: | |
| key_tokens = x.size(-1) |
| from torch import FloatTensor | |
| def softmax(x: FloatTensor, dim=-1) -> FloatTensor: | |
| maxes = x.max(dim, keepdim=True).values | |
| diffs = x-maxes | |
| x_exp = diffs.exp() | |
| x_exp_sum = x_exp.sum(dim, keepdim=True) | |
| quotient = x_exp/x_exp_sum | |
| return quotient |
| No operator found for `memory_efficient_attention_forward` with inputs: | |
| query : shape=(32, 4096, 1, 40) (torch.float16) | |
| key : shape=(32, 77, 1, 40) (torch.float16) | |
| value : shape=(32, 77, 1, 40) (torch.float16) | |
| attn_bias : <class 'torch.Tensor'> | |
| p : 0.0 | |
| `cutlassF` is not supported because: | |
| attn_bias.shape[-1] % 8 != 0 | |
| `flshattF` is not supported because: | |
| attn_bias type is <class 'torch.Tensor'> |
| #!/usr/bin/env bash | |
| # makes a video like: | |
| # https://twitter.com/Birchlabs/status/1632185133435437057 | |
| # expects folders with filenames such as: | |
| # /home/birch/git/diffusers-play/out_dynthreshoff/00000.679566949.cfg07.50.png | |
| i=0 | |
| CFG_COUNTER='' | |
| FONT='fontfile=/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf' |
Mahouko prompts used for latent walk (WD 1.5 beta 1)
positive prompt:
beautiful, 1girl, a smiling and winking girl, wearing a dark kimono, taking a selfie on a bridge over a river. detailed hair, portrait, floating hair, anime, carnelian, best aesthetic, best quality, ribbon, outdoors, good posture, watercolor (medium), traditional media, ponytail'