Skip to content

Instantly share code, notes, and snippets.

@sayakpaul
sayakpaul / inference.md
Last active February 5, 2025 14:13
(Not so rigrously tested) example showing how to use `bitsandbytes`, `peft`, etc. to LoRA fine-tune Flux.1 Dev.

When loading the LoRA params (that were obtained on a quantized base model) and merging them into the base model, it is recommended to first dequantize the base model, merge the LoRA params into it, and then quantize the model again. This is because merging into 4bit quantized models can lead to some rounding errors. Below, we provide an end-to-end example:

  1. First, load the original model and merge the LoRA params into it:
from diffusers import FluxPipeline 
import torch 

ckpt_id = "black-forest-labs/FLUX.1-dev"
pipeline = FluxPipeline.from_pretrained(
@ariG23498
ariG23498 / flux-dev-under-8gbs.py
Created October 20, 2024 09:01
Run FLUX Dev under 8gbs of VRAM.
# Taken from: https://gist.github.com/sayakpaul/23862a2e7f5ab73dfdcc513751289bea
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel
import torch
import gc
def flush():
gc.collect()
@sayakpaul
sayakpaul / run_flux_under_24gbs.py
Last active January 29, 2025 21:11
This gist shows how to run Flux on a 24GB 4090 card with Diffusers.
from diffusers import FluxPipeline, AutoencoderKL
from diffusers.image_processor import VaeImageProcessor
from transformers import T5EncoderModel, T5TokenizerFast, CLIPTokenizer, CLIPTextModel
import torch
import gc
def flush():
gc.collect()
torch.cuda.empty_cache()
@kohya-ss
kohya-ss / forward_of_sdxl_original_unet.py
Created November 14, 2023 03:39
SDXLで高解像度での構図の破綻を軽減する
def forward(self, x, timesteps=None, context=None, y=None, **kwargs):
# broadcast timesteps to batch dimension
timesteps = timesteps.expand(x.shape[0])
hs = []
t_emb = get_timestep_embedding(timesteps, self.model_channels) # , repeat_only=False)
t_emb = t_emb.to(x.dtype)
emb = self.time_embed(t_emb)
assert x.shape[0] == y.shape[0], f"batch size mismatch: {x.shape[0]} != {y.shape[0]}"
@Stella2211
Stella2211 / DeepFloyd_IF_VRAM12GB.py
Last active January 17, 2024 03:59
DeepFloyd IF for around 12GB of VRAM
# huggingface login
from huggingface_hub import login
login()
# load textencorder in 8bit quantized
from transformers import T5EncoderModel
from diffusers import DiffusionPipeline
import datetime
@Norod
Norod / text-to-video-modelscope.py
Created April 16, 2023 08:08
Running ModelScope text2video locally with torch 2 gpu memory optimizations and control over parameters
# Ran it with the following packages installed:
# accelerate 0.18.0
# diffusers 0.16.0.dev0
# torch 2.0.0+cu118
# torchvision 0.15.0+cu118
# transformers 4.28.1
# xformers 0.0.18
import torch
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
from PIL import Image
import stable_inference
import numpy as np
from einops import repeat
# Interpolate video between two conditionings
FOLDER = 'test/painting'
MAX_STRENGTH = 0.5 # Strength at maximum in the middle of the interpolation
SEED = 9001
SECONDS = 10
bl_info = {
"name": "Kuchi Paku Light",
"author": "ds54e",
"version": (1, 1, 1),
"blender": (2, 80, 0),
"location": "View3D > Sidebar > KPL",
"description": "Generate Kuchi-Paku animations from the sound sequences in the VSE",
"warning": "",
"doc_url": "",
"category": "Animation",
@ZoomTen
ZoomTen / Node Tricks (Olive 0.2.x).md
Last active July 25, 2021 14:56
Sick Node Tricks - or, how to reproduce common effects using the node editor.

General Tips and Tricks

  • Think of clip nodes as containers!
    • You can plug in an arbitrary composite node and make them "instances" of said node

Common Node Configurations

Drop shadow (TinglesMagicWords)

A simple drop shadow effect, assuming you have a solid color and transparent clip such as text.

@Celeborn2BeAlive
Celeborn2BeAlive / build_otio_wheel.sh
Created May 20, 2020 17:39
Shell script to build a wheel of OpenTimelineIO 0.12.1
#!/bin/bash
# This script generate a wheel for OpenTimelineIO in dist folder
# We need it has a patch to install otio 0.12.1 in Windows Blender because it requires compilation
# This wheel should be distributed with code that depends on otio 0.12.1 and installed using Blender's python -m pip
# You need python (3.7.*) and Visual Studio 2017+
CURRENT_DIR=`dirname $0`
ROOT_DIR=$CURRENT_DIR/