morrisalp’s gists

morrisalp / prompt_mixing.py

Created April 4, 2024 11:47

example of prompt mixing (different prompts at different denoising steps) with SD2.1

	import torch
	from diffusers import StableDiffusionPipeline
	from matplotlib import pyplot as plt

	device = "cuda" if torch.cuda.is_available() else "cpu"

	pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16)
	pipe = pipe.to(device)

	prompts = ["a hamburger"] * 8 + ["a birthday cake"] * 12

morrisalp / sdxl_with_grad.py

Created January 14, 2024 18:24

Diffusers SDXL pipeline with gradients (overriding no_grad), tested with diffusers v25.0. Use output_type="latent" when calling pipeline to get latents with gradients.

	from diffusers import StableDiffusionXLPipeline


	class CustomPipeline(StableDiffusionXLPipeline):

	@classmethod
	def from_pretrained(cls, args, *kwargs):
	self = super().from_pretrained(args, *kwargs)
	assert self.watermark is None # watermarking currently not supported
	def postprocess_no_grad(image, args, *kwargs):

morrisalp / commons_data.py

Created September 10, 2023 16:43

simple API calls to get lists of files and (sub)categories from WikiMedia Commons

	import requests
	import json
	from tqdm.auto import tqdm
	import pandas as pd

	def kwargs2url(**kwargs):
	url = 'https://commons.wikimedia.org/w/api.php?'
	for k, v in kwargs.items():
	url += f'{k}={v}&'
	return url.rstrip('&')

morrisalp / surprise_ml-100k.py

Last active May 14, 2021 07:49

loading MovieLens 100K dataset with Surprise & Pandas

	import surprise
	import pandas as pd

	data = surprise.Dataset.load_builtin('ml-100k')
	ddir = surprise.get_dataset_dir()

	item_data = pd.read_csv(f'{ddir}/ml-100k/ml-100k/u.item',
	sep='\|',
	header=None,
	encoding='ISO-8859-1',

morrisalp / search-pip.sh

Created February 26, 2021 07:42

Minimal Bash command for searching for Pip packages, now that 'pip search' no longer works. Usage: `./search-pip.sh pyngrok`

	#!/bin/sh
	wget -O - https://pypi.org/simple/ 2>/dev/null \| grep "$1" \| grep -Po "(?<=>).*(?=<)"

morrisalp / bandit_demo.py

Last active July 6, 2022 07:35

Demo for illustrating multi-armed bandit algorithms. The methods of the class Agent are placeholders and may be changed.

	import numpy as np

	class Agent:

	def __init__(self, n_machines=10):
	self.t = 0 # current time step
	self.n_machines = n_machines

	def pick_machine(self):
	# returns index of machine

morrisalp / flask_caching_demo.py

Created August 23, 2020 17:59

demo of flask_caching - calculating pi with ζ(2)

	from flask import Flask
	from flask_caching import Cache

	app = Flask(__name__)
	app.config.from_mapping({"CACHE_TYPE": "simple"})
	cache = Cache(app)

	def approximate_pi(n):
	output = 0
	for i in range(1, n):

morrisalp / top_k_categorical_accuracy.py

Created July 24, 2020 07:46

top K categorical accuracy for numpy arrays (sklearn predict_proba outputs)

	import numpy as np

	def top_k_categorical_accuracy(y_true, y_pred_proba, k=1):
	return np.equal(np.argsort(y_pred_proba)[:, -k:], y_true[:, None]).any(axis=1).mean()

morrisalp / simple_bert.py

Last active June 17, 2020 00:53

minimal example of getting BERT embeddings for sentence, using TF 2.0 + Tensorflow Hub + HuggingFace tokenizers library

	import tensorflow as tf
	import tensorflow_hub as hub
	from tokenizers import BertWordPieceTokenizer
	from tensorflow.keras.layers import Input
	from tensorflow.keras.models import Model
	import numpy as np

	class BERTPreprocessor:
	SEP_TOKEN = '[SEP]'

morrisalp / html2text.py

Created April 7, 2020 11:45

sane text extraction given html string, using BeautifulSoup

	from bs4 import BeautifulSoup as bs

	def html2text(html):
	soup = bs(html, features='lxml')
	for script in soup(["script", "style"]):
	script.decompose()
	for br in soup.find_all("br"):
	br.replace_with("\n")
	return soup.get_text(separator=' ').strip()

Morris Alper morrisalp