twobob’s gists

twobob / qlorsStack.py

Created July 21, 2023 20:23

QLORA from Stackexchange zim exports

	import argparse
	import os
	import sys
	import random
	import codecs
	import json
	from bs4 import BeautifulSoup
	from multiprocessing import Pool
	import colorama
	from colorama import Fore

twobob / Wget command

Created January 9, 2023 06:14

Wget command to download the default ckpt file for fine-tuning directly to google drive from colab

!wget https://model-server.zqevans2.workers.dev/jmann-small-190k.ckpt -O /content/drive/MyDrive/AI/models/jmann-small-190k.ckpt

twobob / duall.sh

Created October 16, 2022 23:18

list disk usage without warnings as precis

du -cBM --max-depth=1 2> >(grep -v 'Permission denied') | grep -v 'cannot access' | sort -n

twobob / gist:68d8a55b2bf4a577c8afb431fb145f60

Created October 12, 2022 22:40

Downloading files and ckpt from Kaggle, minimal hassle

	! pip install kaggle
	! mkdir ~/.kaggle
	!cp /content/drive/MyDrive/kaggle.json ~/.kaggle/kaggle.json

	# AN INDIVIDUAL .ckpt FILE

	LINK='omgsupportteam/dd-tb-fine-22-10-11-22-42-36'
	FILENAME = 'last.ckpt'
	! kaggle datasets download $LINK -f $FILENAME

twobob / left.py

Created October 8, 2022 01:45

print how long you have left in your daily colab session. Naive counter

	import sys, json
	from dateutil.parser import parse
	from datetime import datetime, timezone, timedelta
	with open("/var/colab/app.log", 'r') as fileData:
	for textline in fileData:
	if " started" in textline:
	time = parse (json.loads(textline)['time'])
	now = datetime.now(time.tzinfo)
	later = time + timedelta(hours=4)
	print(now-time, "session started")

twobob / gist:f36342b286006c3830409387d88cc482

Created October 5, 2022 17:58

pruning code. Dance diffusion All props to Waifu

	import torch
	def prune_it(p):
	print(f"prunin' in path: {p}")
	size_initial = os.path.getsize(p)
	nsd = dict()
	sd = torch.load(p, map_location="cpu")
	print(sd.keys())
	for k in sd.keys():
	if k != "optimizer_states":
	nsd[k] = sd[k]

twobob / gist:cd55e1b9145f90b7d3c9537f338804a2

Created October 5, 2022 17:52

zip up training demos for Dance Diffusion

	NAMEZIP = f"/content/drive/MyDrive/AI/models/DanceDiffusion/finetune/{NAME}/{NAME}_zipped_demos.7z"
	!7z a -t7z $NAMEZIP /content/sample-generator/*.wav

twobob / colab.py

Created September 21, 2022 01:12

3 lines to create a shareable gz for a folder of wavs in colab

	!rsync -a /content/drive/MyDrive/PathToSamples/ /content/Temp
	!tar -zcf /content/normalized.gz /content/Temp
	!cp /content/normalized.gz /content/drive/MyDrive/PathForZip/normalized.gz

twobob / example_image_utils.py

Created August 31, 2022 12:02 — forked from pojda/example_image_utils.py

Layer on top of Python Imaging Library (PIL) to write text in images easily

	#!/usr/bin/env python
	# coding: utf-8

	# You need PIL <http://www.pythonware.com/products/pil/> to run this script
	# Download unifont.ttf from <http://unifoundry.com/unifont.html> (or use
	# any TTF you have)
	# Copyright 2011 Álvaro Justen [alvarojusten at gmail dot com]
	# License: GPL <http://www.gnu.org/copyleft/gpl.html>

	from image_utils import ImageText

twobob / example_image_utils.py

Created August 31, 2022 12:01 — forked from turicas/example_image_utils.py

Layer on top of Python Imaging Library (PIL) to write text in images easily

	#!/usr/bin/env python
	# coding: utf-8

	# You need PIL <http://www.pythonware.com/products/pil/> to run this script
	# Download unifont.ttf from <http://unifoundry.com/unifont.html> (or use
	# any TTF you have)
	# Copyright 2011 Álvaro Justen [alvarojusten at gmail dot com]
	# License: GPL <http://www.gnu.org/copyleft/gpl.html>

	from image_utils import ImageText

_ twobob