Skip to content

Instantly share code, notes, and snippets.

@Norod
Norod / translate_file.py
Last active June 25, 2023 13:34
Example of using MarianMTModel to perform line-by-line text file translation on CPU or GPU
import torch
import textwrap
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, MarianTokenizer, MarianMTModel
def chunk_examples(txt_list, width):
chunks = []
for sentence in txt_list:
chunks += textwrap.wrap(sentence, width=width)
return chunks
@Norod
Norod / grokking-stable-diffusion-v1-5.ipynb
Last active July 29, 2023 20:52
grokking-stable-diffusion-v1-5.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / deepspeed-hebrew-gpt_neo_xl-textiteratorstreamer.ipynb
Created June 11, 2023 15:12
deepspeed-hebrew-gpt_neo_xl-TextIteratorStreamer.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / tokenizer-hebrew-wiki-from-gpt-j.ipynb
Created June 4, 2023 18:15
Train a hebrew tokenizer based on GPT-J using Hebrew WiKi as dataset
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / IF-I-XL-v1.0.ipynb
Created April 30, 2023 15:23
My local run of Deep Floyd (XL) model
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / IF-I-L-v1.0.ipynb
Created April 30, 2023 10:31
My local run of Deep Floyd (Large) model (smaller than the XL one)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / text-to-video-modelscope.py
Created April 16, 2023 08:08
Running ModelScope text2video locally with torch 2 gpu memory optimizations and control over parameters
# Ran it with the following packages installed:
# accelerate 0.18.0
# diffusers 0.16.0.dev0
# torch 2.0.0+cu118
# torchvision 0.15.0+cu118
# transformers 4.28.1
# xformers 0.0.18
import torch
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
@Norod
Norod / hebrew-gpt_neo-small-text-streaming-cpu.ipynb
Created March 30, 2023 15:19
hebrew-gpt_neo-small Text Streaming CPU
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Norod
Norod / remove_nikud.py
Last active March 28, 2023 08:34
Input hebrew text with diacritics (Nikud / Niqqud) and output the same text without it
#pip install hebrew
import sys
from hebrew import Hebrew
def open_input_file_name(file_name):
try:
file = open(file_name, "r")
return file
except FileNotFoundError:
@Norod
Norod / multidiffusion_panorama_withqolupdatesbynorod78.ipynb
Created March 27, 2023 20:26
MultiDiffusion_Panorama_withQoLUpdatesByNorod78.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.