Skip to content

Instantly share code, notes, and snippets.

View wassname's full-sized avatar
🙃

wassname (Michael J Clark) wassname

🙃
  • I'm just a guy who likes to machine learn
  • IAU #90377 (Sedna), IAU #137924 (2000 BD19), IAU #85770 (1998 UP1), IAU #66063 (1998 RO1), and minor planet 1992 TY₁. from 2025 Feb 25.
  • X @wassname
  • LinkedIn in/mclark52
View GitHub Profile
@wassname
wassname / load_md_fm_j2_prompt.py
Created March 4, 2025 03:03
IMO the nicest prompt format is prompt.md.j2. Here we make the messages explicit, and the markdown and jinja syntax obvious
def split_frontmatter(fm_md_split :str):
"""Load prompt in md.jinja2 format
In this format we have multiple frontmatters and content sections, each defining a message. The idea here is to use jinja formatting in a promt.md.jinja file to make the markdown, and jinja formatting obvious
e.g.
---
role: system
---
@wassname
wassname / fix_peft_local_invalid_base_model.py
Last active March 1, 2025 00:31
When a peft adapter has an invalid base model, how do I fix it?
peft_model_id = 'v2ray/GPT4chan-8B-QLoRA'
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoConfig, LlamaConfig
from peft import PeftConfig, PeftModelForCausalLM
peft_config = PeftConfig.from_pretrained(peft_model_id)
# This model points to a local base model, which we don't have. So lets redirect it to a public version
peft_config.base_model_name_or_path="unsloth/Llama-3.1-8B"
# now load the modified config in 8bit

Design a physics-based model specifically for an LNG (Liquefied Natural Gas) compressor, suitable for implementation in a Physics-Informed Neural Network (PINN).

Given inputs:

  • Inlet temperature (t_in)
  • Inlet pressure (p_in)
  • Inlet flow rate (flow_in)
  • Inlet Guide Vane position (IGV%)

Required outputs to predict:

  • Outlet temperature (t_out)
@wassname
wassname / transformers_bf16.py
Last active January 27, 2025 11:26
Using 16 bit base weights in huggingface transformers
"""
You can save memory by converting your model to bf16, but ONLY if you use a special optimiser. Otherwise you round away small changes and get worse result.
This is how to use a brainfloat16 base model in huggingface transformers
@author:wassname
@url: https://gist.github.com/wassname/183153f9245b37ae6d08b3c3c4033bda
Usage:
@wassname
wassname / hf_tf_no_print_progress.py
Last active January 27, 2025 02:08
transformers stop the printing, while still logging to wandb, tensorboard, csv etc
from transformers.trainer_callback import ProgressCallback
class ProgressCallbackNoPrint(ProgressCallback):
"""ProgressCallback that doesn't print anything
Usage:
# at top of file, after transformers import, before settin up trainer, monkey patch the default progress callback
import transformers
transformers.DEFAULT_PROGRESS_CALLBACK = ProgressCallbackNoPrint
@wassname
wassname / reddit_thread2md.py
Created December 26, 2024 00:46
Format a reddit thread into markdown suitable for an llm
# from https://github.dev/JosefAlbers/rd2md
import textwrap
from datetime import datetime
def format_flair(obj):
if obj.author_flair_text is not None:
return f" *{obj.author_flair_text}*"
return ""
def setattrattr(cfg, k, v):
"""
Sets an attr even it's like o.a.b
Usage:
setattrattr(obj, 'loss.alpha', 0.1)
https://gist.github.com/wassname/33ef989ab06325daeeef68be1d638a2c/edit
"""
if '.' in k:
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 24 columns, instead of 8 in line 4.
Book Id,Title,Author,Author l-f,Additional Authors,ISBN,ISBN13,My Rating,Average Rating,Publisher,Binding,Number of Pages,Year Published,Original Publication Year,Date Read,Date Added,Bookshelves,Bookshelves with positions,Exclusive Shelf,My Review,Spoiler,Private Notes,Read Count,Owned Copies
60233239,"The Butcher's Masquerade (Dungeon Crawler Carl, #5)",Matt Dinniman,"Dinniman, Matt",,"=""""","=""""",5,4.68,Dandy House,Kindle Edition,726,2022,2022,,2024/08/27,,,read,,,,1,0
12913718,"달빛 조각사 1 (The Legendary Moonlight Sculptor, #1)",Heesung Nam,"Nam, Heesung",,"=""895857903X""","=""9788958579038""",4,4.38,로크미디어,Paperback,344,2007,2007,,2016/11/14,"favs, 5star","favs (#37), 5star (#132)",read,,,,1,0
18527310,"달빛 조각사 4 (The Legendary Moonlight Sculptor, #4)",Heesung Nam,"Nam, Heesung",,"=""8958579064""","=""9788958579062""",4,4.38,로크미디어,Paperback,335,2007,2007,,2024/08/27,,,read,,,,1,0
17661892,"달빛 조각사 2 (The Legendary Moonlight Sculptor, #2)",Heesung Nam,"Nam, Heesung",,"=""8958579048""","=""9788958579045""",4
@wassname
wassname / lighting_save_only_adapter.py
Last active September 6, 2024 03:19
Lightning snippets for use with transformers
from lightning.pytorch.callbacks import ModelCheckpoint
from weakref import proxy
class AdapterModelCheckpoint(ModelCheckpoint):
def _save_checkpoint(self, trainer: "pl.Trainer", filepath: str) -> None:
trainer.model.save_pretrained(filepath)
# trainer.save_checkpoint(filepath, self.save_weights_only)
self._last_global_step_saved = trainer.global_step
self._last_checkpoint_saved = filepath
@wassname
wassname / lightning_bnb_existing.py
Last active September 6, 2024 08:07
DO NOT USE, just set lightning to bfloat16 of float32 instead
# # FIXME DOT NOT USE
# # upon further investigation, it seems the bnb will handle conversion as long as you use bloat16 of float32, float16
# from lightning.pytorch.plugins.precision.precision import Precision
# from lightning.fabric.plugins.precision.utils import (
# _ClassReplacementContextManager,
# _convert_fp_tensor,
# _DtypeContextManager,
# )
# from typing_extensions import Self, override