vahbuna / mp_train.md

Last active January 14, 2025 12:13

Mixed Precision Training

Pytorch Solution

Helper functions

def prep_param_lists(model):
  """ fp32 copy of model parameters """
  model_params = [p for p in model.parameters() if p.requires_grad]

vahbuna / einsum.md

Last active January 14, 2025 12:13

Numpy Einstein Summation

Einsum

A = np.array([[1, 1, 1],
              [2, 2, 2],
              [5, 5, 5]])

B = np.array([[0, 1, 0],
              [1, 1, 0],
 [1, 1, 1]])

vahbuna / fastai-nlp.md

Last active September 1, 2020 11:05

Notes on fastai nlp course

A Code-First Introduction to Natural Language Processing

Introduction

Course Playlist

Course Github

2

vahbuna / feature_platform.md

Created February 26, 2023 09:00

feature platform bibliography

source

[2018]
- Paypal: Data Pipelines for Real-Time Fraud Prevention at Scale (Mikhail Kourjanski)
- Airbnb: Zipline—Airbnb’s Declarative Feature Engineering Framework (Nikhil Simha and Varant Zanoyan)
  - Zipline—Airbnb’s Declarative Feature Engineering Framework (2019)
[2019]
- Stripe: Reproducible Machine Learning with Functional Programming (Oscar Boykin)
Uber: Michelangelo Palette: A Feature Engineering Platform at Uber (Amit Nene)

vahbuna / prompt_engg.md

Created March 31, 2023 06:09

Prompt Engineering - Lilian Weng

https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/

prompt engineering for autoregressive language models

https://github.com/openai/openai-cookbook

https://langchain.readthedocs.io/en/latest/

https://github.com/dair-ai/Prompt-Engineering-Guide

vahbuna / cnlg.md

Created March 31, 2023 08:18

Controllable Neural Text Generation - Lilian Weng

Decoding Strategies

Common Decoding Methods

Greedy search

Beam search

Top-k sampling https://arxiv.org/abs/1805.04833

vahbuna / pymem.md

Last active August 5, 2025 02:45

Debugging PyTorch memory use with snapshots - Zach's Blog

https://zdevito.github.io/2022/08/16/memory-snapshots.html

# enable the recording of stack frame information for each allocation
import torch
torch.cuda.memory._record_memory_history(True)

from torchvision.models import resnet18
from pprint import pprint

vahbuna / orosz_books.md

Created April 5, 2023 22:55

Gergely Orosz Book List

vahbuna / not_enough_data.md

Created April 5, 2023 23:05

references from Learning with not Enough Data - Lilian Weng

Semi-Supervised Learning

[1] Ouali, Hudelot & Tami. “An Overview of Deep Semi-Supervised Learning” arXiv preprint arXiv:2006.05278 (2020).

[2] Sajjadi, Javanmardi & Tasdizen “Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning.” arXiv preprint arXiv:1606.04586 (2016).

[3] Pham et al. “Meta Pseudo Labels.” CVPR 2021.

[4] Laine & Aila. “Temporal Ensembling for Semi-Supervised Learning” ICLR 2017.

[5] Tarvaninen & Valpola. “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.” NeuriPS 2017

vahbuna / hn_bm.md

Last active May 31, 2023 21:12

hacker news bookmark

CS 61B Data Structures, Spring 2023 UC Berkeley https://sp23.datastructur.es/

Brex’s Prompt Engineering Guide https://github.com/brexhq/prompt-engineering

Bark: A transformer based text to audio system https://github.com/suno-ai/bark

Attempto Controlled English https://en.wikipedia.org/wiki/Attempto_Controlled_English

Run Llama 13B with a 6GB graphics card https://gist.github.com/rain-1/8cc12b4b334052a21af8029aa9c4fafc

Anubhav vahbuna