Skip to content

Instantly share code, notes, and snippets.

View alex0dd's full-sized avatar

Alex O. P. alex0dd

View GitHub Profile
@alex0dd
alex0dd / example.py
Last active April 20, 2023 16:59
Visualizing memory consumption of a PyTorch model.
import torch
from pytorch_memory_profiling import memory_logging
from torchvision.models import resnet18
def training_loop(model, criterion, optimizer, dataloader):
model.train()
for (batch_x, batch_y) in dataloader:
pred_y = model(batch_x)
loss = criterion(pred_y, batch_y)
@ejmejm
ejmejm / pytorch_tips_yt_follow.ipynb
Created May 9, 2021 09:14
pytorch_tips_yt_follow.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@arestifo
arestifo / ProGAN v3.py
Created May 19, 2020 01:37
Working (up to 1024x1024) version of my progressively growing GAN implementation in TensorFlow + Keras
import tensorflow as tf
from tensorflow.keras import layers, Model, optimizers, initializers, constraints
from tensorflow.keras import backend as k
from tensorflow.keras.layers import Dense, Conv2D, Input, LeakyReLU, Reshape, Flatten
from tensorflow.keras.layers import AveragePooling2D, UpSampling2D
from tensorflow.keras.constraints import max_norm
import matplotlib.pyplot as plt
import numpy as np
import glob
import os
@danmou
danmou / auto_shape_mixin.py
Created December 5, 2019 10:00
Mixin for `tf.keras.layers.Layer`s and subclasses to automatically define input and output specs the first time the model is called.
from typing import Any, Mapping, Optional, Sequence, TypeVar, Union
import tensorflow as tf
from tensorflow.keras import layers
T = TypeVar('T')
Nested = Union[T, Sequence[T], Mapping[Any, T]]
class AutoShapeMixin:
@BarclayII
BarclayII / movielens.py
Last active January 5, 2026 15:46
PinSage example implementation
import pandas as pd
import dgl
import os
import torch
class MovieLens(object):
def __init__(self, directory):
'''
directory: path to movielens directory which should have the three
files:
@thomwolf
thomwolf / gradient_accumulation.py
Last active August 2, 2025 21:09
PyTorch gradient accumulation training loop
model.zero_grad() # Reset gradients tensors
for i, (inputs, labels) in enumerate(training_set):
predictions = model(inputs) # Forward pass
loss = loss_function(predictions, labels) # Compute loss function
loss = loss / accumulation_steps # Normalize our loss (if averaged)
loss.backward() # Backward pass
if (i+1) % accumulation_steps == 0: # Wait for several backward steps
optimizer.step() # Now we can do an optimizer step
model.zero_grad() # Reset gradients tensors
if (i+1) % evaluation_steps == 0: # Evaluate the model when we...
@MarcoWorms
MarcoWorms / mini-redux.js
Last active June 3, 2024 04:42
Redux in a nutshell
function createStore (reducers) {
var state = reducers()
const store = {
dispatch: (action) => {
state = reducers(state, action)
},
getState: () => {
return state
}
}
@emiller
emiller / git-mv-with-history
Last active January 11, 2026 20:22
git utility to move/rename file or folder and retain history with it.
#!/bin/bash
#
# git-mv-with-history -- move/rename file or folder, with history.
#
# Moving a file in git doesn't track history, so the purpose of this
# utility is best explained from the kernel wiki:
#
# Git has a rename command git mv, but that is just for convenience.
# The effect is indistinguishable from removing the file and adding another
# with different name and the same content.
@jterrace
jterrace / xvfb
Created June 11, 2012 18:46
xvfb init script for Ubuntu
XVFB=/usr/bin/Xvfb
XVFBARGS=":1 -screen 0 1024x768x24 -ac +extension GLX +render -noreset"
PIDFILE=/var/run/xvfb.pid
case "$1" in
start)
echo -n "Starting virtual X frame buffer: Xvfb"
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS
echo "."
;;
stop)
@hellerbarde
hellerbarde / latency.markdown
Created May 31, 2012 13:16 — forked from jboner/latency.txt
Latency numbers every programmer should know

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs