Skip to content

Instantly share code, notes, and snippets.

View scturtle's full-sized avatar
🐢

scturtle

🐢
View GitHub Profile
@scturtle
scturtle / mpi.py
Last active January 20, 2025 09:03
nccl in 500 LOCs
#!/usr/bin/env python3
# https://github.com/FateScript/experiments/blob/main/se/mpi/mpi.py
# https://github.com/facebookincubator/gloo/tree/main/gloo
import math
import multiprocessing
import os
import numpy as np

How to Get Rich

A collection of all my interviews about my ‘How to Get Rich’ tweetstorm.

Seek Wealth, Not Money or Status

Wealth is assets that earn while you sleep

Naval is a prolific tech investor and founder of AngelList

@scturtle
scturtle / a2c.py
Last active December 24, 2024 03:56
Proximal Policy Optimization
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import numpy as np
import gym
# Hyperparameters
num_inputs = 4
num_actions = 2
× /
← → ↔ ⇐ ⇒ ⇔
⟵ ⟶ ⟷ ⟸ ⟹ ⟺
∀ ∃
∂ ∫
∈ ∊ ∉
∏ ∑
∙ ⋅
@scturtle
scturtle / llama3.py
Created May 20, 2024 09:43
llama3 in numpy
import numpy as np
class ModelArgs:
dim = 288
n_layers = 6
n_heads = 6
norm_eps = 1e-6
def build_cos_sin_cache(head_dim, seq_len, base=10000):
theta = 1. / (base ** (np.arange(0, head_dim, 2, dtype=np.float32) / head_dim))
@scturtle
scturtle / flash_attention.py
Last active April 11, 2024 16:38
flash attention v1 v2 in numpy
import numpy as np
N_inp = 64
N_out = 64
d = 128
Q = np.random.randn(N_out, d)
K = np.random.randn(N_inp, d)
V = np.random.randn(N_inp, d)
O = np.random.randn(N_out, d)
@scturtle
scturtle / matmul.py
Created April 8, 2024 11:44
how matmul is tiled in cuda
import numpy as np
from threading import Barrier, Thread
from collections import namedtuple
dim3 = namedtuple("dim3", ["x", "y", "z"], defaults=(1, 1))
TILE = 16
def cdiv(a, b):
@scturtle
scturtle / script.js
Created December 1, 2023 06:39
adventofcode daylight mode userscript
// ==UserScript==
// @name adventofcode daylight mode
// @namespace Violentmonkey Scripts
// @match https://adventofcode.com/*
// @grant none
// @version 1.0
// @author scturtle
// @description 2023/11/27 16:52:13
// ==/UserScript==
#!/usr/bin/env python3
import re
from pathlib import Path
from typing import List
MARK = "// -----// IR Dump"
PAT = re.compile(r'IR Dump \w+ (\w+)')
def main(infile: Path, outdir: Path):
assert infile.exists(), f'{infile} not exists'
@scturtle
scturtle / emutool.py
Created May 16, 2023 16:22
emuiibo data generator
import json
import time
import random
import struct
import urllib.request
from pathlib import Path
# https://www.amiiboapi.com/api/amiibo/
database = json.load(open("amiibos.json"))