Skip to content

Instantly share code, notes, and snippets.

In Plato’s “Republic”, society comprises men of gold, who are fit to rule, men of silver, destined to be soldiers, and those of bronze, who become farmers. Since 1959 the world’s top teenage mathematicians (and, more recently, those of great ability in other fields) have tested their mettle in international competitions called Olympiads. Around 300 medals of gold, silver and bronze are awarded for maths annually. A smaller number are given for physics, chemistry and informatics (ie, computing). From the ranks of the winners has emerged an elite unlike any in the history of capitalism. Behold: Olympiad Man.

The crisis over the safety of artificial intelligence is best understood as a crisis among Olympiad Men. Last year AI models managed gold-medal performances in maths. Relentless advances since have led a pair of researchers at Anthropic, who made up a third of Britain’s maths Olympiad team in 2017, to resign. Anthropic’s boss, Dario Amodei, was once a member of the squad, but not quite the starting line-up

@cosminscn
cosminscn / gist:fd2232484bf6b05e79f419da71b7072f
Created September 16, 2026 19:16
classic demoscene plasma
import numpy as np
from PIL import Image
import math, os
# Classic demoscene-style plasma GIF:
# layered spatial sine waves + a phase-shifting RGB cosine palette.
W, H = 480, 360
N_FRAMES = 72
DURATION_MS = 45

Dwarkesh Patel × Jensen Huang Interview — Key Questions & Timestamps https://www.youtube.com/watch?v=Hrbq66XqtCo

1. The Supply Chain and Hardware Moat

  • Commoditization (00:06) — If software gets commoditized by AI, does Nvidia's hardware business get commoditized as well, since TSMC and others handle the actual manufacturing?

  • Scarce Components (04:28) — With over $100B in purchase commitments, is Nvidia's true moat simply locking up years of scarce components (logic and memory) so competitors can't physically build their accelerators?

  • Scaling Upstream (08:32) — How do you continue to 2x production year-over-year when you are already taking up the vast majority of TSMC's advanced nodes? Are we entering a regime where AI compute growth has to slow down because of the upstream supply chain?

"""
Optimized NanoGPT Adder: 66 params + 0 buffers
transformer.wte.A [10, 1] = 10
transformer.wte.B [1, 4] = 4
transformer.h.0.attn.q_proj.angle_h0 [1] = 1
transformer.h.0.attn.q_proj.angle_h1 [1] = 1
transformer.h.0.attn.q_proj.scale [1] = 1
transformer.h.0.attn.k_proj.weight [4, 2] = 8
transformer.h.0.attn.v_proj.u [4, 1] = 4
"""58-param nanoGPT that adds any two 10-digit numbers. No training.
Down from 130 params. HONEST counting: every stored numerical value is an
nn.Parameter. Zero buffers. The only "free" things are structural choices
(which dim connects where), control flow, and pure math (sin/cos/arange
for PE generation — same convention as the original).
Parameter budget:
wte.A (10×1) + wte.B (1×4) = 14 [factorized embedding]
Q (2 angles + 1 scale) = 3 [rotation-parameterized]
"""
Dynamic NanoGPT Adder: 130 params + 0 buffers
transformer.wte.A [10, 1] = 10
transformer.wte.B [1, 4] = 4
transformer.h.0.attn.c_attn.weight [12, 4] = 48
transformer.h.0.attn.c_proj.weight [4, 4] = 16
transformer.h.0.mlp.c_fc.weight [4, 4] = 16
transformer.h.0.mlp.c_fc.bias [4] = 4
transformer.h.0.mlp.c_proj.u [4, 1] = 4
"""190-param nanoGPT that adds any two 10-digit numbers. All weights hand-coded."""
import math
from dataclasses import dataclass
import torch, torch.nn as nn
from torch.nn import functional as F
# === NanoGPT (from github.com/karpathy/nanoGPT/blob/master/model.py) ===
# Modifications: sinusoidal PE buffer, configurable mlp_hidden, c_fc bias always on
class CausalSelfAttention(nn.Module):
@cosminscn
cosminscn / notes_dt.md
Last active August 28, 2025 01:03
Distributed training notes - aug 27 25

A100 - spec 312 TFLOPS/s

40GB? 80GB HBM ram 20MB cache

Large model run

@cosminscn
cosminscn / cover.py
Last active August 26, 2025 22:50
cover
# This solver covers a rectangular grid of '.' and 'X' with two non-rotatable tiles:
# • A (U-shape): on row i it occupies (i,j) and (i,j+3), and on row i+1 it occupies (i+1, j..j+3).
# • B (horizontal domino): occupies (i,j) and (i,j+1) on the same row.
# We scan left→right, row→row using top-down DP with memoization. The DP state is
# (i, j, needA, anchors):
# – i, j: current row and column being decided.
# – needA: tuple[bool] of length W marking cells in the CURRENT row that are forced to 'A'
# because they are the bottom strip of some A placed in the PREVIOUS row.
# – anchors: tuple of columns where we have already anchored an A in the CURRENT row before j;
# each anchor at column a also forces (i, a+3) to be 'A' (its top-right leg) and
Year Acquirer Target Price / Structure Core Focus
2010 Apple Siri ≈ $200 M Voice assistant / NLP
2013 Google DNNresearch (Hinton) ≈ $44 M Deep CNN research (AlexNet)
2014 Google DeepMind $500–650 M Deep RL & general AI
2015 Facebook Wit.ai n/d Speech / NLU APIs
2016 Intel Nervana $350–400 M DL ASIC & framework
2016