Skip to content

Instantly share code, notes, and snippets.

@qpwo
qpwo / remove-dup-dirs.py
Created April 23, 2025 00:44
remove duplicate directories with ck1sum
#!/usr/bin/env python3
# dups2
from functools import lru_cache
import os
import subprocess
import csv
from concurrent.futures import ProcessPoolExecutor
from collections import Counter, defaultdict
from typing import Generator
import shutil
@qpwo
qpwo / c250.md
Created March 29, 2025 20:31
top 250 c standard library things
  • <stdio.h> (Standard Buffered Input/Output)

    • Provides core functions for performing input and output operations, primarily using buffered streams (FILE*) for efficiency, including standard streams (stdin, stdout, stderr), file operations, and formatted I/O.
    • FILE: Type representing a buffered file stream structure.
    • stdin: (FILE *) Macro representing the standard input stream.
    • stdout: (FILE *) Macro representing the standard output stream.
    • stderr: (FILE *) Macro representing the standard error stream.
    • EOF: (int) Macro representing End-Of-File (typically -1).
    • NULL: Macro representing a null pointer constant (often (void*)0). Also defined in other headers like <stdlib.h> and <stddef.h>.
    • size_t: Unsigned integer type for object sizes, counts. Also defined in <stddef.h>, <stdlib.h>, <string.h>.
@qpwo
qpwo / universal-summarizer.py
Created March 18, 2025 19:03
universal summarizer python again lol
def universal_summary(x, depth=0, max_depth=4, max_items=5, max_chars=350):
"""
Provides a concise summary of any Python object.
Args:
x: Any Python object
depth: Current recursion depth (internal use)
max_depth: Maximum recursion depth
max_items: Maximum number of items to show for collections
max_chars: Maximum characters in the final output
@qpwo
qpwo / hello.py
Created March 18, 2025 11:17
inline cuda kernel pytorch minimal example
import torch
from torch.utils.cpp_extension import load_inline
# Define C++ source code
cpp_source = """
#include <torch/extension.h>
torch::Tensor add_one(torch::Tensor input) {
return input + 1;
}
  • EfficientZero V2: A general sample-efficient RL framework excels in diverse control tasks (discrete/continuous, visual/low-dimensional) outperforming SoTA, including DreamerV3, in 50/66 benchmarks.
  • Gambling-Based Confidence Sequences: A novel gambling framework constructs tight, non-asymptotic confidence sequences for bounded random vectors, including categorical and probability-vector-valued observations, outperforming existing methods like the posterior-prior ratio martingale.
  • In-Context Learning Circuits: Mechanistic study reveals how induction heads, key to in-context learning in transformers, emerge through interactions of three identified sub-circuits during training.
  • Explaining Probabilistic Models with Distributional Values: This paper introduces distributional values, generalising cooperative game theory and value operators to provide fine-grained explanations of probabilistic models like vision and language models by tracking changes in model output rather than s
@qpwo
qpwo / my-validator.test.ts
Last active March 10, 2025 23:57
my typescript type validator (not zod!)
/**
* my-validator.test.ts
*
* Test suite for my-validator.ts library
*/
import { objSchema, arrSchema, unionSchema, validate, Infer } from './my-validator'
// Test 1: Basic object schema validation
function testBasicObject() {
@qpwo
qpwo / int53-collision.py
Created March 6, 2025 18:20
53-bit hash collision (birthday paradox for int53)
import mpmath as mp
import math
# Set high precision
mp.mp.dps = 50
def calculate_collision_probability(n, d):
"""
Calculate probability of at least one collision when choosing n items from d possibilities
Using the birthday paradox/collision probability formula: 1 - exp(-n(n-1)/(2d))
/** Run different types of jobs via postgres queue.
* Aims for exactly-once completion of (job, key) pair.
*/
import 'dotenv/config'
import { Pool, PoolClient } from 'pg'
import { JobStatus } from 'shared'
type StatusObj<TData = any, TResult = any> = {
jobId: string
jobType: string
@qpwo
qpwo / fastdl.sh
Last active January 26, 2025 23:56
fust download with r2 rclone and huggingface parallel
curl https://rclone.org/install.sh | sudo bash
mkdir ~/.config/rclone
echo "
[r2]
type = s3
provider = Cloudflare
access_key_id = $a
secret_access_key = $b
region = auto
endpoint = https://$c.r2.cloudflarestorage.com
@qpwo
qpwo / o1-fail.txt
Created January 22, 2025 02:35
o1 fail
Why does this run instantly after first trial?
import subprocess
import time
import shutil
import os
def run_rclone_command(command, num_trials=3, dest_path_base="/home/ubuntu/hff/405b-base"):
"""