Skip to content

Instantly share code, notes, and snippets.

View a10y's full-sized avatar
🇺🇲
DC

Andrew Duffy a10y

🇺🇲
DC
View GitHub Profile
@a10y
a10y / TurboQuant_Summary.md
Last active March 26, 2026 14:13
Summary of TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (arXiv:2504.19874) - Undergraduate-level explanation

TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate

Paper Summary | arXiv:2504.19874

Authors: Amir Zandieh (Google Research), Majid Daliri (NYU), Majid Hadian (Google DeepMind), Vahab Mirrokni (Google Research)


What This Paper Is About

0000000000040470 t _GLOBAL__sub_I_nvcompManagerFactory.cpp
00000000000e5780 t __device_stub__ZN6nvcomp20fill_cudecomp_paramsEPKPKvPKmPKPvPmP24CUmemDecompressParams_stP14nvcompStatus_tmPi(void const* const*, unsigned long const*, void* const*, unsigned long*, CUmemDecompressParams_st*, nvcompStatus_t*, unsigned long, int*)
00000000000abc30 t __device_stub__ZN6nvcomp38compact_comp_buffers_and_header_outputEPhPKPKhPKmP12CommonHeaderPmmmm10FormatTypeS9_m(unsigned char*, unsigned char const* const*, unsigned long const*, CommonHeader*, unsigned long*, unsigned long, unsigned long, unsigned long, FormatType, unsigned long*, unsigned long)
0000000000115ac0 t __device_stub__ZN14nvcomp_deflate18fill_params_kernelEP24CUmemDecompressParams_stPKPKvPKmPmPKPvbP14nvcompStatus_tmPKi(CUmemDecompressParams_st*, void const* const*, unsigned long const*, unsigned long*, void* const*, bool, nvcompStatus_t*, unsigned long, int const*)
00000000000e3770 t __device_stub__ZN6nvcomp8lowlevel20fill_cudecomp_paramsEPKPKvPKmPKPvPmP24CUmem
# find all refs with remotes that are deleted on the server, and delete them locally
git branch -vv | grep ': gone]' | awk '{print $1}' | xargs -r git branch -D
config:
    flowchart:
        defaultRenderer: elk
flowchart TD
   %% Data Types
   Integer[Integer]:::datatype
   Float[Float]:::datatype
   String[String]:::datatype
@a10y
a10y / macros.rs
Created May 8, 2025 13:31
Vortex DType macros to revisit later
/// Create a new struct type. For example:
///
/// ```
/// use vortex_dtype::{struct_type, DType, PType};
///
/// let the_struct = struct_type! {
/// "x" => DType::from(PType::F64),
/// "y" => DType::from(PType::F64),
/// };
///
@a10y
a10y / Dockerfile
Created April 27, 2025 20:50
Get NanoLLM working on AGX Orin JetPack 6.2
FROM dustynv/nano_llm:r36.4.0
RUN apt update && apt install -y gstreamer1.0-nice
package org.apache.iceberg.parquet;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericRecord;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
@a10y
a10y / jfk_ocr.py
Created March 19, 2025 15:33
JFK Files analysis
import json
import os
import glob
import pytesseract
import multiprocessing
from pdf2image import convert_from_path
def ocr_pdf_to_json(pdf_path):
# Convert PDF to images (one per page)
@a10y
a10y / zstd_bench.rs
Created February 18, 2025 15:04
Benchmarking ZSTD throughput. Code mostly generated with Claude 3.5 Sonnet
// use rand::Rng;
use std::time::Instant;
use zstd::{decode_all, stream::encode_all};
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Generate sample data - 1 million random u32 integers
let count = 64_000;
println!("Generating {} random integers...", count);
// let mut rng = rand::rng();
plugins {
`java-library`
`maven-publish`
`signing`
}
val tokenizerSharedLibrary by configurations.creating {
isCanBeConsumed = false
}