Skip to content

Instantly share code, notes, and snippets.

View jlmelville's full-sized avatar

James Melville jlmelville

View GitHub Profile
@jlmelville
jlmelville / kabsch_best.R
Created December 26, 2024 20:01
Get best alignment of 2D coords over a reference set of points, allowing for a reflection. Useful for orienting dimensionality reduction output in 2D (UMAP, t-SNE etc.)
kabsch_best <- function(coords, ref) {
kabsch_res1 <- kabsch(coords, ref, ret_error = TRUE)
kabsch_res2 <- kabsch(coords |> flipx(), ref, ret_error = TRUE)
if (kabsch_res1$error < kabsch_res2$error) {
coords <- kabsch_res1$coords
} else {
coords <- kabsch_res2$coords
}
coords
}
@jlmelville
jlmelville / faiss-gpu-wsl2.md
Last active January 12, 2025 02:38
Building FAISS GPU for my laptop 1080 on Ubuntu WSL

January 11 2025 I recently upgraded to Ubuntu 12.10 (oracular) and tried all this again. Some settings needed to be changed:

  1. We're up to CUDA 12.5 now, but we also get GCC 14 with the latest ubuntu (I haven't tried any of this with clang). nvcc doesn't want anything above version 13, so you will want to sudo apt-get install g++-13 and then make sure nvcc and cmake know about it. Also, my attempts to adjust the C++ compiler did not affect the C compiler and it was still reporting version 14. I don't know if that would work, but I tried to also make it use version 13 also.
  2. I also wanted to use a version of python set by pyenv, but unsurprisingly I also had to specifically set where to look for python and numpy libs.
  3. Weirdly, I had to edit /usr/share/cmake-3.30/Modules/FindOpenMP.cmake. If your line 628 says something like:
@jlmelville
jlmelville / ubuntu-cuda-wsl2.md
Last active September 5, 2022 17:28
Fixing the legacy gpg keyring warning when installing CUDA on Ubuntu for WSL2

Ubuntu, CUDA, WSL2 and legacy keyrings

In the Ubuntu tutorial on enabling CUDA with WSL2, and specifically in part 3, you are directed to use apt-key to point Ubuntu at the correct toolkit package. On Ubuntu 22.04 this will lead to the following warning whenever you run apt-get update:

 Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
@jlmelville
jlmelville / ng20.md
Last active December 24, 2021 20:19
20 newsgroups python/R
@jlmelville
jlmelville / transitive_dependencies.R
Created November 30, 2021 16:39
transitive dependencies in R
# modified slightly from https://gist.github.com/floybix/6533090
distrib.pkgs <- library(lib.loc=R.home("library"))$results[,1]
dependencies <-
function(pkg, dependencies = c("Depends", "Imports", "LinkingTo"),
pl = installed.packages())
{
if (!(pkg %in% rownames(pl))) stop("unknown pkg ", pkg)
fields <- pl[pkg, dependencies]
@jlmelville
jlmelville / revdepcheck.R
Last active November 29, 2021 05:04
Testing reverse dependencies for R packages
install.packages("devtools")
devtools::install_github("r-lib/revdepcheck")
# may need to install bioconductor
if (!require("BiocManager", quietly = TRUE)) {
install.packages("BiocManager")
BiocManager::install(version = "3.14")
}
library(revdepcheck)
@jlmelville
jlmelville / read-and-pickle.py
Last active November 11, 2021 05:48
reading CSV into Pandas but also pickling it at the same time
import os
import pickle
from pathlib import Path
def read_data(dataset, suffix=None, dir=None, repickle=False):
"""Set `repickle=True` to rewrite pickle file if CSV has changed"""
home = str(Path.home())
if dir is None:
dataset_path = os.path.join(home, "dev", "datasets")
@jlmelville
jlmelville / zca_whiten.R
Created May 6, 2020 02:30
ZCA whitening
whitening_matrix <- function(X, eps = 1e-8, zca = FALSE) {
X <- scale(X, center = TRUE, scale = FALSE)
s <- svd(X, nu = 0)
Linvsqr <- sqrt(nrow(X) - 1) / (s$d + eps)
W <- Linvsqr * t(s$v)
if (zca) {
W <- s$v %*% W
}
@jlmelville
jlmelville / smallvis_bench.R
Created May 3, 2020 23:43
useful functions for running smallvis on multiple datasets
benchmark <- function(datasets = smallvis_datasets(), fun = smallvis::smallvis,
vfun = NULL, mfun = NULL, ...) {
varargs <- list(...)
if (is.null(varargs$ret_extra)) {
varargs$ret_extra <- TRUE
}
dataset_names <- names(datasets)
varargs_orig <- varargs
@jlmelville
jlmelville / spectral.py
Created March 14, 2020 18:00
Graph Laplacians (in Python)
import collections
import pprint
import numpy as np
import scipy as sp
import scipy.linalg
AffDeg = collections.namedtuple("AffDeg", ["W", "D"])
Lap = collections.namedtuple("Lap", ["L", "Lsym", "Lrw", "P"])
Eig = collections.namedtuple("Eig", ["vectors", "values", "lengths"])