Skip to content

Instantly share code, notes, and snippets.

View NegatioN's full-sized avatar

Joakim Rishaug NegatioN

View GitHub Profile
@NegatioN
NegatioN / binary_search.pyx
Last active April 17, 2020 16:08
Binary search in Cython
#uncomment to run in a notebook
#%load_ext Cython
%%cython
cdef int round(int n, int d):
return (n + d // 2) // d
def search(it:int, target) -> bool:
cdef int first = 0
cdef int last = len(target)
cdef int middle = round(first+last, 2)
@NegatioN
NegatioN / help.py
Created November 13, 2020 09:48
help.py
df = ### assumes you have a df
category_column = 'main_category'
number = 1000
dfz = []
for cat in df[category_column].unique():
dfz.append(df[df[category_column]==cat].sample(n=number))
final_df = pd.concat(dfz)
@NegatioN
NegatioN / kwargs_pytorch.cpp
Last active December 1, 2020 16:30
How to input python-esque kwargs to a pytorch scripted c++ model
torch::jit::IValue myIvalue;
torch::jit::IValue myIvalue2;
torch::jit::script::Module module;
std::unordered_map<std::string, torch::jit::IValue> umap = {{"x", myIvalue}, {"opt", myIvalue2}};
auto result = module.get_method("forward")({}, umap);
// shows all potential arguments to model forward
std::cout << module.get_method("forward").function().getSchema().arguments() << std::endl;
@NegatioN
NegatioN / install.sh
Last active October 11, 2024 16:28
Installing Canon Printers Ubuntu
# Origin http://ubuntuhandbook.org/index.php/2020/05/canon-printer-scangear-mp-ubuntu-20-04/
# Ubuntu
sudo apt install printer-driver-gutenprint
sudo add-apt-repository ppa:thierry-f/fork-michael-gruz
sudo apt install scangearmp2
sudo apt install scangearmp-mp495series
# Arch / Manjaro
@NegatioN
NegatioN / rename.sh
Created November 30, 2021 19:37
Rename sed script
for f in *; do NEW_NAME=$(echo $f | sed -e "s/.*EP\([0-9][0-9]\).*/S06E\1.mkv/g"); mv ${f} ${NEW_NAME}; done;
@NegatioN
NegatioN / vim.txt
Last active January 1, 2025 11:19
Vim stuff
\%23c Matches in a specific column. (23), example :%s/\%1c/#/ (insert # on first column of all rows)
:% (all rows), :. (this row), :$ (last row), :.,$ (this->last row), :5,$ (5th->last row)
s, substitute. :%s/a/b/g, change all a->b on all rows.
u, undo
p, paste
m+a, set mark a [works for a-z]
'+a, go to mark a [works for a-z]
<<, de-indent row
>>, indent row
J, join rows
@NegatioN
NegatioN / t4r.py
Last active January 13, 2022 11:39
MVP Inconsistent output shape
'''
#Conda environment file (env.yml). Install with `conda env create -f env.yml`
channels:
- nvidia
- rapidsai
- anaconda
- conda-forge
dependencies:
- python=3.8
@NegatioN
NegatioN / script.sh
Created March 31, 2022 09:34
Debugging segfaults in Python
python -q -X faulthandler -c "#Faulty Code"
@NegatioN
NegatioN / scriptable_minilm_tokenizer.py
Created May 12, 2022 13:26
A Torchscript-able MiniLM Tokenizer
class MiniLMTokenizer(torch.nn.Module):
def __init__(self, vocab: Dict[str, int], vocab_scores: Dict[str, float]):
'''
:param vocab: A dictionary mapping from string to index of token
:param vocab_scores: A dictionary mapping from string to the score of a given token. This is used to decide
which tokenization is most probable for our input string.
For unigram models this should be avaiable under `~/.cache/torch/sentence_transformers/$YOURMODEL/unigram.json`
You might also need to flip the scores if they're negative.
```
with open(f'~/.cache/torch/sentence_transformers/{org}_{model_name}/unigram.json', 'r') as f:
@NegatioN
NegatioN / dedup.py
Created August 2, 2022 17:08
Delete all but one of jpgs taken at the same second
from PIL import Image, ExifTags
from collections import defaultdict
from datetime import datetime
import os
from tqdm import tqdm
from glob import glob
import sys
glob_string = sys.argv[1]
print(f'Searching in globstring: {glob_string}')