Skip to content

Instantly share code, notes, and snippets.

View mtreviso's full-sized avatar

Marcos Treviso mtreviso

View GitHub Profile
@mtreviso
mtreviso / kuma.py
Last active July 31, 2022 17:36
Kuma and HardKuma distributions in JAX using distrax
"""Adapted from https://github.com/bastings/interpretable_predictions"""
import math
import distrax
import jax
import jax.numpy as jnp
EPS = 1e-6
@mtreviso
mtreviso / compute_kendall_taus.py
Last active May 31, 2023 10:19
Kendall's tau implementations as proposed in the paper "Ties Matter: Modifying Kendall's Tau for Modern Metric Meta-Evaluation": https://arxiv.org/abs/2305.14324
import numpy as np
def compute_kendall_taus(h, m):
"""
Compute multiple variants of Kendall's Tau correlations between two rank arrays.
This function calculates several variants of Kendall's Tau correlations (tau_a, tau_b, tau_c,
tau_10, tau_13, tau_14, tau_23) between two input rank arrays. It employs vectorized operations
for computation efficiency and can handle tied ranks. This function is an implementation based on
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""quotacheck.py
This script displays user quotas for a given mount point.
"""
import os
import platform
import subprocess
import sys
@mtreviso
mtreviso / psinfo.py
Created March 18, 2024 05:12
Pretty Print for Slurm `sinfo`. It uses [rich](https://github.com/Textualize/rich). You can install it globally via `sudo pip install rich`.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""psinfo.py
This script displays sinfo in a pretty table.
"""
import platform
import subprocess
import sys
try:
@mtreviso
mtreviso / psqueue.py
Created March 18, 2024 05:13
Pretty Print for Slurm `squeue`. It uses [rich](https://github.com/Textualize/rich). You can install it globally via `sudo pip install rich`.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""psqueue.py
This script displays squeue in a pretty table.
"""
import os
import subprocess
import sys
try:
@mtreviso
mtreviso / removed_unused_bibs.py
Last active June 27, 2024 11:13 — forked from TomWagg/removed_unused_bibs.py
bib reducer: Reduce and combine multiple .bib files to a single file with only *cited* entries. Works with ACL anthology.bib.
"""
>>> python remove_unused_bibs.py -h
usage: remove_unused_bibs.py [-h] [-b BIB_FILES] [-p PAPER_FILES] [-o OUT_FILE]
Remove unused bibtex entries
options:
-h, --help show this help message and exit
-b BIB_FILES, --bib_files BIB_FILES
@mtreviso
mtreviso / minimal_example_xtower_and_xcomet.ipynb
Created June 28, 2024 15:24
Explaining and Correcting xCOMET error spans with xTower
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Slurm Admin Notes: Cluster Management & QoS Setup

Prereq: Slurm is installed and slurmdbd is configured/running. Controller and nodes share the same slurm.conf (including AccountingStorageType=accounting_storage/slurmdbd) and Munge works across nodes.

If Slurm isn’t installed yet, follow: https://mtreviso.github.io/blog/slurm.html


Check or Create the Cluster (sardine-cluster)

Show clusters: