Skip to content

Instantly share code, notes, and snippets.

View GenevieveBuckley's full-sized avatar

Genevieve Buckley GenevieveBuckley

  • Monash University
  • Melbourne
View GitHub Profile
@GenevieveBuckley
GenevieveBuckley / nuke-and-reinstall-conda.sh
Created October 12, 2021 23:18
Nuke and reinstall conda script
#!/bin/bash
# Fernando's nuke and reinstall conda script
# https://twitter.com/fperez_org/status/1447996737063317512
# Download and install miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -O ~/Downloads/miniconda.sh
bash ~/Downloads/miniconda.sh -b -p $HOME/local/conda
conda config --add channels conda-forge
@GenevieveBuckley
GenevieveBuckley / match_dask_keys.py
Created September 9, 2021 05:40
Mater Dask keys, ignoring token strings
def match_key(key, keylist):
# key to match
key = list(key)
key[0] = key[0].rsplit('-', 1)[0] # remove token
# search for matches
for k in keylist:
temp = list(k)
temp[0] = temp[0].rsplit('-', 1)[0] # remove token
if temp == key:
return tuple(k)
@GenevieveBuckley
GenevieveBuckley / 2021.ipynb
Created September 6, 2021 07:21
Dask survey 2021 analysis
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GenevieveBuckley
GenevieveBuckley / correct_tensordot_auto_rechunkiing.py
Last active July 27, 2021 09:47
Tensordot auto-rechunking experiments
import dask
import dask.array as da
import numpy as np
def _inner_axes(a_ndim, b_ndim, axes):
"""Given tensordot axes argument, return list of axes to sum over."""
if isinstance(axes, (int, float)):
if axes == 0:
inner_axes_a = []
def _tensordot_shape_output(a, b, axes):
if isinstance(axes, (int, float)):
if axes == 0:
shape_out = a.shape + b.shape
chunks_out = a.chunks + b.chunks
elif axes > 0:
shape_out = a.shape[:axes-1] + b.shape[axes-1:]
chunks_out = a.chunks[:axes-1] + b.chunks[axes-1:]
else:
axes_a, axes_b = axes
@GenevieveBuckley
GenevieveBuckley / alternative_to_scipy_ndi_findobjects.py
Last active July 23, 2021 06:51
Slightly simpler way to get the array location for any dask block
import numpy as np
import pandas as pd
def _find_slices(x):
"""An alternative to scipy.ndi.find_objects"""
unique_vals = np.unique(x)
unique_vals = unique_vals[unique_vals != 0]
result = {}
for val in unique_vals:
@GenevieveBuckley
GenevieveBuckley / map-overlap-length-without-materializing.ipynb
Created June 28, 2021 13:27
map-overlap-length-without-materializing-the-task-graph
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GenevieveBuckley
GenevieveBuckley / combine_slicing.py
Last active June 28, 2021 12:58
combine_slicing.py
import math
import numpy as np
import pytest
def combine_slices(slices):
starts = [s.start for s in slices if s.start is not None]
stops = [s.stop for s in slices if s.stop is not None]
steps = [s.step for s in slices if s.step is not None]
@GenevieveBuckley
GenevieveBuckley / plot.py
Created June 23, 2021 08:15
Simple matplotlib line plot with error bars
import matplotlib.pyplot as plt
%matplotlib inline
plt.errorbar(
[1,2,3],
[1,2,3],
yerr=[.1, .2, .3],
color='blue', marker='o', linestyle='dashed',
)
plt.title("Example")
@GenevieveBuckley
GenevieveBuckley / highlevelgraph-html.py
Last active June 4, 2021 09:19
HighLevelGraph HTML
from html import escape
class HighLevelGraphHTML():
def __init__(self, highlevelgraph):
self.highlevelgraph = highlevelgraph
pass
def _repr_html_(self):
highlevelgraph = self.highlevelgraph