fionaRust

Dask Notes

Introduction

Dask is a flexible parallel computing library for analytic computing. Dask is composed of two components:

1. Dynamic task scheduling optimized for computation. This is similar to Airflow, Luigi, Celery, or Make, but optimized for interactive computational workloads.
2. “Big Data” collections like parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to larger-than-memory or distributed environments. These parallel collections run on top of the dynamic task schedulers.

Collections in Dask

jrleeman

from datetime import datetime
from scipy import interpolate
from siphon.catalog import TDSCatalog
from netCDF4 import Dataset
import matplotlib.pyplot as plt
from matplotlib import patheffects
import cartopy.feature as cfeat
import cartopy.crs as ccrs
import numpy as np
from metpy.plots import add_logo

rokuingh

JiaweiZhuang

dpq

#!/bin/bash
sudo apt -y install checkinstall libhdf4-dev libhdf5-dev
wget https://support.hdfgroup.org/ftp/HDF5/h4toh5/src/h4h5tools-2.2.3.tar.bz2
tar -xf h4h5tools-2.2.3.tar.bz2
cd h4h5tools-2.2.3
for f in $(rgrep -l 'include "hfile.h"' .); do sed -i 's/"hfile.h"/<hfile.h>/' $f; done
for f in $(rgrep -l 'include "mfhdf.h"' .); do sed -i 's/"mfhdf.h"/<mfhdf.h>/' $f; done
for f in $(rgrep -l 'include "hdf.h"' .); do sed -i 's/"hdf.h"/<hdf.h>/' $f; done
CFLAGS='-lmfhdf -ldf -I/usr/include/hdf' ./configure --with-hdf5=/usr/lib/x86_64-linux-gnu/hdf5/serial
make

WillKoehrsen

# selenium for web driving
import selenium
from selenium import webdriver

# time for pausing between navigation
import time

# Datetime for recording time of submission
import datetime

JiaweiZhuang

JiaweiZhuang

dkapitan

# https://stackoverflow.com/questions/35042255/how-to-plot-multiple-seaborn-jointplot-in-subplot

import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import seaborn as sns
import numpy as np


class SeabornFig2Grid():

JiaweiZhuang