Skip to content

Instantly share code, notes, and snippets.

View ashiklom's full-sized avatar

Alexey Shiklomanov ashiklom

View GitHub Profile
@ashiklom
ashiklom / streaming.py
Created March 19, 2025 19:42
Streaming GMAO data using fsspec and Xarray
#!/usr/bin/env python
import fsspec
from fsspec.implementations.http import HTTPFile, HTTPFileSystem
import xarray as xr
url = "https://portal.nccs.nasa.gov/datashare/gmao/geos-fp/das/Y2025/M03/D18/GEOS.fp.asm.inst1_2d_lfo_Nx.20250318_0000.V01.nc4"
fs = HTTPFileSystem()
file = fs.open(url)
@ashiklom
ashiklom / codecs.py
Created October 8, 2024 13:20
Usage of numcodecs
#!/usr/bin/env python
import numpy as np
import numcodecs
shuf = numcodecs.Shuffle()
zlib = numcodecs.Zlib(level=9)
# generate some random data
dat = np.random.randn(10_000)
@ashiklom
ashiklom / conda-environment.yml
Last active August 2, 2023 14:58
Testing H5Coro
name: h5coro
channels:
- conda-forge
dependencies:
- gdal
- ipython
- proj
- pybind11
- python
@ashiklom
ashiklom / rechunk.sh
Created June 30, 2023 22:15
Re-chunking GOES data
#!/usr/bin/env bash
source ~/.bash_functions
mod_py39
conda activate vdp-common
FILES=$(find css-geo/GOES-16-ABI-L1B-FULLD/2022/203 -type f)
# Define directory where outptus will be saved
OUTDIR=GOES-16-ABI-L1B-FULLD-rc1000
@ashiklom
ashiklom / obdaac-download.sh
Created June 29, 2023 20:39
Query and download data from OB DAAC
#!/usr/bin/env bash
# Prerequisite:
#
# (1) Create a `~/.netrc` file (if it doesn't exist); e.g., `touch ~/.netrc`
#
# (2) Open the `~/.netrc` file in a plain text editor and add a single line like the following:
#
# machine urs.earthdata.nasa.gov login [email protected] password YourEDLPassword123
#
@ashiklom
ashiklom / goes-nasa.py
Created June 22, 2023 17:26
Kerchunk-ing GOES data
from dask.distributed import Client, LocalCluster, progress
cluster = LocalCluster()
client = Client(cluster)
import s3fs
import itertools
from fsspec.implementations.local import LocalFileSystem
import dask
from dask.delayed import delayed
@ashiklom
ashiklom / h5-features
Created June 16, 2023 18:30
HDF5 (or at least, H5) calls found anywhere in the NetCDF library
H5Aclose
H5Acreate1
H5Adelete
H5Aexists
H5Aget_name
H5Aget_num_attrs
H5Aget_space
H5Aget_storage_size
H5Aget_type
H5A_info_t
@ashiklom
ashiklom / turing-canopy-optics.jl
Created May 10, 2023 20:10
Experimenting with PROSPECT inversion using Turing.jl
using CanopyOptics
using Unitful
using Turing
using Optim
using Distributions
using ForwardDiff
using Plots
using StatsPlots
using FillArrays
using LinearAlgebra
@ashiklom
ashiklom / zarr-chunks.py
Created March 8, 2023 17:58
Experimenting with Zarr chunking
import xarray as xr
import numpy as np
d1 = np.random.rand(133, 10, 10)
t1 = np.arange(1, 134)
xs = np.arange(1, 11)
ys = np.arange(1, 11)
chunking = {"time": 100, "x": 10, "y": 10}
@ashiklom
ashiklom / aws-mfa
Last active April 9, 2025 14:20
AWS MFA script
#!/bin/bash
if (return 0 2> /dev/null)
then
: # Pass
else
echo 'This script must be `source`d, not executed!'
exit 255
fi