Skip to content

Instantly share code, notes, and snippets.

View jpivarski's full-sized avatar

Jim Pivarski jpivarski

View GitHub Profile
@jpivarski
jpivarski / Awkward Arrays in spatialpandas.md
Last active June 21, 2022 18:32
Awkward Arrays in spatialpandas

I managed to iterate over Awkward Arrays and rasterize the NYC buildings as polygons. The spatialpandas code is pretty well integrated with the ragged data structures you've built; there's a lot of code that twiddles offset arrays. I couldn't use the build_polygon function directly, but ported over enough of it into my own Numba-compiled function to reproduce the output.

These are Matplotlib's imshow displays of images made by iterating over Awkward Arrays; the axes are flipped from the normal longitude, latitude because I'm just dumping the array as an image, but I verified on one complex building that I am exactly reproducing spatialpandas's output (including the short-circuit code paths, in which a polygon is smaller than a pixel). The first is low-resolution and the second is high-resolution, the minimum and maximum number of pixels in the performance studies later in this email.

@jpivarski
jpivarski / argo-demo.ipynb
Last active April 20, 2022 02:09
Argo-Awkward Array demo
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jpivarski
jpivarski / CMakeLists.txt
Last active March 31, 2022 12:01
Very provisional ctypes interface to Clang Incremental
# Author: Vassil Vassilev
project(aarray-example)
cmake_minimum_required(VERSION 3.10)
#conda install -c conda-forge/label/llvm_rc clangdev=14.0.0.rc2
set(conda_path "/home/jpivarski/mambaforge/envs/vassil-clang-python/")
set(CMAKE_FIND_PACKAGE_SORT_ORDER NATURAL)
set(CMAKE_FIND_PACKAGE_SORT_DIRECTION DEC)
@jpivarski
jpivarski / kerchunk-root-buffer-layout.svg
Created February 17, 2022 02:06
Description of the structure of a ROOT TBasket for the kerchunk meeting
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jpivarski
jpivarski / avro-to-awkward.ipynb
Created January 11, 2022 21:18
Converting Avro Schemas into Awkward Forms
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jpivarski
jpivarski / final-notebook.ipynb
Last active August 11, 2021 11:58
Replaced vibrating string's overtones with hydrogen atom in https://gist.github.com/endolith/3066664
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jpivarski
jpivarski / assign-all-argtypes.py
Created May 26, 2021 21:34
Assigning ctypes argument types to all Awkward Array kernels from the YAML specification
import yaml
import awkward as ak
arg2key = {
"List[bool]": "np.bool_",
"Const[List[bool]]": "np.bool_",
"List[int8_t]": "np.int8",
"Const[List[int8_t]]": "np.int8",
"List[uint8_t]": "np.uint8",
SELECT
DATE(timestamp) AS date,
details.system.name AS os,
file.project AS project,
REGEXP_REPLACE(file.version, "\\.[0123456789]{1,}$", "") AS version,
COUNT(*) AS count
FROM `the-psf.pypi.downloads*`
WHERE
_TABLE_SUFFIX BETWEEN '20160101' AND '20210224'
AND (file.project = "rootpy" OR
@jpivarski
jpivarski / matplotlib-awkward.ipynb
Created July 29, 2020 16:36
Matplotlib should fail for unflattened arrays (as it does here)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Requires scikit-hep/uproot4#35.

>>> import uproot4
>>> tbranchelement1 = uproot4.open("Event.root : T/fH")
>>> tbranchelement1
<TBranchElement 'fH' at 0x7f5ef03c92d0>
>>> tbranchelement1.interpretation
AsObjects(Model_TH1F)
>>> tbranchelement1.array(library="np", entry_stop=5)