Jim Pivarski jpivarski

First, I fetched all of the Argo data up through 2021 via ftp:

wget -r ftp://ftp.ifremer.fr/ifremer/argo

# wait a long time

tree ftp.ifremer.fr
└── ifremer
 └── argo

I managed to iterate over Awkward Arrays and rasterize the NYC buildings as polygons. The spatialpandas code is pretty well integrated with the ragged data structures you've built; there's a lot of code that twiddles offset arrays. I couldn't use the build_polygon function directly, but ported over enough of it into my own Numba-compiled function to reproduce the output.

These are Matplotlib's imshow displays of images made by iterating over Awkward Arrays; the axes are flipped from the normal longitude, latitude because I'm just dumping the array as an image, but I verified on one complex building that I am exactly reproducing spatialpandas's output (including the short-circuit code paths, in which a polygon is smaller than a pixel). The first is low-resolution and the second is high-resolution, the minimum and maximum number of pixels in the performance studies later in this email.

	# Author: Vassil Vassilev

	project(aarray-example)
	cmake_minimum_required(VERSION 3.10)

	#conda install -c conda-forge/label/llvm_rc clangdev=14.0.0.rc2
	set(conda_path "/home/jpivarski/mambaforge/envs/vassil-clang-python/")

	set(CMAKE_FIND_PACKAGE_SORT_ORDER NATURAL)
	set(CMAKE_FIND_PACKAGE_SORT_DIRECTION DEC)

	import yaml

	import awkward as ak

	arg2key = {
	"List[bool]": "np.bool_",
	"Const[List[bool]]": "np.bool_",
	"List[int8_t]": "np.int8",
	"Const[List[int8_t]]": "np.int8",
	"List[uint8_t]": "np.uint8",

	SELECT
	DATE(timestamp) AS date,
	details.system.name AS os,
	file.project AS project,
	REGEXP_REPLACE(file.version, "\\.[0123456789]{1,}$", "") AS version,
	COUNT(*) AS count
	FROM `the-psf.pypi.downloads*`
	WHERE
	_TABLE_SUFFIX BETWEEN '20160101' AND '20210224'
	AND (file.project = "rootpy" OR