e84 tech radar - tools

e84-tech-radar-tools.md

Geopandas

• Moved from trial to adopt
• You're probably already familiar, but recent under-the-hood upgrades to IO (pyogrio over fiona) and geometry functionality (shapely 2.0), both of which are aimed at better vectorized operations.
• In practice, things are more streamlined and potentially you spend less of your life trying to load a 20 GB GeoJSON into a DataFrame.

pyogrio and shapely

• pyogrio is optimized for bulk reading/writing of spatial vector data

Pyogrio is fast because it uses pre-compiled bindings for GDAL/OGR to read and write the data records in bulk. This approach avoids multiple steps of converting to and from Python data types within Python, so performance becomes primarily limited by the underlying I/O speed of data source drivers in GDAL/OGR. We have seen >5-10x speedups reading files and >5-20x speedups writing files compared to using row-per-row approaches (e.g. Fiona).

• shapely 2.0 brings array/vectorized functionality from pygeos, and in general better interoperability with numpy

uv

• I suggested this to Greg a few weeks ago and also spent some time tinkering with and enjoying it
• Manage python environments, multiple python versions, and project/app dependency management in one tool with minimal overhead
• Avoids bootstrapping issues that some other tools like poetry have that require python to already be installed, compatible with python installs from other distributions (conda, system, etc.)
• fast

AI Coding assitants

• My only experience is from enabling copilot through Sparkgeo org
• I've rarely used prompt, mostly just ocassionally accepting the autosuggestion if it seems mostly in line with what I was planning on writing
• Ethical/licencing concerns? GDAL RFC on stance about contributions made with the help from generative AI

kepler.gl

• Geospatial data analysis/visualization framework using MapLibre GL and deck.gl
• related: lonboard, deck.gl rendering of spatial data in Jupyter notebooks

Dovetail

• Notion-like (ai integration, rich document editing) designed for client/user interviews and notetaking.
• Looks flashy, I imagine it can be quite useful depending on how much effort you put into it, but probably costs a lot of money

Keeping an eye on

• duckdb
• rastervision/torchgeo