Skip to content

Instantly share code, notes, and snippets.

View underchemist's full-sized avatar

Sebastien Tremblay-Johnston underchemist

10 followers · 1 following
View GitHub Profile
@underchemist
underchemist / projsync-no-ssl-check.bat
Created June 23, 2020 18:14
Disabling ssl checks while using projsync
rem https://github.com/OSGeo/PROJ/blob/475e7fd9f11af184e7e1e1618b3e1bb110a79714/src/networkfilemanager.cpp#L1613
set PROJ_UNSAFE_SSL=YES
projsync --list-files
@underchemist
underchemist / rasterio-warp-with-rpcs-example.py
Created November 5, 2020 02:25
rasterio warp with rpcs example
"""
RPC_DEM_SRS only used in GDAL >= 3.2.0, otherwise ignored
"""
import rasterio
import rasterio.warp
from rasterio import logging
logging.basicConfig(filename='rasterio.log', level=logging.DEBUG)
@underchemist
underchemist / python39dllloading
Last active November 16, 2020 05:36
Python 3.9 DLL loading change
>>> import os
>>> os.add_dll_directory('C:\path\to\dll\dir')
>>> import rasterio
@underchemist
underchemist / wheel_repair.py
Last active November 16, 2020 16:01
Windows version of auditwheel adapted from @vinayak-mehta and @duburcqa
#!/usr/bin/env python
# This tool has been copied from https://github.com/vinayak-mehta/pdftopng/blob/main/scripts/wheel_repair.py
# and extended to supported hierarchical folder architecture with mulitple .pyd
# to update, and to move all the DLL in a common folder *package*.lib installed
# jointly with the package itself, similarly to auditwheel on Linux platform.
#(see also https://discuss.python.org/t/delocate-auditwheel-but-for-windows/2589/9).
# Additionally adapted from https://github.com/Wandercraft/jiminy/blob/648c0ec918ca2c2f734a32bada1dbfaf6226de48/build_tools/wheel_repair.py
# added sorting of pyd and dll files such that repaired dlls pointing to their own
@underchemist
underchemist / ipython-sqlite3.gotcha
Created October 31, 2022 19:35
WIN GOTCHA: IPython and sqlite3.dll
If you are working with python module that depends on a custom sqlite3.dll, IPython on startup will load sqlite3.dll from the installation's vendored DLLs. This can cause a conflict resulting in a failed DLL load since the module that needs the custom sqlite3.dll cannot override the currently loaded DLL (DLL HELL).
WORKAROUND: copy the custom sqlite3.dll to the installation location
@underchemist
underchemist / e84-tech-radar-tools.md
Created October 10, 2024 04:53
e84 tech radar - tools

Geopandas

  • Moved from trial to adopt
  • You're probably already familiar, but recent under-the-hood upgrades to IO (pyogrio over fiona) and geometry functionality (shapely 2.0), both of which are aimed at better vectorized operations.
  • In practice, things are more streamlined and potentially you spend less of your life trying to load a 20 GB GeoJSON into a DataFrame.

pyogrio and shapely

  • pyogrio is optimized for bulk reading/writing of spatial vector data

Pyogrio is fast because it uses pre-compiled bindings for GDAL/OGR to read and write the data records in bulk. This approach avoids multiple steps of converting to and from Python data types within Python, so performance becomes primarily limited by the underlying I/O speed of data source drivers in GDAL/OGR. We have seen >5-10x speedups reading files and >5-20x speedups writing files compared to using row-per-row approaches (e.g. Fiona).

@underchemist
underchemist / rasterio-install-from-source-vcpkg-GDAL.md
Created November 5, 2024 17:27
Install rasterio from source using vcpkg for GDAL on windows

Install GDAL

Install GDAL and its dependencies as a release version using triplet. This will install to %VCPKG_ROOT%\installed\<triplet>

vcpkg install --triplet=x64-windows-release gdal[tools,webp,zstd,sqlite3,postgresql,openjpeg,png,geos,libspatialite,curl]

Install Rasterio

Virtual Environment