Skip to content

Instantly share code, notes, and snippets.

View robbibt's full-sized avatar
🛰️

Robbi Bishop-Taylor robbibt

🛰️
View GitHub Profile
@robbibt
robbibt / check_dataset_counts.py
Last active March 31, 2025 01:47
Check ODC dataset counts (e.g. "final", "nrt", "interim")
import pandas as pd
import datacube
dc = datacube.Datacube()
# Get table of dataset counts
product_dict = {
product: {maturity: dc.index.datasets.count(product=product, dataset_maturity=maturity)
for maturity in ["final", "nrt", "interim"]}
for product in ["ga_s2am_ard_3", "ga_s2bm_ard_3", "ga_s2cm_ard_3"]
@robbibt
robbibt / Intro_to_GeoBox_GridSpec.ipynb
Created November 29, 2024 05:49
Introduction to GeoBox and GridSpec
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@robbibt
robbibt / AEO24_DEA_workshop.md
Last active September 9, 2024 04:01
AEO 2024 DEA workshop material

AEO 2024 Digital Earth Australia workshops

To download these notebooks, please click "Download ZIP" to the top right: image

@robbibt
robbibt / load_era5.py
Last active August 20, 2024 06:56
Load ARCO-ERA5 data from Google Cloud
# Loads cloud-friendly Zarr format Analysis-Ready, Cloud Optimized ERA5 data from Google Cloud.
# Available: https://github.com/google-research/arco-era5
import xarray
import gcsfs
import odc.geo.xr
from odc.geo.geom import BoundingBox
def load_era5(
@robbibt
robbibt / voronoi_polygons_on_polygon_data.py
Last active July 2, 2024 02:02
Generating Voronoi polygons from polygon inputs (not points)
import momepy
import geopandas as gpd
# Read data and convert to projected CRS
gdf = gpd.read_file(
"/gdata1/projects/coastal/cem/sediment_compartments/compartments_checklist_final.geojson"
).to_crs("EPSG:3577")
col = "ID_Seconda"
# Create a limit for tessellation (join all polygons and buffer that)
@robbibt
robbibt / weighted_median.py
Created June 25, 2024 00:20
Weighted median in Numpy
import numpy as np
def weighted_median(values, weights):
"""
Compute the weighted median of an array of values.
This implementation sorts values and computes the cumulative
sum of the weights. The weighted median is the smallest value for
which the cumulative sum is greater than or equal to half of the
total sum of weights.
@robbibt
robbibt / dea_in_r.R
Last active July 19, 2024 01:13
Loading and analysing Digital Earth Australia Sentinel-2 data in R with `rstac` and `gdalcubes`
"""
This code demonstrates how to load Digital Earth Australia Sentinel-2 Analysis Ready Data into R.
It uses `rstac` to search for available data for a time and location using DEA's STAC endpoint,
and `gdalcubes` to load and analyse the data.
Functionality includes:
* Creating a custom pixel grid to reproject data into
* Apply a cloud mask using the "s2cloudless" cloud mask
* Combine data into seasonal composites
* Create an RGB animation
@robbibt
robbibt / stac_load_from_item.py
Created April 3, 2024 03:41
Load raster data directly from a STAC item using `odc-stac`
from pystac import Item
from odc.stac import load
import rasterio
import odc.geo.xr
path = 'https://dea-public-data-dev.s3-ap-southeast-2.amazonaws.com/derivative/ga_s2ls_intertidal_cyear_3/1-0-0/x128/y165/2016--P1Y/ga_s2ls_intertidal_cyear_3_x128y165_2016--P1Y_final.stac-item.json'
# path = 'https://dea-public-data-dev.s3-ap-southeast-2.amazonaws.com/derivative/ga_s2ls_intertidal_cyear_3/1-0-0/x128/y165/2017--P1Y/ga_s2ls_intertidal_cyear_3_x128y165_2017--P1Y_final.stac-item.json'
items = [Item.from_file(path)]
data = load(items, chunks={})
@robbibt
robbibt / STAC_loading_explorer.ipynb
Created February 29, 2024 23:28
Explorer STAC loading limit issue
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@robbibt
robbibt / s3_to_vrt.py
Created February 7, 2024 03:44
Create virtual raster for files on S3
import s3fs
import pandas as pd
s3 = s3fs.S3FileSystem(anon=True)
file_list = s3.glob(
"dea-public-data-dev/derivative/ga_ls8cls9c_gm_cyear_3/4-0-0/*/*/2023--P1Y/ga_ls8cls9c_gm_cyear_3_*_2023--P1Y_final_nbart_red.tif"
)
pd.DataFrame(file_list).iloc[:, 0].str.replace(
"dea-public-data-dev/",