This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT | |
| * | |
| FROM | |
| read_parquet ('s3://.../**/*.parquet') | |
| WHERE | |
| country = 'PL' -- hive partition filter | |
| AND h3_res_3 IN (590501584509599743) -- hive partition filter | |
| AND ( -- statistics filter pushdown from compacted cells | |
| h3_res_12 BETWEEN 631033929750544895 AND 631033929750547967 -- H3 parent cell 626530330123177983 | |
| OR h3_res_12 BETWEEN 631033929750548991 AND 631033929750552063 -- H3 parent cell 626530330123182079 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def compacted_h3_cells_to_ranges( | |
| compacted: list[int], h3_resolution: int = 12 | |
| ) -> list[tuple[int, int]]: | |
| """ | |
| Compacted H3 cells -> list of (r_min, r_max) integer ranges. | |
| Each pair tightly brackets all descendants of the given cell at the same resolution. | |
| Ranges across cells are disjoint (H3 bit layout property). | |
| """ | |
| RES_SHIFT = 52 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import math | |
| from itertools import pairwise | |
| import geopandas as gpd | |
| import numpy as np | |
| from shapely import LineString, Point, Polygon, distance | |
| from shapely.coords import CoordinateSequence | |
| def locate_farthest_intersection_point( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| import geopandas as gpd | |
| # keep a geodataframe of all points creating the edges | |
| all_points = gpd.GeoSeries( | |
| clipped_edges_gdf.get_coordinates(ignore_index=True).apply( | |
| lambda row: Point(row["x"], row["y"]), axis=1 | |
| ), | |
| crs=4326, | |
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # this is a code snippet, whole logic with checks is in the final notebook | |
| subgraph_edges = ox.graph_to_gdfs(subgraph, nodes=False, edges=True) | |
| # find all endpoints and check their edges outside. Clip edges exactly at the distance point. | |
| edges_to_clip = {} | |
| for node in set(subgraph.nodes).union([center_node]): | |
| # iterate all edges starting from this node | |
| for u, v, data in G.edges(node, keys=False, data=True): | |
| # if whole edge is inside clipped subgraph - skip it | |
| if v in subgraph: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- Select all pubs in the Great Britain and calculate how many pubs are in a 5 km radius around each pub. | |
| WITH pubs AS ( | |
| -- irish_pub....................................eat_and_drink > bar > irish_pub | |
| -- pub..........................................eat_and_drink > bar > pub | |
| SELECT | |
| id, | |
| JSON_EXTRACT_PATH_TEXT(names, 'primary') as name, | |
| JSON_EXTRACT_PATH_TEXT(categories, 'primary') as category, | |
| confidence, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| WITH division_ids AS ( | |
| SELECT | |
| id, | |
| CASE | |
| WHEN country = 'DE' THEN 'Berlin' | |
| WHEN country = 'FR' THEN 'Paris' | |
| ELSE 'London' | |
| END as city_name | |
| FROM | |
| OVERTURE_MAPS__DIVISIONS.CARTO.DIVISION |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!doctype html> | |
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8" /> | |
| <title>Trip animation – sequences (deck.gl + MapLibre)</title> | |
| <meta name="viewport" content="width=device-width,initial-scale=1" /> | |
| <link href="https://unpkg.com/maplibre-gl@5.12.0/dist/maplibre-gl.css" rel="stylesheet" /> | |
| <!-- <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons" /> --> | |
| <script src="https://unpkg.com/maplibre-gl@5.12.0/dist/maplibre-gl.js"></script> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import multiprocessing | |
| from pathlib import Path | |
| from queue import Queue | |
| from time import sleep | |
| from typing import Callable | |
| import pyarrow as pa | |
| import pyarrow.parquet as pq | |
| from tqdm import tqdm |