Skip to content

Instantly share code, notes, and snippets.

View la-mar's full-sized avatar

Brock Friedrich la-mar

  • Dallas
View GitHub Profile
@la-mar
la-mar / Chamber Load from Dotenv using Python
Last active December 12, 2019 20:48
Chamber: Update SSM environment variables using a local dotenv file
# Update SSM environment variables using a local dotenv file (.env.production by default)
# Note: `chamber import` doesn't convert keys to lowercase, so that is handled in Python
python3 -c 'import json, os, dotenv; values={k.lower():v for k,v in dotenv.dotenv_values(".env.production").items()}; print(json.dumps(values))' | jq | aws-vault exec ${ENV} -- chamber import ihs -
# more about chamber: https://github.com/segmentio/chamber
@la-mar
la-mar / h3_polyfill.py
Created March 6, 2020 14:56
Export hex grids from Uber's H3 using Python
from h3 import h3
import geojson
from geojson import Feature, Polygon, FeatureCollection
from pathlib import Path
export_path = Path(".")
bbox = {
"type": "Polygon",
@la-mar
la-mar / coverage.py.css
Created April 20, 2020 21:48
Dark theme for coverage.py HTML report
/* Dark theme for coverage.py HTML report
config reference: https://coverage.readthedocs.io/en/coverage-5.1/config.html#html
original: https://github.com/nedbat/coveragepy/blob/master/coverage/htmlfiles/style.css
*/
@charset "UTF-8";
/* Licensed under the Apache License: http://www.apache.org/licenses/LICENSE-2.0 */
/* For details: https://github.com/nedbat/coveragepy/blob/master/NOTICE.txt */
@la-mar
la-mar / toml.py
Created April 25, 2020 16:18
Extract project name, version number, and other metadata from a pyproject.toml file in the project root directory
import os
from typing import Dict, Optional
import tomlkit
def _get_project_meta(pyproj_path: str = "./pyproject.toml") -> Dict[str, str]:
if os.path.exists(pyproj_path):
with open(pyproj_path, "r") as pyproject:
file_contents = pyproject.read()
@la-mar
la-mar / pydantic_handle_geometry.py
Created August 16, 2020 23:02
Load/Dump geometry objects using Pydantic + Geoalchemy
import logging
from enum import Enum
from typing import Dict, Optional, Union
import geoalchemy2.shape
from geoalchemy2 import WKBElement
from pydantic import BaseModel, Field, validator
from shapely.geometry import Point, asShape
from shapely.geometry.base import BaseGeometry
/* This query yields space count aggregates at 10 minute resolution for a given set of spaces over a three day period,,
where each 10 minute bucket is represented in the output, even if that bucket contains no data. The query first
selects data from a sparse 1 minute cagg and unions that data to a set of empty records, generated with the Postgres
generate_series function, then rolls up the unioned records into 10 minute aggregates.
The union against the set of empty records ensures that all 10 minute intervals are represented in the final results.
This step is necessary as the 1 minute data is sparse, meaning a given 10 minute interval could contain no data, and
the time_bucket_gapfill function does not register that a bucket needs to be injected if no records exist within
an interval.
-- Definitions for a set of informational views and useful procedures for working with compressed hypertables.
----------------------------------------------------------------------------------------------------
---- VIEWS ------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
drop view if exists chunks_ts, chunks_tstz, chunk_stats, jobs, job_stats, compression_stats;
---- chunks_tstz -----------------------------------------------------------------------------------