Skip to content

Instantly share code, notes, and snippets.

@seahrh
seahrh / mypy.ini
Created April 6, 2020 04:00
mypy config
[mypy]
python_version = 3.7
warn_return_any = True
warn_unused_configs = True
[mypy-apache_beam.*]
ignore_missing_imports = True
[mypy-google.cloud.*]
ignore_missing_imports = True
@seahrh
seahrh / cloudbuild.yaml
Created April 6, 2020 04:03
cloudbuild minimal working example: tests, build docker and push to gcr
steps:
- id: 'tests'
name: 'python:3.7-slim-buster'
args: ['bash', './scripts/tests.sh']
- id: 'docker_build'
name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', '${_IMAGE_NAME}:${_IMAGE_TAG}', '.']
images: ['${_IMAGE_NAME}:${_IMAGE_TAG}']
substitutions:
@seahrh
seahrh / Dockerfile
Created April 6, 2020 04:05
fastapi Dockerfile
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
WORKDIR /app
EXPOSE 80
COPY app app
COPY *.py ./
COPY *.ini ./
RUN python --version \
&& python -m pip install --upgrade pip setuptools wheel \
@seahrh
seahrh / install.sh
Created April 6, 2020 04:11
Helm deploy microservice to gke, use Helm image from google cloud community builders
#!/bin/bash
set -e
set -x #echo on
# entrypoint requires the following variables to be set
case $ENVIRONMENT in
"prod")
export CLOUDSDK_COMPUTE_ZONE="prod"
export CLOUDSDK_CONTAINER_CLUSTER="prod"
export GCLOUD_PROJECT="prod"
@seahrh
seahrh / cloudbuild.yaml
Created April 6, 2020 04:14
cloudbuild cicd single pipeline: test, build, deploy
steps:
- id: 'tests'
name: 'python:3.7-slim-buster'
args: ['bash', './scripts/tests.sh']
- id: 'build'
name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', '${_IMAGE_NAME}:${BRANCH_NAME}', '.']
- id: 'push'
name: 'gcr.io/cloud-builders/docker'
args: ['push', '${_IMAGE_NAME}:${BRANCH_NAME}']
@seahrh
seahrh / ex_merge_key_airflow.yaml
Created April 6, 2020 04:57
Reusing data with YAML Anchors, Aliases and Merge Keys
defaults:
envs: &envs_defaults
ENVIRONMENT: prod
DATE: "{{ ds }}"
task: &task_defaults
type: my_type
envs: *envs_defaults
tasks:
@seahrh
seahrh / istio-prometheus.txt
Created April 7, 2020 01:28
Istio prometheus queries for RED metrics on grafana
histogram_quantile(0.50, sum(rate(istio_request_duration_seconds_bucket{reporter="source",destination_service=~"$service"}[1m])) by (le))
round(sum(rate(istio_requests_total{reporter="source",destination_service=~"$service"}[1m])), 0.1)
-- success rate should be replaced with error rate
sum(rate(istio_requests_total{reporter="source",destination_service=~"$service",response_code!~"5.*"}[1m])) / sum(rate(istio_requests_total{reporter="source",destination_service=~"$service"}[1m]))
-- alerts P99
histogram_quantile(0.99, sum(rate(istio_request_duration_seconds_bucket{reporter="source",destination_service="my_service"}[60m])) by (le))
@seahrh
seahrh / latency.txt
Created April 10, 2020 06:29 — forked from jboner/latency.txt
Latency Numbers Every Programmer Should Know
Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns 3 us
Send 1K bytes over 1 Gbps network 10,000 ns 10 us
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
#
# stopwords.txt
#
# Freely available stopword list, balancing coverage and size.
#
# From http://www.lextek.com/manuals/onix/stopwords1.html
a
about
above
across
i
me
my
myself
we
our
ours
ourselves
you
your