bbuechler

Deploy Cumulus Config

This bash script is intended to demonstrate how to programmatically deploy config (collections, providers, rules) to cumulus via a proxy through the bastion host

Environment

Most of the ENV is in cumulus' app/.env file. However, there are some non-standard configs assumed:

AWS Env:

bbuechler

MacOS/Linux - console and Firefox

Generate AWS Long Term Access keys for the appropriate CloudTamer Project (like asf-application-cumulus-uat-7211)

Save the keys to your ~/.aws/credentials:

[cumulus-uat-appdev]
aws_access_key_id = AKIAVA2VMIABCDEFGH12
aws_secret_access_key = 35IkW2dkPUsg5St4ioKB9MKXZjYYJ3lpNI527pXXYXXFIds8so

bbuechler

# Upload your key to NGAP Bastion Self-serve bucket
# https://wiki.earthdata.nasa.gov/display/ESKB/SSH+Bastion+Key+Upload+-+Self-Service
export key_bucket=$(aws s3 ls | xargs -n1 echo | grep ngap-bastion-authorized-public-keys)
export my_ssh_key="/path/to/.ssh/<your_private_key>"
aws s3 cp $my_ssh_key.pub s3://$key_bucket/

# Optional Params with Reasonable Defaults that you may want to change
export bastion_name="SSM Bastion"
export instance_type="t2.micro"

bbuechler

import logging
import statistics
from math import sqrt

from shapely import wkt
from shapely.ops import linemerge, unary_union, polygonize
from shapely.geometry import Polygon

# Move the box out of earth coordinate system
def shift_polygon(bbox):

bbuechler

Cloud-based URL disambiguator

(esentially a distributed load-balancer)

There needs to be an API Gateway w/ a /{proxy+} GET method.

Important Config

• HOSTS List hosts where a request may be serviced

HOSTS = [ 'host1.path.com', 'host2.path.com', ... 'hostX.path.com']

bbuechler

import os
import time
import json
import argparse
from urllib.parse import urlencode
from urllib.request import Request, urlopen
from statistics import mean
from random import randrange

MIN_DELAY=0.5

bbuechler

Onion Peeler

Setup

Create a lambda with these two files

Config

Enviroment

bbuechler

# Stupid CloudFormation doesn't support this right now
TeaStackName=<TEA>-UAT-TEA
TeaLambdaName=$(aws cloudformation describe-stacks  --query "Stacks[?StackName=='$TeaStackName'].Outputs" | jq -r '.[][] | select ( .OutputKey == "EgressLambdaName" ).OutputValue')
TeaLambdaLogFileName=$(echo -n "/aws/lambda/$TeaLambdaName")

dimensions='dimensions={service=$.message.timing.service}'
transformation=$(echo metricName=duration,metricNamespace=TEA,metricValue='$.message.timing.duration',$dimensions,unit=Milliseconds)

aws logs put-metric-filter \
        --log-group-name $TeaLambdaLogFileName \

bbuechler

import os
import base64
import logging
import json
from urllib import request
from urllib.request import Request, urlopen
from urllib.parse import urlencode
from urllib.error import HTTPError
from http import cookiejar
from remotezip import RemoteZip

bbuechler

import shapely.wkt

#wkt = 'POLYGON ((-164.488235 53.59074, -163.863495 55.320713, -167.797485 55.721722, -168.259628 53.987499, -164.488235 53.59074))'
wkt = 'POLYGON ((-166.071014 55.533596, -165.424973 53.755867, -161.644806 54.152309, -162.122131 55.934521, -166.071014 55.533596))'

# Load polygon and order the points by LATITUDE
poly = shapely.wkt.loads(wkt)
points = poly.exterior.coords[:4]
points.sort(key = lambda x: x[1])