Skip to content

Instantly share code, notes, and snippets.

View monkut's full-sized avatar

monkut monkut

29 followers · 1 following
View GitHub Profile
@monkut
monkut / update_awslambda_envars.bash
Created January 11, 2020 05:11
update aws lambda environment variables via awscli and jq
# the `update-function-configuration` overwrites the existing set envars.
# In order to *ADD* variables we need to read the existing envars and add to that.
# This command uses `jq` to read and transform the json result to an envar then update the lambda configuration
# create the updated envar set
export YOUR_FUNCTION_NAME={populate this}
export UPDATED_ENVIRONMNET_VARIABLES=$(aws lambda get-function-configuration --function-name ${YOUR_FUNCTION_NAME} | \
jq --compact-output ".Environment + {\"Variables\": (.Environment.Variables + {\"NEW_ENVAR_NAME\": \"NEW_ENVAR_VALUE\"})}")
# check
@monkut
monkut / boto3_secretsmanager_usage_example.py
Created November 25, 2019 00:53
import json
import boto3
secretsmgr_endpoint_url = 'http://localhost:4584'
SM_CLIENT = boto3.client('secretsmanager', endpoint_url=secretsmgr_endpoint_url)
# Set Secret
sample_secret_id = 'some:id:2'
secret_data = json.dumps({'key1': 'value1', 'key2': 'value2'})
@monkut
monkut / boto3_service_endpoints.py
Created September 20, 2019 06:00
boto3 defined default service endpoints definition for use with localstack
import os
AWS_REGION = os.getenv('AWS_DEFAULT_REGION', 'ap-northeast-1')
DEFAULT_S3_SERVICE_ENDPOINT = f'https://s3.{AWS_REGION}.amazonaws.com'
DEFAULT_SQS_SERVICE_ENDPOINT = f'https://sqs.{AWS_REGION}.amazonaws.com'
DEFAULT_SNS_SERVICE_ENDPOINT = f'https://sns.{AWS_REGION}.amazonaws.com'
AWS_SERVICE_ENDPOINTS = {
's3': os.getenv('S3_SERVICE_ENDPOINT', DEFAULT_S3_SERVICE_ENDPOINT),
@monkut
monkut / mediainfo_xml_parse.py
Created September 20, 2019 05:56
a python parser for `mediainfo` output
import json
import subprocess
from xml.etree import ElementTree
import boto3
from . import settings
S3 = boto3.client(
's3',
endpoint_url=settings.AWS_SERVICE_ENDPOINTS['s3'],
)
@monkut
monkut / tilemill_ubuntu18.04_install.md
Last active September 11, 2019 09:22
install tilemill in ubuntu 18.04

install libs

sudo apt-get install autoconf automake libtool curl make g++ unzip -y

clone and install protobuf

cd ~
@monkut
monkut / alllow-aws-crossaccount-role-access.md
Last active September 9, 2019 09:54
notes on creating an AWS cross account role

  1. Create User (User1A) in Account-A

    • Get AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

  2. Create Group in Account-A

  3. Create Role (MyRoleB) in Account-B with Permissions you want user to have:

@monkut
monkut / parse_aws_arn.py
Last active August 8, 2019 23:51
parse AWS ARN
def parse_arn(arn_str: str) -> Tuple[str, str, str, Optional[str], str, Optional[List[str]]]:
"""
https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
arn:partition:service:region:account-id:resource
arn:partition:service:region:account-id:resourcetype/resource
arn:partition:service:region:account-id:resourcetype/resource/qualifier
arn:partition:service:region:account-id:resourcetype/resource:qualifier
arn:partition:service:region:account-id:resourcetype:resource
arn:partition:service:region:account-id:resourcetype:resource:qualifier
@monkut
monkut / gist:1d00900b4a633f407cd1a00c2217666c
Created June 30, 2019 07:12
video_transfer.sh
#!/bin/bash
# Command to transfer
export VIDEO_DIRECTORY=/storage/VIDEO/HOME/$(python3 -c "import datetime;print(datetime.datetime.now().strftime('%Y%m%d-video-dump'), end='')")
echo Making directory: ${VIDEO_DIRECTORY}
mkdir ${VIDEO_DIRECTORY}
echo Copying...
rsync --progress -h /media/monkut/JVCCAM_MEM/AVCHD/BDMV/STREAM/*.MTS ${VIDEO_DIRECTORY}
@monkut
monkut / PrepareAWSClientVPN.md
Created June 14, 2019 06:58
Prepare AWS Client VPN for connecting to a VPC without NAT/IGW

Prepare AWS Client VPN README

Prerequisites

  • AWS Account
  • Linux(like) System
  • awscli
@monkut
monkut / get_s3_latest_sagemaker_model_assets_key.py
Last active March 26, 2019 07:25
"""
Get the latest model assets key from resulting sagemaker training session
"""
import boto3
S3 = boto3.client('s3')
def get_latest_model_assets_key(bucket: str, prefix: str):
model_assets = []
paginator = S3.get_paginator('list_objects')