Skip to content

Instantly share code, notes, and snippets.

@SQLadmin
SQLadmin / aws-athena-auto-partition-between-dates.py
Last active March 23, 2023 21:08
AWS Athena automatically add partitions for given two dates for cloudtrail logs via lambda / Python
# Lambda function / Python to create athena partitions for Cloudtrail log between any given days.
# If you run this in AWS Lambda then it can't able to ceate all the partitions.
# Because lambda can run any functions up to 5mins. But create partition query will take avg 6 secs.
# I did a benchmark on lambda, it created upto 3 months of partitions on 16 regions.
'''
-----------------------------------------------------------------
AWS Athena Create Partitions Automatically For Given Any TWO DAYS
-----------------------------------------------------------------
Version 1.0
@tongpu
tongpu / aws-open-role-console.py
Created May 25, 2018 12:43
Open the AWS console with an AWS role secured by `ExternalID`
#!/usr/bin/env python
import click
import configparser
import json
import os
import requests
import webbrowser
from boto.sts import STSConnection
@eagleEggs
eagleEggs / plotly_offline_graph_cli
Last active August 11, 2018 02:42
plotly offline graph from command line
Setting up the following will enable running a notebook from CLI and create a plotly image from the generated graph:
CLI:
sudo jupyter nbconvert --execute notebook.ipynb
Python:
At top of notebook: offline.init_notebook_mode()
After defining data and figure:
offline.iplot(fig, filename='graph', image='png')
"""An AWS SNS example."""
import logging
import time
import boto3
logging.basicConfig(
level=logging.INFO,
format='[%(asctime)s] %(levelname)s:%(name)s:%(message)s'
@jwieringa
jwieringa / aws_inspector.md
Last active November 15, 2019 07:09
Exploring running AWS Inspector on CoreOS Container Linux

AWS Inspector Support in a Containerized environment

Summary

  • If behavioral analysis is required, then the kernel module is required
  • The inspector agent cannot be compiled for container linux because the source code is not open
  • The inspector agent is dynamically linked and therefore must be run on a supported filesystem/OS
  • Therefore, the inspector agent must be run inside of a container
  • Once the inspector agent in the container needs to have the correct mounts and capabilites to scan the host systems
@kojiisd
kojiisd / file0.py
Last active May 20, 2018 09:39
Amazon CloudWatch Metric MathをLambdaから使ってみた ref: https://qiita.com/kojiisd/items/3f6b33bd5ca5f84034ae
response = client.get_metric_data(
MetricDataQueries=[
{
'Id': 'm1',
'MetricStat': {
'Metric': {
'Namespace': 'AWS/DynamoDB',
'MetricName': 'ProvisionedReadCapacityUnits',
'Dimensions': [
{
@copley
copley / aws describe-services --query "services[*].[code,name]" --region us-east-1
Created April 20, 2018 03:44
aws describe-services --query "services[*].[code,name]" --region us-east-1
AWS PRODUCTS
Amazon EC2
Amazon Elastic Container Registry
Amazon Elastic Container Service
AWS Lambda
Amazon Virtual Private Cloud
AWS Elastic Beanstalk
Auto Scaling
Elastic Load Balancing
Amazon S3
# -*- coding: utf-8 -*-
import os, boto3, botocore
S3BUCKET=os.environ.get('S3BUCKET', None)
S3PREFIX=os.environ.get('S3PREFIX', 'rds/')
LASTRECEIVEDFILENAME=os.environ.get('LASTRECEIVEDFILENAME', '.timestamp')
def lambda_handler(event, context):
log_file_data = ""
@adrianhall
adrianhall / CognitoUserPool.yaml
Created April 13, 2018 22:02
Some resources to generate a CloudFormation template for Amazon Cognito user pools
UserPool:
Type: "AWS::Cognito::UserPool"
Description: "A Cognito user pool for authenticating users"
Properties:
UserPoolName: !Sub ${APIName}-user-pool
AutoVerifiedAttributes:
- phone_number
MfaConfiguration: "ON"
SmsConfiguration:
ExternalId: !Sub ${APIName}-external
@adrianhall
adrianhall / AppSyncAPI.yaml
Last active September 25, 2024 01:43
A CloudFormation template for DynamoDB + Cognito User Pool + AppSync API for the Notes tutorial
---
Description: AWS AppSync Notes API
Parameters:
APIName:
Type: String
Description: Name of the API - used to generate unique names for resources
MinLength: 3
MaxLength: 20
AllowedPattern: '^[a-zA-Z][a-zA-Z0-9_]*$'