This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Lambda function / Python to create athena partitions for Cloudtrail log between any given days. | |
# If you run this in AWS Lambda then it can't able to ceate all the partitions. | |
# Because lambda can run any functions up to 5mins. But create partition query will take avg 6 secs. | |
# I did a benchmark on lambda, it created upto 3 months of partitions on 16 regions. | |
''' | |
----------------------------------------------------------------- | |
AWS Athena Create Partitions Automatically For Given Any TWO DAYS | |
----------------------------------------------------------------- | |
Version 1.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Lambda function to create partition for Cloudtrail log on daily basis. | |
# You need to schedule it in AWS Lambda. | |
''' | |
------------------------------------------- | |
AWS Athena Create Partitions Automatically | |
------------------------------------------- | |
Version 1.0 | |
Author: SqlAdmin |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- Export the users | |
pg_dumpall -g > users.sql | |
-- Importing to another PostgresSQL | |
awk '/CREATE/' users.sql > migrate.sql | |
-- Import to RDS; It won't support super user and replication roles. But we can grant minimal superuser with rds_superuser | |
psql -h localserver -d postgres -t -c"select 'grant rds_superuser to '||rolname ||';' from pg_roles where rolsuper='t';" -P "footer=off" >> migrate.sql | |
sed -i -e's/NOSUPERUSER//g; s/SUPERUSER//g; s/NOREPLICATION//g; s//REPLICATION/g' migrate.sql | |
psql -h rds-endpoint -U adminuser -d postgres < migrate.sql |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Run any query on any RDS(MYSQL) while a cloudwatch alarm triggers. | |
# Version 1.0 | |
# Author: SqlAdmin | |
# Twitter: https://twitter.com/SqlAdmin | |
# Blog: www.sqlgossip.com | |
#---------------------------------------------------------------------------------------------------------------- | |
# Make a note: | |
# In this function we used to get the RDS instance Identifier from the SNS, It won't give the complete endpoint | |
# So in your account go and see the endpoint, it should like identifier+some-random-string+region+amazon.com | |
# The random string which is next to the Identifier is differ from other account. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# my code | |
from __future__ import print_function | |
import json | |
import sys | |
import os | |
print('Loading function') | |
NewerOlder