Skip to content

Instantly share code, notes, and snippets.

Crisp-DM on AWS

Six phase of DM

Phase 1: Business Understanding

@gene1wood
gene1wood / boto3_paginator_pattern.py
Last active May 26, 2020 05:45
An example of a pattern for collecting all of the results using pagination from a paginated boto3 paginator
import boto3
# This example uses the CloudFormation product
client = boto3.client('cloudformation')
# and is listing all of the deployed CloudFormation stacks
paginator = client.get_paginator('list_stacks')
collector = []
# "StackSummaries" is the name of the key AWS returns for the "list_stacks" call.
# This will differ for every paginated AWS call
result = list(map(collector.extend, [x['StackSummaries'] for x in paginator.paginate()]))
@geerlingguy
geerlingguy / aws-sts-token
Created September 19, 2018 16:05
AWS STS Token update playbook for MFA
#!/usr/bin/env ansible-playbook -c local
#
# AWS STS token update playbook.
#
# Updating AWS session tokens with STS can be a pain. But MFA is good. So let's
# automate the management of the .aws/credentials file to make it not painful!
#
# Usage:
#
# 1. Save this to a file like /usr/local/bin/aws-sts-token
@jgamblin
jgamblin / instantkali.sh
Created March 5, 2018 00:39
Start an EC2 Kali Instance In Under 60 Seconds.
#!/bin/bash
set -e
set -u
clear
ami="ami-10e00b6d"
size="t2.medium"
today=$(date +"%m-%d-%y-%H%M")
localip=$(curl -s https://ipinfo.io/ip)
@apolloclark
apolloclark / lock_down_public_s3_buckets.md
Last active January 27, 2023 08:10
Bash one-liner to find public facing AWS S3 buckets, and make them private

Command

aws s3api list-buckets --query 'Buckets[*].[Name]' --output text | xargs -I {} bash -c 'if [[ $(aws s3api get-bucket-acl --bucket {} --query '"'"'Grants[?Grantee.URI==`http://acs.amazonaws.com/groups/global/AllUsers` && Permission==`READ`]'"'"' --output text) ]]; then aws s3api put-bucket-acl --acl "private" --bucket {} ; fi'



1. List all of the user's buckets, and output the name, as text.

@sherzodinaev
sherzodinaev / generate-iam-assets.sh
Last active December 31, 2019 17:09
IAM asset generator
#!/bin/bash
export AWS_PROFILE='default'
mkdir -p iam/policies
POLICIES=$(aws iam list-policies --scope Local --query "Policies[*].{name: PolicyName, version: DefaultVersionId, arn: Arn}")
echo $POLICIES | jq -c '.[]' | \
while read policy; do
@shortjared
shortjared / list.txt
Last active April 11, 2025 14:12
List of AWS Service Principals
a4b.amazonaws.com
access-analyzer.amazonaws.com
account.amazonaws.com
acm-pca.amazonaws.com
acm.amazonaws.com
airflow-env.amazonaws.com
airflow.amazonaws.com
alexa-appkit.amazon.com
alexa-connectedhome.amazon.com
amazonmq.amazonaws.com
@ibraheem4
ibraheem4 / postgres-brew.md
Last active April 19, 2025 14:27 — forked from sgnl/postgres-brew.md
Installing Postgres via Brew (OSX)

Installing Postgres via Brew

Pre-Reqs

Brew Package Manager

In your command-line run the following commands:

  1. brew doctor
  2. brew update
{
"AttributeDefinitions": [
{
"AttributeName": "Deleted",
"AttributeType": "S"
},
{
"AttributeName": "Id",
"AttributeType": "S"
}
@joshbuchea
joshbuchea / semantic-commit-messages.md
Last active April 25, 2025 01:26
Semantic Commit Messages

Semantic Commit Messages

See how a minor change to your commit message style can make you a better programmer.

Format: <type>(<scope>): <subject>

<scope> is optional

Example