Skip to content

Instantly share code, notes, and snippets.

@jpbarto
jpbarto / qt5-al2-install.sh
Last active September 14, 2022 23:13
QT5 Install Script
#!/usr/bin/bash
# This script is built to compile Qt5 on an Amazon Linux2-based Amazon Workspace
# This script is known to fail on a Standard Workspace with 2 vCPU and 4 GB of RAM (Out of Memory error suspected)
# Script has been tested on a Power Workspace with 4 vCPU and 32 GB of RAM
# based upon documentation at
# https://doc.qt.io/qt-5/linux-requirements.html
set -e
@jpbarto
jpbarto / ML 101 README.md
Last active July 28, 2021 00:16
Lab instructions for an introduction to machine learning
@jpbarto
jpbarto / ec2_detach_volume.sh
Last active April 8, 2021 19:57
Chaos Shell Scripts
#!/usr/bin/env bash
###############################################################################################
#
# The following shell script uses the AWS CLI to forcefully detach a non-root volume from a
# running EC2 instance. In preliminary testing this produced a read-only filesystem on a
# running host with the volume mounted. This is intended to grossly simulate communication
# errors with the EBS volume from an EC2 instance.
#
# Note that this script will NOT detach the root volume from an EC2 instance.
@jpbarto
jpbarto / transfer_lambda_response_directory_details.json
Created January 4, 2021 16:47
Example response format from Lambda to Transfer for SFTP service
[
{
"Entry": "/public/research",
"Target": "/maxld-public-bucket"
},
{
"Entry": "/subscribed/2018/indices",
"Target": "/maxld-subscribe-bucket/historical/2018/indices"
},
{
@jpbarto
jpbarto / list-database-auto-backups.sh
Created October 30, 2020 10:12
List the automated RDS backups available for PITR
#!/bin/bash
set -e
AWS_REGION=eu-west-1
aws rds describe-db-instance-automated-backups --region $AWS_REGION --query 'DBInstanceAutomatedBackups[*].{DatabaseID:DBInstanceIdentifier,Earliest:RestoreWindow.EarliestTime,Latest:RestoreWindow.LatestTime}' --output table
@jpbarto
jpbarto / restore-database-with-pitr.sh
Created October 30, 2020 10:10
Bash Shell Script to restore and time the restoration of an RDS database using Point in Time Recovery (PITR)
#!/bin/bash
set -e
AWS_REGION=eu-west-2
SOURCE_DATABASE_NAME=database-1
RESTORE_TIME='2020-10-28T22:02:01+00:00'
NEW_DATABASE_NAME="rds-db-$RANDOM"
@jpbarto
jpbarto / stop_run_instance.json
Created August 20, 2020 09:40
Example of how to use IAM Policy Simulator to test IAM policy effects on permissions (also works for SCPs)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Deny",
"Action": "ec2:RunInstances",
"Resource": "*"
}
]
@jpbarto
jpbarto / query_cloudtrail.sh
Created August 20, 2020 04:04
Simple shell script to query AWS CloudTrail for particular actions in an AWS account
#!/bin/bash
# The following shell script uses Amazon Athena to query AWS CloudTrail logs for any occurrences of the
# action sagemaker:ListNotebookInstances. The query returns the user identity who invoked the API, along
# with the region where the API was called. A simple count of the number of invokes is outputted as a
# result.
SQL="SELECT useridentity.arn, eventname, eventsource, awsregion, sourceipaddress, errorcode, eventtime FROM cloudtrail_logs_account_logging WHERE eventsource = 'sagemaker.amazonaws.com' AND eventname in ('ListNotebookInstances') AND eventtime > '2020-04-15' LIMIT 3;"
EXEC_ID=$(aws athena start-query-execution --query-string "$SQL" --result-configuration OutputLocation=s3://my-s3-logging-bucket/queries --query 'QueryExecutionId' --output text)
@jpbarto
jpbarto / Deep Dive with Amazon SageMaker.md
Last active August 26, 2020 05:37
Overarching lab guide for an introduction to Amazon SageMaker

Lab 1: Getting started

Create a Jupyter notebook

  1. Visit https://github.com/awslabs/amazon-sagemaker-examples, at the bottom of the page you will find a link to create an Amazon SageMaker notebook
  2. Follow the instructions using an ml.m5.2xlarge instance type.
  3. When the notebook has been created click Open JupyterLab.

Clone the lab materials

Clone the Amazon SageMaker Examples to your notebook from GitHub.

@jpbarto
jpbarto / workshops-worth-shopping.md
Created May 29, 2020 21:52
Collection of Workshops worth working through