Peter Mescalchin magnetikonline

Transfer MySQL databases between AWS RDS instances

The situation

Two AWS RDS MySQL databases - moving databases from source to destination.
Both databases are not publicly accessible, only via an EC2 instance(s) - e.g. you have setup your security groups.
Can SSH to a target EC2 instance (but of course).

What it does

Sets up two SSH port forwards on local machine - one to source database, another to target.
Via calls to transferDatabase():

Using BFG to clean out old binary assets from Git repository

Reference: http://rtyley.github.io/bfg-repo-cleaner/

Example

$ wget http://repo1.maven.org/maven2/com/madgag/bfg/1.11.8/bfg-1.11.8.jar
$ git clone [email protected]:org-name/repo-name.git repo-name.cleanup
$ java -jar bfg-1.11.8.jar --delete-files '*.{gif,jpg,png}' repo-name.cleanup
$ cd repo-name.cleanup
$ git gc --prune=now --aggressive

AWS CLI S3 usage examples

Local dir to remote bucket sync
Local dir to remote bucket copy with gzip content encoding header
Remote to remote bucket copy

Local dir to remote bucket sync

$ aws s3 sync \

Extract unique HTTP request lines from access.log

$ cat access.log \
	| grep "] \"GET /wp-content/themes/" \
	| awk '{print $7}' \
	| sort -u

Ensure HTTP redirects are not cached

Chrome (at least) likes to cache HTTP redirects, an issue if you are running logging/reporting on those redirect actions server side. Combat this with some HTTP headers.

Location: http://domain.com/redirect
Cache-Control: must-revalidate,no-cache,no-store
Expires: Sat, 01 Jan 2000 00:00:00 GMT

Or in PHP land that could be...

SSH forward local port to remote AWS RDS instance

The scenario:

We have connectivity to remote server (EC2 instance) REMOTE_EC2_HOST over SSH.
Connectivity to AWS RDS MySQL instance RDS.ENDPOINT.rds.amazonaws.com from remote server only (due to security group/firewall settings).

# forward requests from 127.0.0.1:6400 -> RDS.ENDPOINT.rds.amazonaws.com:3306
# add [-f] switch to background ssh process
$ ssh -vvvN \
	-L 6400:RDS.ENDPOINT.rds.amazonaws.com:3306 \

AWS summary of EC2 storage types

For my own sanity, as I always seem to struggle with this.

Elastic block store (EBS)

Can be attached to any EC2 instance in the same availability zone.
Recommended for long term persistence.
Can attach multiple EBS volumes to a single EC2 instance.
Survives reboot, stop/start and terminate (but only if not the root EBS volume).

Instance storage (ephemeral)

Nginx 1.20.1 modules

Optional
Default
Connection processing methods
Building list data

	#!/bin/bash

	git clone --depth 1 \
	-qb $BRANCH_NAME \
	$REPOSITORY_URL $TARGET_CLONE_DIR
	cd $TARGET_CLONE_DIR
	git submodule update -q --init --depth 1

	#!/bin/bash

	find . -type f -exec touch -r "{}" -d '+1 day' "{}" \;