Peter Mescalchin magnetikonline

Extract unique HTTP request lines from access.log

$ cat access.log \
	| grep "] \"GET /wp-content/themes/" \
	| awk '{print $7}' \
	| sort -u

Ensure HTTP redirects are not cached

Chrome (at least) likes to cache HTTP redirects, an issue if you are running logging/reporting on those redirect actions server side. Combat this with some HTTP headers.

Location: http://domain.com/redirect
Cache-Control: must-revalidate,no-cache,no-store
Expires: Sat, 01 Jan 2000 00:00:00 GMT

Or in PHP land that could be...

SSH forward local port to remote AWS RDS instance

The scenario:

We have connectivity to remote server (EC2 instance) REMOTE_EC2_HOST over SSH.
Connectivity to AWS RDS MySQL instance RDS.ENDPOINT.rds.amazonaws.com from remote server only (due to security group/firewall settings).

# forward requests from 127.0.0.1:6400 -> RDS.ENDPOINT.rds.amazonaws.com:3306
# add [-f] switch to background ssh process
$ ssh -vvvN \
	-L 6400:RDS.ENDPOINT.rds.amazonaws.com:3306 \

AWS summary of EC2 storage types

For my own sanity, as I always seem to struggle with this.

Elastic block store (EBS)

Can be attached to any EC2 instance in the same availability zone.
Recommended for long term persistence.
Can attach multiple EBS volumes to a single EC2 instance.
Survives reboot, stop/start and terminate (but only if not the root EBS volume).

Instance storage (ephemeral)

Nginx 1.20.1 modules

Optional
Default
Connection processing methods
Building list data

Optional

AWS `s3curl.pl` example usage

Perl wrapper script for curl from Amazon to GET/PUT items to/from S3 buckets, adding the required headers to the curl command line.

Download: http://aws.amazon.com/code/128

Listing a bucket

$ chmod u+x s3curl.pl
$ ./s3curl.pl \
	--id [AWS_KEY] \

	#!/bin/bash

	git clone --depth 1 \
	-qb $BRANCH_NAME \
	$REPOSITORY_URL $TARGET_CLONE_DIR
	cd $TARGET_CLONE_DIR
	git submodule update -q --init --depth 1

	#!/bin/bash

	IAM_BASE_URL="http://169.254.169.254/latest/meta-data/iam/security-credentials"


	IAMRoleName=$(curl -s $IAM_BASE_URL/)
	IAMRoleData=$(curl -s $IAM_BASE_URL/$IAMRoleName/)
	IAMRoleAccessKeyID=$(echo -n "$IAMRoleData" \| sed -nr 's/.*?"AccessKeyId"[^"]+"([^"]+)",?/\1/p')
	IAMRoleAccessKeySecret=$(echo -n "$IAMRoleData" \| sed -nr 's/.*?"SecretAccessKey"[^"]+"([^"]+)",?/\1/p')
	IAMRoleToken=$(echo -n "$IAMRoleData" \| sed -nr 's/.*?"Token"[^"]+"([^"]+)",?/\1/p')

	#!/bin/bash

	rm turbo/*
	rm mozjpeg/*
	cd orig/

	echo libjpeg-turbo
	find . -name "*.jpg" -exec jpegtran -optimize -copy none -outfile "{}.tmp" "{}" \; -exec mv "{}.tmp" "../turbo/{}" \;
	echo Done

	#!/bin/bash

	find . -type f -exec touch -r "{}" -d '+1 day' "{}" \;