Skip to content

Instantly share code, notes, and snippets.

View paulmwatson's full-sized avatar

Paul Watson paulmwatson

View GitHub Profile
# Python to return the IAM details of an access key
import boto3
client = boto3.client('sts', aws_access_key_id="ACCESS_KEY_ID", aws_secret_access_key="SECRET")
client.get_caller_identity()
=> {'UserId': '...', 'Account': '...', 'Arn': 'arn:aws:iam::...:user/...', 'ResponseMetadata': {'RequestId': '...', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '...', 'content-type': 'text/xml', 'content-length': '409', 'date': '...'}, 'RetryAttempts': 0}}
docker run -it --rm -v /dir/to/code:/app python:3.9 bash
@paulmwatson
paulmwatson / explode-geojson.rb
Created May 23, 2022 07:39
Convert multiple feature/layers in a geoJSON file to individual files
# Creates multiple files from one geojson file
require 'json'
file = File.read('data.geojson')
data_hash = JSON.parse(file)
data_hash['features'].each do |feature|
file_name = feature['properties']['OBJECTID']
File.open("#{file_name}.geojson", 'w') do |f|
f.write(feature.to_json)
end
import tweepy
import os 
API_KEY = os.getenv('API_KEY')
API_SECRET = os.getenv('API_SECRET') 
auth = tweepy.AppAuthHandler(API_KEY, API_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True) 
items = tweepy.Cursor(api.user_timeline, screen_name='NPATweets', count=200).items(3200)
@paulmwatson
paulmwatson / docker-compose.yml
Created January 3, 2022 13:50
Using PgBouncer with Django and Docker Compose
version: '3'
services:
app:
build: .
image: app
depends_on:
- db
environment:
- DATABASE_URL=postgresql://db:db@pgbouncer/db
@paulmwatson
paulmwatson / 302-redirect.js
Created December 23, 2021 14:31
302 redirect
'use strict';
exports.handler = (event, context, callback) => {
var response = {
"statusCode": 302,
"headers": {
"location": "https://new-domain.tld"
},
"body": "{}",
"isBase64Encoded": false
@paulmwatson
paulmwatson / README.md
Last active July 8, 2021 10:28
Accessing MySQL 5.1 data without the root password

Alter existing mysql database without root password

Unzip oldwebsitedata.tar.gz into this directory so that you end with ./mnt/mysql-snapshot-2015-03-31/data

Then run:

docker-compose run mysql bash

Once inside the container run:

@paulmwatson
paulmwatson / new_url_manipulation.js
Created June 7, 2021 12:42
Manipulating the query string with URLSearchParams
const url = 'https://host.test/path/here.filetype?query=string&more=params#anchor';
let newUrl = new URL(url);
let params = new URLSearchParams(newUrl.search);
params.set('query', `${params.get('query')}-changed`);
params.delete('more');
params.set('new', 'param');
newUrl.search = params;
@paulmwatson
paulmwatson / string_url.js
Last active June 7, 2021 12:37
Manipulating the URL via strings
let url = 'https://host.test/path/here.filetype?query=string&more=params#anchor';
const querystringStart = url.indexOf('?');
const anchorStart = url.indexOf('#');
const querystring = url.slice(querystringStart + 1, anchorStart);
let querystringParams = querystring.split('&');
let newQuerystringParams = [];
querystringParams.forEach((queryparam, i) => {
let param = queryparam.split('=');
if (param[0] !== 'more') {
newQuerystringParams.push(`${param[0]}=${param[1]}-changed`)
@paulmwatson
paulmwatson / cdxt_commoncrawl_example_domain.sh
Created May 25, 2021 10:46
Use CDXT Toolkit to search CommonCrawl for certain domains/URLs
cdxt --cc --limit 100 --filter '=status:200' iter 'github.com/*'