Skip to content

Instantly share code, notes, and snippets.

View masteinhauser's full-sized avatar

Myles Steinhauser masteinhauser

View GitHub Profile
@harperreed
harperreed / s3_mirror_site.sh
Created July 10, 2012 02:57
Create an static S3 bucket to mirror a site
#!/bin/bash
#this will mirror a website, upload to s3, and make it publically available.
#install s3cmd. install wget. (i use homebrew).
#configure s3cmd (s3cmd --configure)
# run this by doing: mirror_site.sh www.example.org
wget --mirror -p --html-extension --convert-links -e robots=off -P . $1
s3cmd mb s3://$1
@paulczar
paulczar / logstash-perf.md
Last active May 14, 2018 03:13
logstash performance deets

Logstash Performance Testing

Server Details

HP BL460

  • 48 Gb Memory
  • 2 x X5675 @ 3.07GHz
  • 2 x 10 gbps NIC
  • 2tb NetApp NFS volume for ES data
@bbeck
bbeck / jenkins-swarm-client
Created October 28, 2013 23:05
jenkins swarm client init.d script for CentOS
#!/bin/bash
#
# Jenkins Swarm Client
#
# chkconfig: 2345 89 9
# description: jenkins-swarm-client
source /etc/rc.d/init.d/functions
@hvrauhal
hvrauhal / README.md
Last active May 23, 2022 05:51
Aggressively caching squid for http://registry.npmjs.org/

Caching http://registry.npmjs.org/ aggressively with squid

We run a lot of project in our automated test environment and they spend a lot of time doing npm install. One way to speed up npm install and hammer the registry a little less is to use the http-version of the registry and to pass the requests through a proxy.

However, the npm requests include an authorization-header and the responses come with an ETag and Cache-Control: max-age=60, so with default squid settings there is a lot of roundtrips to registry.npmjs.org that result in 304's.

This is what the requests look like once npm has cached the first set of responses:

GET http://registry.npmjs.org/sntp HTTP/1.1

@tg
tg / add_healtcheck_ips_to_sg.sh
Last active August 10, 2018 16:29
Get AWS Route53 health check IPs
# This script adds aws route53 health check IPs to security group
# Requires jq to be installed (https://github.com/stedolan/jq)
# Pass security group ID as first argument and port (range) as a second
curl -s https://ip-ranges.amazonaws.com/ip-ranges.json |
jq '.prefixes[] | select(.service == "ROUTE53_HEALTHCHECKS") | .ip_prefix' |
xargs -t -n1 aws ec2 authorize-security-group-ingress --group-id "$1" --protocol tcp --port "$2" --cidr
@SuperPaintman
SuperPaintman / npm-f3-install.sh
Last active April 14, 2025 18:18
NPM install for low RAM machins. And "npm install ... killed" problem
#!/bin/bash
#
# Author: SuperPaintman <[email protected]>
#
###
# Constants
###
RETVAL=0
@kitschysynq
kitschysynq / gist:867caebec581cee4c44c764b4dd2bde7
Created April 10, 2017 13:45
List running qemu instances with nicely formatted output
ps -ef | awk -e '/qemu/ && !/awk/' | sed -e 's/[^/]*//' -e 's/ -/\n\t-/g'
@JoeyG1973
JoeyG1973 / aws_saml.py
Last active September 9, 2023 10:15
aws saml login with session that auto refreshes.
# Took this:
# https://s3.amazonaws.com/awsiammedia/public/sample/SAMLAPICLIADFS/samlapi_formauth_adfs3.py
# converted to boto3 and smooshed it together with this:
# https://gist.github.com/kapilt/ac8e222081f63ba64e93
# which gave birth too this:
import sys
import botocore
import boto3
import requests
@lesovsky
lesovsky / fn_upgrade_partitioning.sql
Created November 13, 2017 13:04
Upgrading old partitioning with ALTER TABLEs and PLPGSQL.
-- Upgrading old partitioning with ALTER TABLEs and PLPGSQL.
-- IN: _orig_table - master table which should be upgraded
-- IN: _partkey - column which used as partition key
-- IN: _seq_col - sequence column
CREATE OR REPLACE FUNCTION fn_upgrade_partitioning(_orig_table text, _partkey text, _seq_col text) RETURNS void AS
$function$
DECLARE
_new_table text = _orig_table ||'_new'; -- parent relation's name
_child_table text; -- child relation's name
_v_from timestamp without time zone;