Skip to content

Instantly share code, notes, and snippets.

View hartfordfive's full-sized avatar

Alain L hartfordfive

View GitHub Profile
@hartfordfive
hartfordfive / .git-verification-patterns
Last active November 13, 2024 15:08
Git pre-commit hook to search for potentially sensitive data
# List of the patterns to search for
declare -a git_verification_patterns # Create an associative array
git_verification_patterns[0]="(\"|')?(AWS|aws|Aws)?_?(SECRET|secret|Secret)?_?(ACCESS|access|Access)?_?(KEY|key|Key)(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9/\+=]{40}(\"|'|\s)?"
git_verification_patterns[1]="(\"|')?(AWS|aws|Aws)?_?(SECRET|secret|Secret)?_?(ACCESS|access|Access)?_?(KEY|key|Key)?_?(ID|id|Id)?(\"|')?\s*(:|=>|=)\s*(\"|')?[A-Za-z0-9/\+=]{20}(\"|'|\s)?"
git_verification_patterns[2]="(\s+|=|:)[a-z0-9]{64}(\s+|\|)" # DigitalOcean Personal Access Token
git_verification_patterns[3]='Authorization\s+"?Basic\s+[a-zA-Z0-9+/]+={0,2}'
git_verification_patterns[4]="BEGIN RSA PRIVATE KEY"
git_verification_patterns[5]="END RSA PRIVATE KEY"
# List of the pattern descriptions (in same order)
@hartfordfive
hartfordfive / packages.rb
Created July 8, 2016 16:23
Easily specifying different packages based on OS in Chef
packages = value_for_platform_family(
%w(rhel fedora suse) => %w(pcre-devel),
%w(gentoo) => [],
%w(default) => %w(libpcre3 libpcre3-dev)
)
packages.each do |p|
package p do
action :install
end
@hartfordfive
hartfordfive / gist:1a9647a02273db1d38e1b32a001c7b56
Created July 8, 2016 20:39
One-line bash command to delete multiple Elasticsearch indices, based on a pattern
# Useful when the "action.disable_delete_all_indices" setting is set to "false"
for I in $(curl --silent 'http://[ES_HOST]:9200/_cat/indices/[INDEX_PATTERN]*?v&h=i'); do curl -XDELETE http://[ES_HOST]:9200/$I; done
@hartfordfive
hartfordfive / asg-ssh
Last active September 12, 2016 14:15
SSH into all servers from an auto-scaling group via iTerm on Mac
#!/usr/bin/php
<?php
$sshKey = "~/.ssh/id_rsa";
$sshUser = getenv('USER'); // Get the user from your environment variables
$autoScalingGroups = $argv[1]; // A blob pattern of the auto scalling group name to match
$userPrivateIp = true; // Use the VPC private IP to connect?
$profile = getenv('WORKENV');
@hartfordfive
hartfordfive / verify_s3_file.rb
Created November 1, 2016 17:47
Check if file exists in S3 bucket with Ruby aws-sdk gem
require 'aws-sdk'
s3 = Aws::S3::Resource.new(
region: 'us-east-1',
credentials: Aws::InstanceProfileCredentials.new()
)
bucket = s3.bucket('my-daily-backups')
file = (DateTime.now).strftime("%Y.%m.%d-backup")
if bucket.object(file).exists?
@hartfordfive
hartfordfive / close_channel_on_complete.go
Created December 5, 2016 16:18
Closing Go channel once all values have been acknowledged/received.
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
@hartfordfive
hartfordfive / gist:4fe09f1e0d3f09dad489ab307e99bf67
Created December 8, 2016 10:22
Exclude Elasticsearch Node from Cluster and Relocate Data
# Remove the node from the cluster
curl -XPUT "http://[ES_NODE_IP]:9200/_cluster/settings" -d '{
"transient" :{
"cluster.routing.allocation.exclude._ip" : "[ES_NODE_IP_TO_REMOVE]"
}
}'
# Check the pending cluster task status
curl "http://[ES_NODE_IP]:9200 /_cluster/pending_tasks
@hartfordfive
hartfordfive / sample-git-commands.txt
Created December 13, 2016 15:53
Contributing to other Go projects
# Easy way to work in go projects to submit PRs to other projects from your fork
# Switch to the project directory in the Go path
$GOPATH/src/github.com/elastic/beats/metricbeat`
# If there isn't already an origin for your fork, then add it
git remote add hartfortdrive [email protected]:hartfordfive/beats.git
# Once changes are made and you're ready to push to the branch on your origin
@hartfordfive
hartfordfive / instance_termination.py
Created January 19, 2017 14:40 — forked from gswallow/instance_termination.py
Crude AWS Lambda to deregister nodes from Chef upon EC2 instance termination
import json
import chef
# Copy your .chef directory into the root folder of the deployment package:
# http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#deployment-pkg-for-virtualenv
# See also https://github.com/coderanger/pychef/issues/41
print('Loading function')
def lambda_handler(event, context):
@hartfordfive
hartfordfive / create-keystore.sh
Last active February 2, 2017 14:25
Setting up keystone/trustore to use in StreamSets for Kafka hosted on CloudKarafka
#!/bin/bash
# Notes:
# - You must seperate the certificates CloudKarafka provides in the single file into three different files: ca.pem (CA certificate), cert.pem (certificate), key.pem (private key)
# - Once completed, you must copy over the keystore.jks and truststore.jks files over to SDC server in the directory where your config parameters will point to.
# - You SHOULD pick something other than "test1234" for your password
echo -n "Enter a password to use for the keystore: "
read -s PW
echo ""