Skip to content

Instantly share code, notes, and snippets.

View aerostitch's full-sized avatar

aerostitch aerostitch

View GitHub Profile
@aerostitch
aerostitch / aws_etag_checksum.rb
Last active August 29, 2015 14:07
This little gist calculates the md5sum as the aws/s3 does for the etag property of AWS::S3::S3Object (either using multipart version or not).
#!/usr/bin/env ruby
require 'digest/md5'
##
# This little gist calculates the md5sum as the aws/s3 does.
# Either use a non-zero chunk size to specify the chunk size used while
# uploading the file, or just don't specify it if you didn't do
# a multipart upload.
#
# To have an example of how to get an aws etag, see:
# https://gist.github.com/aerostitch/22992e88315215f100b8
@aerostitch
aerostitch / aws_get_remote_etag.rb
Last active August 29, 2015 14:07
Get the AWS' etag/checksum from an s3:// uri with the required checks.
#!/usr/bin/env ruby
require 'aws/s3'
##
# This little gist returns the checksum of and aws file.
# This can be the checksum of the file or, if the file has been uploaded in a
# multipart format, it's the checksum of the binary concatenation of the
# checkusms of the chunks of the multiparts followed by the number of chunks of
# the multipart object.
#
# Tested with aws-sdk-v1 gem.
@aerostitch
aerostitch / aws_guess_chunk_size.rb
Last active August 29, 2015 14:07
How to guess the chunk size of an AWS file uploaded using multipart upload method.
#!/usr/bin/env ruby
##
# This gist tries to guess the chunk size in Megabytes of a multipart upload of
# an AWS file using its local copy and the etag of the remote file.
# Here we suppose the chunk size chosen for the upload is an integer in Megabytes.
#
# To run correctly, it requires the `get_etag` function from
# https://gist.github.com/aerostitch/538eda0b2d1d8dd914bf
#
# You can get the etag of the remote file by using the `get_AWS_file_checksum` function
@aerostitch
aerostitch / aws_verify_checksum_local_copy.rb
Last active August 29, 2015 14:07
This gist checks if an AWS file's etag matches any of the checksum of its local copy, this is done without knowing if the file has been uploaded using multipart method or not.
#!/usr/bin/env ruby
##
# This gist checks if an AWS file's etag matches any of the checksum of its local copy,
# this is done without knowing if the file has been uploaded using multipart method or not.
#
# Here we suppose the chunk size chosen for the upload is an integer in Megabytes
# when checking for checksum of a multipart upload.
#
# To run correctly, it requires the `get_etag` function from
# https://gist.github.com/aerostitch/538eda0b2d1d8dd914bf
@aerostitch
aerostitch / puppet_rakefile_test_erb_templates.rb
Last active August 29, 2015 14:07
This gist is a part of a Rakefile I use to test my Puppet's ERB (templates) files.
# This gist is a part of a Rakefile I use to test my Puppet's ERB (templates) files.
# Add this to the Rakefile at the root of your puppet module.
#
# == Examples:
# rake test:erb
# or
# rake test
#
# Author:: Joseph Herlant ([email protected])
# Copyright:: Copyright (c) 2014 Joseph Herlant
@aerostitch
aerostitch / curl_test.sh
Last active August 29, 2015 14:16
Script to test the response time of an url by doing a curl loop and retreiving several network metrics.
#!/bin/bash
# Author: Joseph Herlant
# Last updated: 2015-06-23
# Source: https://gist.github.com/aerostitch/5e5027ee1f6e2e828cd7
# Description
# This script's purpose is to test the response time on a url
# by doing a simple curl loop (no pause, so a bit hammering).
# It returns the several curl metrics in a csv file for further analysis.
#
TO="localhost"
@aerostitch
aerostitch / xfs_defrag_file_by_file.sh
Last active August 29, 2015 14:16
Defragment an xfs partition file by file if the given file has more than 1 fragment on the disk.
#!/bin/bash
# Author: Joseph Herlant
# Last updated: 2015-05-14
# Source: https://gist.github.com/aerostitch/0ddaea68d0034c26986c
# Description:
# This script is used to defragment an xfs partition file by file
# if the given file has more than 1 fragment.
# It writes to a log file that is overwritten every time.
#
ROOT_DIR="/mysql*"
@aerostitch
aerostitch / aws_get_user_from_access_key.go
Last active August 29, 2015 14:18
List the username corresponding to a given access key or just list all the tuples username/acess_key of an aws root account
package main
import (
"flag"
"fmt"
"github.com/awslabs/aws-sdk-go/aws"
"github.com/awslabs/aws-sdk-go/service/iam"
)
// Author: Joseph Herlant
@aerostitch
aerostitch / files_size_over_period.sh
Created June 25, 2015 19:18
bash sum files size foe a specific period
# Find all the mysql-bin* files >=2015-06-24 and < 2015-06-25 and sum their size
find /mysql/logs -type f -name 'mysql-bin*' -newermt 2015-06-24 ! -newermt 2015-06-25 -exec du -ch {} + | grep total$
@aerostitch
aerostitch / list_all_zknodes.sh
Last active April 12, 2016 20:49
I know there are a lot cleaner ways to do that, I just had to do that for a 1 time rush on a really bare server so bash recurse to list all the zk nodes there
#!/bin/bash
# I know there are a lot cleaner ways to do that, I just had to do that for a 1 time rush on a really bare server
# so bash recurse to list all the zk nodes there
list_children ()
{
if [ $# -le 0 ]
then
return