arikfr

Setup

$ pip install click requests

Usage

$ python query_export.py --redash-url "https://app.redash.io/" --api-key ""

ktuite

import boto.mturk.connection

sandbox_host = 'mechanicalturk.sandbox.amazonaws.com'
real_host = 'mechanicalturk.amazonaws.com'

mturk = boto.mturk.connection.MTurkConnection(
    aws_access_key_id = 'XXX',
    aws_secret_access_key = 'XXX',
    host = sandbox_host,
    debug = 1 # debug = 2 prints out all requests. but we'll just keep it at 1

hummus

wget http://stedolan.github.io/jq/download/linux64/jq 
aws ec2 describe-instances --filters "Name=tag:Name,Values=$NAME" \
  "Name=instance-state-name,Values=running" \
  |  jq -r \
  ".Reservations[] | .Instances[] | .InstanceId" \

aws ec2 describe-volumes --filters \
  "Name=status,Values=available" \
  | jq -r ".Volumes[] | .VolumeId" \

KartikTalwar

The fastest remote directory rsync over ssh archival I can muster (40MB/s over 1gb NICs)

This creates an archive that does the following:

rsync (Everyone seems to like -z, but it is much slower for me)

• a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.
• H: preserves hard-links
• A: preserves ACLs