Skip to content

Instantly share code, notes, and snippets.

View chicagobuss's full-sized avatar

Joshua Buss chicagobuss

  • TRM Labs
  • Chicago
View GitHub Profile
@chicagobuss
chicagobuss / mail.yml
Last active February 23, 2016 15:53
parallelized gce launch role
---
# file: roles/launch/tasks/main.yml
- name: Launch instances
local_action: gce instance_names={{ names }} machine_type={{ type }}
image={{ image }} network={{ network }} tags={{ tags }}
register: gce_instances
async: 7200
poll: 0
@chicagobuss
chicagobuss / conf_core-site.xml
Last active June 11, 2022 11:53
How to get spark 1.6.0 with hadoop 2.6 working with s3
<configuration>
<property>
<name>fs.s3a.access.key</name>
<description>AWS access key ID. Omit for Role-based authentication.</description>
<value>YOUR_ACCESS_KEY</value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<description>AWS secret key. Omit for Role-based authentication.</description>
@chicagobuss
chicagobuss / boto_download.py
Last active March 9, 2016 23:18
how to get boto working with s3proxy (with GCS on the backend)
#!/usr/bin/python
from boto import connect_s3
from boto.s3.connection import OrdinaryCallingFormat
from boto.s3.key import Key
import sys
s3 = connect_s3(aws_access_key_id = 'access',
aws_secret_access_key = 'secret',
@chicagobuss
chicagobuss / - output of ansible playbook run when it was broken -
Last active June 12, 2024 22:17
Trying to get vault and vars to work in templates with Ansible.... colons in file names represent directories
fatal: [ge-spark-master1] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'vaulted_access_key' is undefined", 'failed': True}
fatal: [ge-spark-master1] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'vaulted_access_key' is undefined", 'failed': True}
fatal: [ge-spark-master2] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'vaulted_access_key' is undefined", 'failed': True}
fatal: [ge-spark-master2] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'vaulted_access_key' is undefined", 'failed': True}
I 03-09 11:13:32.723 main org.eclipse.jetty.util.log:186 |::] Logging initialized @1095ms
I 03-09 11:13:32.755 main o.eclipse.jetty.server.Server:327 |::] jetty-9.2.z-SNAPSHOT
I 03-09 11:13:32.782 main o.e.j.server.ServerConnector:266 |::] Started ServerConnector@2c4d1ac{HTTP/1.1}{127.0.0.1:7000}
I 03-09 11:13:32.783 main o.eclipse.jetty.server.Server:379 |::] Started @1157ms
I 03-09 12:04:20.153 main org.eclipse.jetty.util.log:186 |::] Logging initialized @1085ms
I 03-09 12:04:20.182 main o.eclipse.jetty.server.Server:327 |::] jetty-9.2.z-SNAPSHOT
I 03-09 12:04:20.207 main o.e.j.server.ServerConnector:266 |::] Started ServerConnector@2c4d1ac{HTTP/1.1}{127.0.0.1:7000}
I 03-09 12:04:20.208 main o.eclipse.jetty.server.Server:379 |::] Started @1142ms
D 03-09 12:04:24.941 S3Proxy-13 o.gaul.s3proxy.S3ProxyHandler:292 |::] request: Request(HEAD /bucket/)@6bf36d0b
T 03-09 12:04:24.942 S3Proxy-13 o.gaul.s3proxy.S3ProxyHandler:314 |::] header: Authorization: AWS access:#############################
@chicagobuss
chicagobuss / DO_ALL_THREE.md
Last active March 14, 2016 18:28
make selinux modules for jupyterhub using sudospawner

Put this in sudo_exec_selinux.te:

module sudo_exec 1.1;

require {
        type unconfined_t;
        type sudo_exec_t;
        class file { read entrypoint };
}
@chicagobuss
chicagobuss / example_provision.yml
Created March 14, 2016 22:31
How do I provision several hosts at once?
- name: Create a zookeeper cluster
hosts: localhost
gather_facts: no
vars:
names: zoo1,zoo2,zoo3
type: g1-small
image: new-debian
network: new-network
tags: new
service_account_email: [email protected]
@chicagobuss
chicagobuss / libcloud_error.txt
Created March 16, 2016 16:40
libcloud error
File "/home/me/project/env/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 2368, in create_node
size = self.ex_get_size(size, location)
File "/home/me/project/env/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 4141, in ex_get_size
zone = self.ex_get_zone(zone)
File "/home/me/project/env/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 4282, in ex_get_zone
if name.startswith('https://'):
AttributeError: 'NoneType' object has no attribute 'startswith'
@chicagobuss
chicagobuss / kairos_delete.py
Last active March 18, 2016 18:35
Simple script for deleting metrics from kairos matching a particular name
#!/usr/bin/env python
"""
Deletes metrics from KairosDB that match a certain keyword in the name in a slice between two timestamps.
Note that this script SUBTRACTS 1 from the given end timestamp to make the end time EXCLUSIVE.
Install requests dependency:
> pip install requests
"""
import argparse
import requests
@chicagobuss
chicagobuss / BUILD-multiple-files
Last active April 9, 2018 11:31
Examples for using pants build to build a simple flask app
python_binary(
name='theapp',
dependencies=[
'3rdparty/python:Flask',
'3rdparty/python:gevent',
':theapp_lib'
],
resources=globs('static/*', 'templates/*'),
source='main.py'
)