Skip to content

Instantly share code, notes, and snippets.

View lextoumbourou's full-sized avatar

Lex Toumbourou lextoumbourou

View GitHub Profile
- name: compress capture file
command: gzip ${cap_file} chdir=/tmp
- name: copy logs to local boxes webroot
fetch: src=/tmp/${cap_file}.gz dest=/var/www/ flat=yes
- name: remove files from server
file: path=/tmp/${cap_file}.gz state=absent
(ENV)> ansible-playbook parallel-tcpdump.yml -i hosts
(ENV)> ls -1 /var/www/ | grep packet_capture
packet_capture_server1_1376450197.cap.gz
packet_capture_server2_1376450500.cap.gz
packet_capture_server3_1376451234.cap.gz
@lextoumbourou
lextoumbourou / auto_nested_dict.py
Created December 7, 2014 02:31
Super simple auto nested dict in Python
from collections import defaultdict
def auto_nested_dict():
return defaultdict(auto_nested_dict)
if __name __ == '__main__':
d = auto_nested_dict()
d['wats']['up']['world'] = 'Here I am!'
@lextoumbourou
lextoumbourou / nginx.conf
Created January 14, 2015 10:08
Conditional Nginx error pages based on Accept (or other) headers
http {
map $http_accept $extension {
default html;
application/json json;
}
server {
listen 80;
error_page 500 /500;
@lextoumbourou
lextoumbourou / gist:ac085cd8002141d036c9
Created January 16, 2015 03:10
Install a Pip package via Git (or any VCS) with setuptools extras
> pip install git+ssh://[email protected]/lextoumbourou/MyProject.git#egg=MyModule[extra]
@lextoumbourou
lextoumbourou / hbase_heap.py
Last active May 10, 2019 10:20
Hbase Heap Size Calculator
def get_regionserver_heap_size(
storage_capacity_in_gb,
region_max_filesize=10737418240,
memstore_flush_size=134217728,
replication_factor=3,
memstore_heap_fraction=0.4
):
"""
Calculates heap size required based on storage requirements.
@lextoumbourou
lextoumbourou / compile.sh
Created December 6, 2017 05:00
Compile Tensorflow for Lambda
BASE_PATH=./
yum install -y wget
wget https://copr.fedorainfracloud.org/coprs/vbatts/bazel/repo/epel-7/vbatts-bazel-epel-7.repo -O /etc/yum.repos.d/vbatts-bazel-epel-7.repo
# Appear to have to run this twice to get it to run reliably.
yum install -y bazel; yum install -y bazel
if [ ! -d "$BASE_PATH/env" ]; then
@lextoumbourou
lextoumbourou / dataframe_dataset.py
Created August 20, 2018 09:18
Torchtext dataset from DataFrame
from torchtext import data
class DataFrameDataset(data.Dataset):
def __init__(self, df, text_field, label_field, is_test=False, **kwargs):
fields = [('text', text_field), ('label', label_field)]
examples = []
for i, row in df.iterrows():
label = row.sentiment if not is_test else None
text = row.text