Skip to content

Instantly share code, notes, and snippets.

@nicor88
nicor88 / bootstrap_jupyter.sh
Created April 20, 2017 10:23
Bootstrap action to install Conda and Jupyter on EMR
#!/usr/bin/env bash
set -x -e
JUPYTER_PASSWORD=${1:-"myJupyterPassword"}
NOTEBOOK_DIR=${2:-"s3://myS3Bucket/notebooks/"}
# home backup
if [ ! -d /mnt/home_backup ]; then
sudo mkdir /mnt/home_backup
sudo cp -a /home/* /mnt/home_backup
@simonw
simonw / recover_source_code.md
Last active September 28, 2024 08:10
How to recover lost Python source code if it's still resident in-memory

How to recover lost Python source code if it's still resident in-memory

I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6

Attach a shell to the docker container

Install GDB (needed by pyrasite)

apt-get update && apt-get install gdb
@ueokande
ueokande / benchmark-commands.md
Last active February 7, 2025 07:46 — forked from jkreps/benchmark-commands.txt
Kafka Benchmark Commands

Benchmark commands

Producer

Setup

bin/kafka-topics.sh \
  --zookeeper zookeeper.example.com:2181 \
  --create \

Kafka installation with systemd

0. Create kafka user

sudo adduser kafka
sudo adduser kafka sudo
su -l kafka

1. Download and Install kafka archive

@marcelog
marcelog / aws-sqs.policy
Last active December 4, 2023 07:07
SQS Policy to allow an S3 bucket to publish messages
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:YOUR-AWS-REGION:YOUR-AWS-ACCOUNT-ID:YOUR-QUEUE-NAME/SQSDefaultPolicy",
"Statement": [
{
"Sid": "example-statement-ID",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
@guillaumevincent
guillaumevincent / README.md
Last active December 9, 2024 14:37
Windows Service with Python 3.5 and pyinstaller
@ryanmaclean
ryanmaclean / promiscous_enable.sh
Last active July 18, 2024 14:58
Allow VMWare Fusion Promiscuous Mode on macOS Sierra
#!/bin/bash
sudo touch "/Library/Preferences/VMware Fusion/promiscAuthorized"
@ducas
ducas / increase-replication-factor.js
Last active November 8, 2021 14:42
Increase Replication Factor for Kafka Topics
/*
* This script reads the existing partition assignments in JSON (exported using kafka-reassign-partitions.sh) and generates
* the appropriate reassignment JSON file for input to kafka-reassign-partitions.sh.
*
*/
var usage = `
Usage: node increase-replication-factory.js (brokers) (replication factor) (current assignment file) (output file) [-f]
* brokers: comma separated list of brokers
* replication factor: the new replication factor to use
# This is an example of a bring your own (byo) host inventory
# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
#etcd
#lb
# Set variables common for all OSEv3 hosts
@diegopacheco
diegopacheco / ansible-aws.md
Last active May 15, 2020 13:44
How to Install Ansible on Amazon Linux / CentOS?
sudo su
yum update
yum install -y git
cd /usr/local/src
yum -y install git python-jinja2 python-paramiko PyYAML make MySQL-python
git clone git://github.com/ansible/ansible.git
cd ansible
git submodule update --init --recursive
make install