Skip to content

Instantly share code, notes, and snippets.

### Keybase proof
I hereby claim:
* I am perryism on github.
* I am perrylee (https://keybase.io/perrylee) on keybase.
* I have a public key ASB8r9SSmhz7ykeAh3aaFwfAgfpm90Fs8XpStxg0BVoNLwo
To claim this, I am signing this object:
@perryism
perryism / gist:87e382cbc9d6ef0e73ba247bd99a1f0c
Last active September 26, 2017 23:33
Use TensorFlow with GPU support
# https://www.tensorflow.org/install/install_linux
# https://developer.nvidia.com/cuda-downloads
wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb -qO cuda.deb
sudo dpkg -i cuda.deb
sudo apt-get update
sudo apt-get install -y python-pip python-dev build-essential libcupti-dev cuda
sudo pip install -U pip
@perryism
perryism / make_week_dir.py
Last active September 19, 2017 23:35
Create a folder for the week
from datetime import datetime, timedelta
import os, logging
def week_dir():
now = datetime.now()
now = now - timedelta(days=now.weekday())
return now.strftime("%Y-%m-%d")
def make_week_dir():
try:
@perryism
perryism / backup.py
Created September 19, 2017 23:39
Make a copy of the original file
"""
Make a copy of a file in case the block is modifying it
Example:
with Backup('filepath') as f:
pickle(f, xx)
"""
from datetime import datetime
@perryism
perryism / Dockerfile.gpu
Created December 17, 2017 07:06
nvidia docker
FROM nvidia/cuda
MAINTAINER perry
COPY ./keyboard /etc/default/keyboard
RUN apt-get update && \
apt-get install -y python-pip python-dev build-essential libcupti-dev cuda wget git && \
wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb -qO cuda.deb && \
dpkg -i cuda.deb && \
@perryism
perryism / gist:75cf03242e79559f755297d9ae28977b
Created March 29, 2018 16:34
Hiding user input on terminal when typing password to unlock keychain
read -s PASSWORD && security -v unlock-keychain -p $PASSWORD "/Users/plee/Library/Keychains/login.keychain-db" 2>&1 | grep -v 'unlock-keychain'
@perryism
perryism / firehose_record_decorator
Last active August 21, 2019 15:58
firehose json record decorator
class Record:
def __init__(self, record):
self.record = record
def decoded_data(self):
return base64.b64decode(self.record['data'])
def merge(self, new_data):
return {
'recordId': self.record['recordId'],
@perryism
perryism / aws-elastisearch-failed.py
Created September 9, 2019 23:43
get errors from elastic search failed logs
import json
import base64
def each_data(stream):
for line in stream.readlines():
data = json.loads(line)
data['errorMessage'] = json.loads(data['errorMessage'])
yield data, json.loads(base64.b64decode(data['rawData']).decode())
import boto3
@perryism
perryism / aws_elastic_search_client.py
Created September 13, 2019 21:17
elastic search client using aws credentials
import boto3
from requests_aws4auth import AWS4Auth
from elasticsearch import Elasticsearch, RequestsHttpConnection
class AwsElasticSearchClient(Elasticsearch):
def __init__(self, **args):
session = boto3.Session()
creds = session.get_credentials().get_frozen_credentials()
awsauth = AWS4Auth(
creds.access_key, creds.secret_key, args["region"], "es", session_token=creds.token
@perryism
perryism / http_dump.py
Created December 3, 2019 19:12
the script dumps the headers and the body of a request
from flask import Flask, request
app = Flask(__name__)
import logging, sys
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logger = logging.getLogger(__name__)
@app.route("/", methods=["POST", "GET"])
def call():