Skip to content

Instantly share code, notes, and snippets.

View lebedov's full-sized avatar

Lev E. Givon lebedov

View GitHub Profile
@lebedov
lebedov / Makefile
Last active October 3, 2016 21:03
How to expose the raw bytes in Python container classes to C functions using Cython.
funcs:
python setup.py build_ext --inplace
test: funcs
gcc -L. test.c -o test -lmyfuncs
clean:
rm -f test *.o *.so funcs.c
@lebedov
lebedov / msgpack_pandas_attrib.py
Created July 23, 2014 16:48
Serialize/unserialize a class with a pandas data structure attribute using msgpack.
#!/usr/bin/env python
"""
Serialize/unserialize a class with a pandas data structure attribute using msgpack.
"""
import msgpack
import numpy as np
import pandas as pd
@lebedov
lebedov / prof_dec.py
Created July 21, 2014 15:01
Decorator for profiling functions.
import cProfile
import functools
def do_cprofile(*dec_args):
"""
Decorator for profiling functions.
If a file name is passed to the decorator as an argument, profiling data
will be written to that file; otherwise, it will be displayed on the screen.
"""
@lebedov
lebedov / zmq_router_dealer_sync.py
Created July 15, 2014 17:35
How to synchronize ZeroMQ routers and dealers
#!/usr/bin/env python
"""
How to synchronize ZeroMQ routers and dealers.
"""
import multiprocessing as mp
import zmq
IPC_PATH = 'ipc://zmq_router_dealer_sync'
@lebedov
lebedov / linkedin_access_oauth1.py
Last active February 13, 2019 02:46
Get LinkedIn access tokens without having to open a web browser.
#!/usr/bin/env python
"""
Get LinkedIn OAuth1 access tokens without having to open a web browser.
Notes
-----
Based upon https://developer.linkedin.com/documents/getting-oauth-token-python
Assumes that the application API key, secret key, user name, and password are stored
@lebedov
lebedov / queue_pool_demo.py
Created June 11, 2014 15:08
How to use multiple queues for passing data to/from a multiprocessing pool.
#!/usr/bin/env python
"""
How to use multiple queues for passing data to/from a multiprocessing pool.
"""
from multiprocessing import Pool, Queue
import shortuuid
def f(q_in, q_out):
@lebedov
lebedov / qwrap.py
Created April 11, 2014 17:32
Automatically create and submit Torque job script with specified commands to qsub.
#!/usr/bin/env python
"""
Automatically create and submit Torque job script with specified commands to qsub.
"""
import argparse
import os
import pwd
import tempfile
@lebedov
lebedov / filter.py
Created March 23, 2014 19:39
torque submit filter that sets CUDA_VISIBLE_DEVICES based on specified number of GPUs.
#!/usr/bin/python2.7
"""
torque submit filter that automatically sets CUDA_VISIBLE_DEVICES
based upon the number of GPUs requested in a job.
Notes
-----
Assumes that /var/spool/torque/filters/trqgpu.py is available.
"""
@lebedov
lebedov / sshtunnel.sh
Created February 24, 2014 14:10
Functions for managing ssh tunnels in bash
#!/bin/bash
# Add these functions to your ~/.bashrc file.
function mktunnel {
if [[ $* == '' ]] || [[ $1 == '-h' ]]; then
echo 'Usage: mktunnel LOCALPORT REMOTEPORT REMOTEHOST'
else
ssh -fCNL $1:localhost:$2 $3;
fi
@lebedov
lebedov / ipynb_renumber.py
Last active March 15, 2020 21:10
Consecutively renumber prompts in an IPython notebook.
#!/usr/bin/env python
"""
Consecutively renumber prompts in an IPython notebook.
"""
# Copyright (c) 2015, Lev Givon
# All rights reserved.
# Distributed under the terms of the BSD license:
# http://www.opensource.org/licenses/bsd-license