Skip to content

Instantly share code, notes, and snippets.

View MikeDacre's full-sized avatar

Mike Dacre MikeDacre

View GitHub Profile
@MikeDacre
MikeDacre / .python-cluster
Created September 7, 2016 22:06
Cluster profile file for fraser cluster
[opts]
[queue]
max_jobs = 1000
sleep_len = 2
queue_update = 5
[prof_default]
cores = 1
mem = 4000
@MikeDacre
MikeDacre / run_rename.py
Created September 13, 2016 23:25
Run simple script with python-cluster
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Submit a bunch of jobs."""
import sys
import argparse
from time import sleep
import cluster
CMD = '/home/dacre/mike_tools/bin/rename_mesh_snps.py {bed} -i {infile} -o {outfile}'
@MikeDacre
MikeDacre / .cvimrc
Last active September 22, 2016 20:51
My cVimrc
" Settings
" set nohud
" set nosmoothscroll
" set noautofocus " The opposite of autofocus; this setting stops
" sites from focusing on an input box when they load
set typelinkhints
let searchlimit = 30
let scrollstep = 70
let barposition = "bottom"
+ [pyenv:24] enable -f /home/dacre/.pyenv/libexec/../libexec/pyenv-realpath.dylib realpath
+ [pyenv:30] '[' -z '' ']'
++ [pyenv:32] type -p greadlink readlink
++ [pyenv:32] head -1
+ [pyenv:32] READLINK=/usr/bin/readlink
+ [pyenv:33] '[' -n /usr/bin/readlink ']'
+ [pyenv:54] '[' -z /home/dacre/.pyenv ']'
+ [pyenv:57] PYENV_ROOT=/home/dacre/.pyenv
+ [pyenv:59] export PYENV_ROOT
+ [pyenv:62] '[' -z '' ']'
@MikeDacre
MikeDacre / troubleshoot.txt
Created November 18, 2016 02:50
jupyter troubleshoot outpyt
$PATH:
/usr/local/anaconda3/bin
/usr/local/anaconda3/bin
/home/dacre/bin
/usr/local/sbin
/usr/local/bin
/usr/bin
/home/dacre/usr/bin
/usr/local/MATLAB/R2013a/bin
/usr/lib/jvm/default/bin
@MikeDacre
MikeDacre / Use SQLAlchemy with the UCSC Database.ipynb
Created February 9, 2017 19:17
Use SQLAlchemy with the UCSC Genome Database
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@MikeDacre
MikeDacre / filter_dataframe.py
Last active May 25, 2017 16:35
A python 3 function to make a pandas DataFrame unique on multiple columns while preserving other columns and filtering by multiple columns
def run_filter(df, group_cols: list, merge_col: str, sort_col: str, cutoff: float = 0.6):
"""Make a DataFrame unique on group_cols based on boolean merge_col and float sort_col.
Picks most frequent result of boolean merge_col if the result represents a portion of the
total greater than cutoff. Then sorts the results ascending by sort_col and returns the
first row only.
If filter fails (i.e. top hit is less than cutoff of total) then the entire group is dropped.
Parameters
@MikeDacre
MikeDacre / Filter by multiple columns.ipynb
Last active May 25, 2017 16:34
Make a pandas DataFrame unique on multiple columns while preserving other columns and filtering
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@MikeDacre
MikeDacre / pascal.py
Created June 9, 2017 02:19
Run PASCAL: Allows easy running of https://www2.unil.ch/cbg/index.php?title=Pascal from any path
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Rewrite of the shell script for easy running from anywhere with cluster support.
Place this in the root of pascal and link to it from your path
"""
from __future__ import print_function
import os as _os
import sys as _sys
@MikeDacre
MikeDacre / cloudflare_refresh_nginx.sh
Created September 12, 2017 19:52
Sync cloudflare IPs with NGINX
# Based on https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-
# Save this file to /usr/bin/clodflare_refresh_nginx.sh
# Add the line "include /etc/nginx/global/cloudflare-ip.conf" to the server
# block of every nginx site you wish to expose the real IP for
# Add crontab (run a 03:05 every day): 5 3 * * * /usr/bin/clodflare_refresh_nginx.sh >/dev/null 2>&1
NGINX_CONF=/etc/nginx/global/cloudflare-ip.conf
curl https://www.cloudflare.com/ips-v4 | perl -p -e 's/^(.*)$/set_real_ip_from \1;/g' > $NGINX_CONF
curl https://www.cloudflare.com/ips-v6 | perl -p -e 's/^(.*)$/set_real_ip_from \1;/g' >> $NGINX_CONF
# use any of the following two
echo "real_ip_header CF-Connecting-IP;" >> $NGINX_CONF