Skip to content

Instantly share code, notes, and snippets.

View donkirkby's full-sized avatar
🐍

Don Kirkby donkirkby

🐍
65 followers · 9 following
View GitHub Profile
@donkirkby
donkirkby / gist:5312490
Last active December 15, 2015 19:38 — forked from agiliq/gist:131679
Markov text generator, with bug fix to wrap around from end of source text to the start. Also add a method to generate complete sentences based on the location of periods in the source.
import random
class Markov(object):
def __init__(self, open_file):
self.cache = {}
self.open_file = open_file
self.words = self.file_to_words()
self.word_size = len(self.words)
self.database()
@donkirkby
donkirkby / drop_everything.sql
Last active August 29, 2015 14:01
Drop all tables and sequences in an Oracle database
set heading off;
set echo off;
set pages 999;
spool dropall.sql
select 'drop table ' || table_name || ';' from user_tables
union all
select 'drop sequence ' || sequence_name || ';' from user_sequences order by 1;
spool off
-- Remove header and footer from dropall.sql, then...
@donkirkby
donkirkby / mnem_plot.py
Last active August 29, 2015 14:06
Mnemosyne history graph
from bz2 import BZ2File # @UnresolvedImport
from datetime import datetime
from glob import glob
import itertools
import matplotlib
import matplotlib.pyplot as plt
MNEMOSYNE_ROOTS = ["/home/don/Dropbox/Mnemosyne_old/.mnemosyne/",
"/home/don/.local/share/mnemosyne/"]
@donkirkby
donkirkby / multitasking.py
Last active February 18, 2023 15:20
Worker processes with mpi4py
#! /usr/bin/env python
import argparse
import csv
from mpi4py import MPI
import logging
import time
def parseOptions(comm_world):
parser = argparse.ArgumentParser(
@donkirkby
donkirkby / testmanager.py
Last active August 29, 2015 14:11
MPI4py spawn
from mpi4py import MPI
import sys
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('worker_count', type=int)
args = parser.parse_args()
mpi_info = MPI.Info.Create()
@donkirkby
donkirkby / aln2fasta.py
Created January 23, 2015 19:49
Convert aligned reads CSV to FASTA format
import argparse
import csv
import itertools
from operator import itemgetter
parser = argparse.ArgumentParser()
parser.add_argument('aligned',
type=argparse.FileType('rU'),
help='aligned reads CSV file')
parser.add_argument('--out',
@donkirkby
donkirkby / polling_mpi.py
Last active August 29, 2015 14:14
Example that shows how to wait for an MPI message without pegging the CPU to 100%.
#!/usr/bin/env python
from mpi4py import MPI
import logging
import time
import sys
def polling_receive(comm, source):
# Set this to 0 for maximum responsiveness, but that will peg CPU to 100%
@donkirkby
donkirkby / beowulf_top.sh
Last active January 7, 2016 00:09
Report processes with top CPU across a Beowulf cluster.
{ bpsh -pa ps -eo pcpu,pid,user,args ; bpsh -p -1 ps -eo pcpu,pid,user,args | bpstat -P -1 | sed -e 's/^ */ /' ; } | sort -g -k2 -r | head -30
@donkirkby
donkirkby / udjango.py
Last active July 5, 2016 17:16
𝜇Django - a minimal, but runnable, Django snippet
# Tested with Django 1.9.2
import sys
import django
from django.apps import apps
from django.apps.config import AppConfig
from django.conf import settings
from django.db import connections, models, DEFAULT_DB_ALIAS
from django.db.models.base import ModelBase
@donkirkby
donkirkby / stack_tracker.py
Created April 20, 2015 18:38
Record all different call stacks that arrive at a method
from collections import Counter
import traceback
stack_counts = Counter()
def record(limit=10):
stack = traceback.extract_stack(limit=limit+1)[:-1]
stack_report = '\n'.join(map(str, stack))
stack_counts[stack_report] += 1