Skip to content

Instantly share code, notes, and snippets.

View alexstorer's full-sized avatar

Alex Storer alexstorer

View GitHub Profile
@alexstorer
alexstorer / gmap_api.R
Created December 14, 2012 21:33
Download images from the streetview API
# An R Script to do some scraping
getSearchString <- function(loc,heading,pitch) {
s <- paste('http://maps.googleapis.com/maps/api/streetview?size=640x640',
paste('location=',loc,sep=''),
paste('heading=',heading,sep=''),
paste('pitch=',pitch,sep=''),
'sensor=false',
sep = '&')
s <- gsub(' ','%20',s)
@alexstorer
alexstorer / parse.py
Created December 13, 2012 21:48
Parse highlighter files.
import re
fname = 'inp.txt'
colors = ['red','green','blue','orange','yellow']
colorheaders = ['red is for X','green is for y','blue','orange','yellow']
colorstrings = []
for c in colors:
@alexstorer
alexstorer / extract_doe.py
Created December 6, 2012 21:35
This script will extract the PS-B and PS-E tables from DOE .sim building files.
import glob
import re
import csv
'''
This script will extract the PS-B and PS-E tables from
DOE .sim building files.
This script was tested on DOE 2.1e .sim files.
@alexstorer
alexstorer / example.py
Created December 5, 2012 21:53
Parsing Text Files
import glob
import re
import csv
allfiles = glob.glob('1972_1998_batch/*.sim')
fw = open('psb.csv','w')
dw = csv.DictWriter(fw,["filename","month","electricity","gas"])
dw.writeheader()
fwt = open('psb_total.csv','w')
@alexstorer
alexstorer / parsetext.py
Created November 30, 2012 19:37
How to scrape text for regular expressions
import re
import csv
import glob
fc = open('signers.csv','w')
c = csv.DictWriter(fc,["Name","Chamber","Number","Year Filename"])
c.writeheader()
# I used the pdftotext utility to convert the pdf documents
# Look here for details: http://www.bluem.net/en/mac/packages/
@alexstorer
alexstorer / parsexml.py
Created November 8, 2012 19:21
Patent Processing
# Use this file by navigating in the terminal to the directory where your file is located and typing:
# python parsexml.py [name of xml file]
# You can parse multiple xml files by using the wildcard operator.
# To process every xml file in a directory, do this:
# python parsexml.py *.xml
from lxml import etree
import re
import csv
import os.path as op
@alexstorer
alexstorer / gist:3455295
Created August 24, 2012 20:30
For PyLucene! # This file demonstrates how to... # 1) Make a new analyzer # 2) Make a new filter # 3) Apply an analyzer chain to a string (via a query) # 4) Include phrases as tokens
# This file demonstrates how to...
# 1) Make a new analyzer
# 2) Make a new filter
# 3) Apply an analyzer chain to a string (via a query)
# 4) Include phrases as tokens
from lucene import *
class AnalyzerUtils(object):
@alexstorer
alexstorer / gist:3307317
Created August 9, 2012 19:24
Convert relevant html docs to text!
# open the files in the directory, parse them and get the text.
#eg.text_content()
import glob
from lxml import etree
import lxml.html
import csv
import urllib2
@alexstorer
alexstorer / gist:3297899
Created August 8, 2012 19:30
How to submit a basic matlab job to the RCE using condor_submit
Universe = vanilla
Executable = /usr/local/bin/matlab
Arguments = -nodisplay -r \"addpath('/nfs/home/A/astorer/Work/dchan');
test($(Process))\"
when_to_transfer_output = ON_EXIT_OR_EVICT
transfer_output_files = out.$(PROCESS)
input =
output = out.$(Process)
error = error.$(Process)
@alexstorer
alexstorer / gist:3295676
Created August 8, 2012 15:00
Pseudo-code for Matt's Experiment
num_statements = 5;
num_blocks = 9;
for run = 1:8
% 150 statements, 50 of each type
physical_order = Shuffle(1:50);
mental_order = Shuffle(1:50);
emotional_order = Shuffle(1:50);
physical_num = 1