Skip to content

Instantly share code, notes, and snippets.

View lbjay's full-sized avatar

Jay Luker lbjay

73 followers · 31 following
View GitHub Profile
@lbjay
lbjay / logstash graphite output
Created June 25, 2013 19:29
filter {
metrics {
meter => [ "http.%{response}" ]
add_tag => "metric"
}
}
output {
graphite {
tags => ["metric"]
debug => true
@lbjay
lbjay / gist:5630391
Created May 22, 2013 19:53
from adsabs.app import create_app
if __name__ == '__main__':
app = create_app()
app.run()
@lbjay
lbjay / invenio.py
Created May 14, 2013 18:18
getting keyword data from invenio
from invenio.search_engine import perform_request_search, get_record, print_record
# ... get the bibcodes somehow
bibs = ['2012PhRvA..86e3813B', ...]
for bib in bibs:
for rec_id in perform_request_search(p="970:%" % bib, cc="ADS metadata repository"):
# 'rec' is a list/tuple structure that is structured according to the MARC
# format. Field #653 is where any "free" keywords are stored; field #695 is for
@lbjay
lbjay / gist:5481851
Created April 29, 2013 14:19
def test_user_logged_in_cookie_1(self):
"""test that if the user is logged in and the cookies are not right, some new cookies are sent back"""
print "between here"
with self.app.test_client() as c:
with c.session_transaction() as sess:
sess['user_id'] = u'4d2203d39f'
sess['_fresh'] = True
resp = c.get('/')
#print resp.headers
@lbjay
lbjay / gist:5135092
Created March 11, 2013 15:35
INFO 2013-03-11T10:56:11 supybot tea called by "mistym!~mistym@pdpc/sup
porter/active/mistym".
ERROR 2013-03-11T10:56:11 supybot Uncaught exception in ['tea'].
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/supybot/callbacks.py", line 1180,
in _callCommand
self.callCommand(command, irc, msg, *args, **kwargs)
File "/usr/lib/pymodules/python2.6/supybot/utils/python.py", line 86,
in g
f(self, *args, **kwargs)
@lbjay
lbjay / gist:5051399
Last active December 14, 2015 07:29
[email protected]:~> 15:19:36 1644 $ mtr --report api.stackexchange.com
HOST: reallywow.com Loss% Snt Last Avg Best Wrst StDev
1. router1-dal.linode.com 0.0% 10 0.4 4.9 0.4 20.5 7.4
2. xe-2-0-0.car03.dllstx2.netwo 0.0% 10 0.2 6.8 0.2 66.0 20.8
3. po101.dsr01.dllstx2.networkl 0.0% 10 0.5 0.4 0.4 0.6 0.1
4. po21.dsr01.dllstx3.networkla 0.0% 10 0.6 0.7 0.6 1.0 0.1
5. ae16.bbr02.eq01.dal03.networ 0.0% 10 0.5 0.5 0.5 0.5 0.0
6. dls-bb1-link.telia.net 0.0% 10 0.5 0.5 0.5 0.7 0.0
7. nyk-bb1-link.telia.net 0.0% 10 61.5 51.9 45.6 86.1 13.0
8. nyk-b6-link.telia.net 0.0% 10 47.6 47.6 47.5 47.6 0.0
@lbjay
lbjay / gist:3749998
Created September 19, 2012 14:32
<?xml version="1.0" encoding="utf-8"?>
<opml version="1.0">
<head>
<title>huffduff</title>
<dateCreated>Thu, 12 Sep 2003 23:35:52 GMT</dateCreated>
<dateModified>Fri, 12 Sep 2003 23:45:37 GMT</dateModified>
</head>
<body>
<outline text="Podasts">
<outline text="Huffduffer" type="link" url="http://huffduffer.com/lbjay/rss" dateCreated="Thu, 12 Sep 2003 23:35:52 GMT"/>
@lbjay
lbjay / gist:3698781
Created September 11, 2012 14:10
[email protected]:/proj/adsx/fulltext-crawler> 10:04:52 5067 (master) $ curl -H "If-Modified-Since: Mon, 14 May 2012 13:56:18 GMT" -H "user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1 (NASA/ADS crawler; [email protected])" -I http://www.agu.org/journals/wr/wr1205/2011WR011460/2011WR011460.xml
HTTP/1.1 304 Not Modified
Date: Tue, 11 Sep 2012 14:08:55 GMT
Server: Apache/1.3.34 Ben-SSL/1.57 (Unix) PHP/5.2.8 mod_jk/1.2.1
ETag: "85dd93-1c21e-4fa7d482"
[email protected]:/proj/adsx/fulltext-crawler> 10:06:29 5068 (master) $ curl -H "If-Modified-Since: Mon, 14 May 2012 13:56:18 GMT" -H "user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1 (NASA/ADS crawler; [email protected])" -I http://www.agu.org/journals/wr/wr1205/2011WR011460/2011WR011460.xml
HTTP/1.1 200 OK
Date: Tue, 11 Sep 2012 14:09:09 GMT
Server: Apache/1.3.34 Ben-SSL/1.57 (Unix) PHP/5.2.8 mod_jk/1.2.1
@lbjay
lbjay / BibstemTransformer
Created August 16, 2012 19:26
Bibstem Transformer
package org.apache.solr.handler.dataimport;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
public class BibstemTransformer extends Transformer {
private static Pattern fourDigit = Pattern.compile("^\\d{4}.+");
private static Pattern lastFour = Pattern.compile("^[\\.\\d]+$");
@lbjay
lbjay / gist:2007670
Created March 9, 2012 17:31
python import slope
import os
import sys
import time
import httplib2
import threading
from Queue import Queue
import robotparser as rp
from optparse import OptionParser
from datetime import datetime, timedelta