lbjay’s gists

lbjay / gist:aa15f52b8f872ad6ba48

Last active March 12, 2019 08:47

@harvardedits configured IP ranges

lbjay / force_encoding.rb

Created June 11, 2014 13:11

Hackish workaround to wrongly encoded event data in the logstash pipeline


	require "logstash/filters/base"
	require "logstash/namespace"

	# This filter is a mildly hackish workaround for wrongly character encoded event data
	# and the issues it causes in the logstash pipeline. (See: LOGSTASH-1443,LOGSTASH-1308
	# LOGSTASH-1353, etc)
	#
	# Simply list the event fields that are causing you problems, along with a tag to
	# attach for offending events, like so:

lbjay / gist:11259832

Last active August 29, 2015 14:00

logstash - invalid byte sequence in UTF-8

	{
	:timestamp=>"2014-04-24T11:09:00.344000+0000",
	:message=>"Exception in filterworker",
	"exception"=>#<ArgumentError: invalid byte sequence in UTF-8>,
	"backtrace"=>[
	"org/jruby/RubyRegexp.java:1674:in `=~'",
	"(eval):234:in `initialize'",
	"org/jruby/RubyProc.java:271:in `call'",
	"/opt/logstash/lib/logstash/pipeline.rb:262:in `filter'",
	"/opt/logstash/lib/logstash/pipeline.rb:203:in `filterworker'",

lbjay / gist:9536563

Created March 13, 2014 20:41

	67.134.207.238 - - [13/Mar/2014:15:51:37 -0400] "GET /adsabs/search/?q=+author%3A%22long%2C+D.%22&month_from=&year_from=&month_to=&year_to=&db_f=%28astronomy+OR+phys
	ics%29&nr=&bigquery=&aut_f=-(%221%2FLong%2C+D%2FLong%2C+D+A%22+OR+%221%2FLong%2C+D%2FLong%2C+David+G%22+OR+%221%2FLong%2C+D%2FLong%2C%20D%20G%22+OR+%221%2FLong%2C%20
	D%2FLong%2C%20Daniel%20C%22+OR+%221%2FLong%2C%20D%2FLong%2C%20Dan%22+OR+%221%2FLong%2C%20D%2FLong%2C%20Didier%22+OR+%221%2FLong%2C%20D%2FLong%2C%20Donald%22+OR+%221%
	2FLong%2C%20D%2FLong%2C%20David%20T%22)&aut_f=-(%221%2FLong%2C%20D%2FLong%2C%20D%20D%22+OR+%221%2FLong%2C%20D%2FLong%2C%20D%20G%20F%22)&aut_f=-(%221%2FLong%2C%20D%2F
	Long%2C%20D%20T%22+OR+%221%2FLong%2C%20D%2FLong%2C%20David%20A%22+OR+%221%2FLong%2C%20D%2FLong%2C%20Di%22+OR+%221%2FLong%2C%20D%2FLong%2C%20Dongyang%22+OR+%221%2FLon
	g%2C%20D%2FLong%2C%20D%20C%22+OR+%221%2FLong%2C%20D%2FLong%2C%20D%20F%22)&aut_f=-(%221%2FLong%2C%20D%2FLong%2C%20Darrel%20G%20F%22+OR+%221%2FLong%2C%20D%2FLong%2C%20
	Dong%20Yang%22+OR+%221%2FLon

lbjay / gist:9194543

Created February 24, 2014 18:56

zoia fail

	INFO 2014-02-24T13:54:46 supybot Creating new Irc for freenode.
	INFO 2014-02-24T13:54:46 supybot Sending PASS command, not logging the password.
	INFO 2014-02-24T13:54:46 supybot Connecting to chat.us.freenode.net:6667.
	INFO 2014-02-24T13:54:46 supybot Loading plugins (connecting to freenode).
	WARNING 2014-02-24T13:54:46 supybot You need to have dictclient installed to use this plugin. Download it at <http://quux.org:70/devel/dictclient>
	WARNING 2014-02-24T13:54:48 supybot Failed to load Words: No module named Words.
	WARNING 2014-02-24T13:54:48 supybot Error connecting to chat.us.freenode.net:6667: An error occurred while connecting: -9: Unknown error 4294967287.
	INFO 2014-02-24T13:54:51 supybot Connecting to holmes.freenode.net:6667.
	INFO 2014-02-24T13:54:56 supybot Server holmes.freenode.net has version ircd-seven-1.1.3
	INFO 2014-02-24T13:54:56 supybot Got end of MOTD from holmes.freenode.net

lbjay / build_docs.py

Created January 17, 2014 16:51

updated build_docs.py

	'''
	Created on Feb 28, 2013

	@author: jluker
	'''

	import os
	import sys
	sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

lbjay / gist:7828935

Created December 6, 2013 17:36

	2013-12-05 05:06:01,212: Pipeline - WARNING - Pipeline started
	2013-12-05 05:06:01,423: Pipeline - WARNING - Last extraction was fine: proceeding with a new one
	2013-12-05 20:16:05,896: Pipeline - ERROR - ERROR: Record 2708047 has no bibcode.
	2013-12-05 20:16:06,056: Pipeline - ERROR - ERROR: Record 3491793 has no bibcode.
	2013-12-05 20:16:06,685: Pipeline - ERROR - ERROR: Record 2679160 has no bibcode.
	2013-12-05 20:16:06,740: Pipeline - ERROR - ERROR: Record 3491797 has no bibcode.
	2013-12-05 20:16:06,968: Pipeline - ERROR - ERROR: Record 3491795 has no bibcode.
	2013-12-05 20:16:09,427: Pipeline - ERROR - ERROR: Record 2015453 has no bibcode.
	2013-12-05 20:16:10,211: Pipeline - ERROR - ERROR: Record 2540266 has no bibcode.
	2013-12-05 20:16:10,350: Pipeline - ERROR - ERROR: Record 8481207 has no bibcode.

lbjay / gist:7010064

Created October 16, 2013 15:49

	bash-4.1$ less logs_pipeline/pipeline_2013_10_16.log
	2013-10-16 11:47:14,541: Pipeline - WARNING - Pipeline started
	2013-10-16 11:47:15,741: Pipeline - WARNING - Last extraction was not fine: recovering
	2013-10-16 11:47:16,847: Pipeline - WARNING - Process-1 (Manager) Starting all the workers
	2013-10-16 11:47:16,854: Pipeline - WARNING - Process-1:1 (done bibcodes worker) Process started
	2013-10-16 11:47:16,856: Pipeline - WARNING - Process-1:2 (probl. bibcodes worker) Process started
	2013-10-16 11:47:16,859: Pipeline - WARNING - Process-1:3 (upload worker) Process started
	2013-10-16 11:47:16,861: Pipeline - WARNING - Process-1:4 (upload worker) Process started
	2013-10-16 11:47:16,864: Pipeline - WARNING - Process-1:5 (upload worker) Process started
	2013-10-16 11:47:16,867: Pipeline - WARNING - Process-1:6 (upload worker) Process started

lbjay / gist:6904760

Created October 9, 2013 17:13

	@contextmanager
	def socket_send_wrapper(send_args):
	def fake_send(args, *kwargs):
	send_args.append(args, kwargs)
	mocked_send = patch("socket.send", fake_send)
	mocked_send.start()
	yield
	mocked_send.stop()

lbjay / gist:6167230

Created August 6, 2013 18:32

Jay Luker lbjay