John-Paul Bader hukl

hukl / parser_2.rb

Created February 12, 2011 19:01

	require 'time'

	class LogfileParser

	# Regexp to match the timestamps in the apache common log format
	TIME_REGEXP = /\[\d{2}\/\w{3}\/\d{4}\:\d{2}:\d{2}:\d{2}\s.{5}\]/

	def initialize path, starting_at
	raise ArgumentError unless File.exists?( path )

hukl / parser_3.rb

Created February 12, 2011 19:14

	require 'time'

	class LogfileParser

	TIME_REGEXP = /\[\d{2}\/\w{3}\/\d{4}\:\d{2}:\d{2}:\d{2}\s.{5}\]/

	def initialize path, starting_at
	raise ArgumentError unless File.exists?( path )

	@log = File.open( path )

hukl / parser_4.rb

Created February 12, 2011 19:30

	require 'time'

	class LogfileParser

	TIME_REGEXP = /\[\d{2}\/\w{3}\/\d{4}\:\d{2}:\d{2}:\d{2}\s.{5}\]/

	def initialize path, starting_at
	raise ArgumentError unless File.exists?( path )

	@log = File.open( path )

hukl / parser_5.rb

Created February 12, 2011 20:21

Yields the last 5 minutes of a pound log file

	require 'time'
	require 'stringio'

	class LogfileParser

	TIME_REGEXP = /\[\d{2}\/\w{3}\/\d{4}\:\d{2}:\d{2}:\d{2}\s.{5}\]/

	def initialize path, starting_at
	raise ArgumentError unless File.exists?( path )

hukl / parser_6.rb

Created February 14, 2011 22:03

	require 'stringio'

	module Munchies
	class Logfile

	include Enumerable

	# Matches timestamps in logfile
	TIME_REGEXP = /^\d{4}-\d{2}-\d{2}T\d{2}\:\d{2}\:\d{2}\b/

hukl / gist:961310

Created May 8, 2011 11:10 — forked from clintongormley/gist:961303

Ngram example

hukl / gist:961418

Created May 8, 2011 15:15

Sample ElasticSearch session for ngram

hukl / gist:989055

Created May 24, 2011 16:24

	int cursor = 0;

	SearchResponse searchResponse = new SearchRequestBuilder(client)
	.setIndices(alias)
	.setSearchType(SearchType.SCAN)
	.setQuery(matchAllQuery())
	.setSize(10)
	.setFrom(cursor)
	.setScroll(TimeValue.timeValueMinutes(10))
	.execute()

hukl / custom_analyzer.sh

Created May 24, 2011 16:34 — forked from karmi/elastic_search_ngram_analyzer_for_urls.sh

NGram Analyzer in ElasticSearch

	curl -X DELETE localhost:9200/custom_analyzer_test
	curl -X PUT localhost:9200/custom_analyzer_test -d '
	{
	"settings" : {
	"index" : {
	"number_of_shards" : 1,
	"number_of_replicas" : 0,
	"analysis" : {
	"analyzer" : {
	"url_analyzer" : {

hukl / base64.rb

Created June 21, 2011 19:08

Manual Base64 Padding

payload << "=" until ( payload.length % 4 == 0 )