Skip to content

Instantly share code, notes, and snippets.

@winkler1
Created August 14, 2012 17:51
Show Gist options
  • Select an option

  • Save winkler1/3351223 to your computer and use it in GitHub Desktop.

Select an option

Save winkler1/3351223 to your computer and use it in GitHub Desktop.
Unit-testing Grok assertions
require 'rubygems'
require 'solid_assert' # http://jorgemanrubia.net/2011/09/19/solid_assert-a-simple-ruby-assertion-utility/
require 'grok-pure'
SolidAssert.enable_assertions
# Grok Docs: http://rubydoc.info/gems/jls-grok/Grok
g = Grok.new
# Test SU Patterns.
g.add_patterns_from_file("E:/downloads/logstash/SUPatterns/SU")
g.add_patterns_from_file("E:/downloads/logstash/logstash-1.1.1-monolithic/patterns/grok-patterns")
def assert_maps_equal ( m1, m2 )
keys1 = m1.keys.sort!
keys2 = m2.keys.sort!
assert keys1 == keys2, "Map1 keys(#{keys1}) != Map2 keys(#{keys2}) "
m1.each { |key,m1val|
m2val = m2[key]
assert m1val == m2val, "Map1[#{key}](#{m1val.to_s}) != Map2[#{key}](#{m2val.to_s})"
}
end
def should_match( g, pattern, textToMatch, resultMap )
g.compile(pattern)
match = g.match(textToMatch)
# puts g.expanded_pattern
assert (match.kind_of?Grok::Match), "'#{textToMatch}' does not match pattern '#{pattern}'"
assert_maps_equal match.captures, resultMap
end
#------------ Test individual patterns
EXPECTED_MATCHES = {
'SUTIMESTAMP' => '2012/08/12 23:59:30.338',
'SUSESSIONID' => '14B56EC9A83F6AD6CDDDC8D3272E32A3'
}
EXPECTED_MATCHES.each { |patternName,matchingText|
g.compile("%{#{patternName}}")
match = g.match(matchingText)
# puts g.expanded_pattern
should_match( g, "%{#{patternName}}", matchingText, {patternName=>[matchingText]})
}
#----------------- Now do a full line.
line='2012/03/20 01:24:00.884 [INFO ] [ProductPluginService] placeParamName is null or blank for : origin2 [B28C448CE23F44D140F20E77967ED02B]'
suPattern = '^%{SUTIMESTAMP:mytime} \\[%{LOGLEVEL:log_level}\\s*\] \\[%{WORD:java_class}\\s*\]%{GREEDYDATA:message}\\[%{SUSESSIONID:session_id}\\]$'
EXPECTED = {
'SUTIMESTAMP:mytime' => ['2012/03/20 01:24:00.884'],
'LOGLEVEL:log_level' => ['INFO'],
'WORD:java_class' => ['ProductPluginService'],
'GREEDYDATA:message' => [' placeParamName is null or blank for : origin2 '],
'SUSESSIONID:session_id' => ['B28C448CE23F44D140F20E77967ED02B']
}
should_match( g, suPattern, line, EXPECTED)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment