Last active
October 17, 2019 14:26
-
-
Save skriebel/8b80d51a75c99e64eb42 to your computer and use it in GitHub Desktop.
Logstash split field solution.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Problem: We have a log line that includes a perl class that we want to log the class | |
and method in their respected fields. An example class and method in perl: | |
Animal::Dog::bark | |
In this example, "bark" is the method. "Animal::Dog" is the class. | |
After some searching and hacking, I found a solution that works with Logstash 1.4.2 | |
Assume the input is "Animal::Dog::bark". | |
For completeness, I'm going to just add my entire configuration file I used for testing. | |
Comments are included and should explain what's going on. | |
input { | |
stdin { } | |
} | |
filter { | |
grok { | |
match => { "message" => "%{GREEDYDATA:api_class}" } | |
} | |
mutate { | |
# split the field on :: | |
split => ["api_class" , "::"] | |
# save the last element of the array as the api_method. | |
add_field => ["api_method", "%{[api_class][-1]}" ] | |
} | |
ruby { | |
# Go directly to the array and remove the last element. | |
code => "event['api_class'].pop()" | |
} | |
mutate { | |
# Join together whats left as the class name. | |
join => ["api_class", "::"] | |
} | |
} | |
output { | |
stdout { codec => rubydebug } | |
} | |
I tried to use mutate's remove_field to remove the last element of the array but it didn't work. | |
There are tickets created and possibly even a fix in the new version, however, this should continue | |
to work as long as the ruby filter is around. | |
The output: | |
{ | |
"message" => "Animal::Dog::bark", | |
"@version" => "1", | |
"@timestamp" => "2014-12-09T13:38:58.178Z", | |
"host" => "host.example.com", | |
"api_class" => "Animal::Dog", | |
"api_method" => "bark" | |
} | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I totally forgot about this gist! I'm glad it's helping others and inspiring better solutions.