Skip to content

Instantly share code, notes, and snippets.

@toast38coza
Last active March 15, 2016 15:08
Show Gist options
  • Save toast38coza/725479fcb33d562ad0af to your computer and use it in GitHub Desktop.
Save toast38coza/725479fcb33d562ad0af to your computer and use it in GitHub Desktop.
An example of creating a custom filter to parse a log file that is not in a standardized format
input {
stdin { }
}
filter {
grok {
match => { "message" => "%{DATE:date}[- ]%{TIME:time} - %{WORD:status}: %{GREEDYDATA:state}"}
}
}
## Add your filters here
output {
stdout { codec => rubydebug }
}
03-15-2016 16:26:37 - up: American Samoa
03-15-2016 16:27:55 - up: Ohio
03-15-2016 16:28:03 - up: Arkansas
03-15-2016 16:28:11 - up: Guam
03-15-2016 16:28:12 - down: Michigan
03-15-2016 16:28:20 - up: Connecticut
03-15-2016 16:28:25 - up: Georgia
03-15-2016 16:28:29 - up: Utah
03-15-2016 16:28:30 - up: Delaware
03-15-2016 16:28:34 - up: Rhode Island
03-15-2016 16:28:43 - up: Puerto Rico
03-15-2016 16:28:49 - up: Virginia
03-15-2016 16:28:50 - down: Minnesota
03-15-2016 16:28:56 - up: North Carolina
03-15-2016 16:29:03 - up: Wyoming
03-15-2016 16:29:06 - up: Massachusetts
03-15-2016 16:29:10 - up: Connecticut
03-15-2016 16:29:16 - up: Tennessee
03-15-2016 16:29:18 - up: Kansas
03-15-2016 16:29:24 - up: Kansas
03-15-2016 16:29:30 - up: Rhode Island
$ logstash -f logstash/examples/custom_filter.conf
>> Settings: Default pipeline workers: 4
>> Logstash startup completed
>> 03-15-2016 16:28:11 - up: Guam
{
"message" => "03-15-2016 16:28:11 - up: Guam",
"@version" => "1",
"@timestamp" => "2016-03-15T15:06:37.148Z",
"host" => "Christos-MacBook-Pro.local",
"date" => "03-15-2016",
"time" => "16:28:11",
"status" => "up",
"state" => "Guam"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment