Last active
March 15, 2016 15:08
-
-
Save toast38coza/725479fcb33d562ad0af to your computer and use it in GitHub Desktop.
An example of creating a custom filter to parse a log file that is not in a standardized format
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| input { | |
| stdin { } | |
| } | |
| filter { | |
| grok { | |
| match => { "message" => "%{DATE:date}[- ]%{TIME:time} - %{WORD:status}: %{GREEDYDATA:state}"} | |
| } | |
| } | |
| ## Add your filters here | |
| output { | |
| stdout { codec => rubydebug } | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 03-15-2016 16:26:37 - up: American Samoa | |
| 03-15-2016 16:27:55 - up: Ohio | |
| 03-15-2016 16:28:03 - up: Arkansas | |
| 03-15-2016 16:28:11 - up: Guam | |
| 03-15-2016 16:28:12 - down: Michigan | |
| 03-15-2016 16:28:20 - up: Connecticut | |
| 03-15-2016 16:28:25 - up: Georgia | |
| 03-15-2016 16:28:29 - up: Utah | |
| 03-15-2016 16:28:30 - up: Delaware | |
| 03-15-2016 16:28:34 - up: Rhode Island | |
| 03-15-2016 16:28:43 - up: Puerto Rico | |
| 03-15-2016 16:28:49 - up: Virginia | |
| 03-15-2016 16:28:50 - down: Minnesota | |
| 03-15-2016 16:28:56 - up: North Carolina | |
| 03-15-2016 16:29:03 - up: Wyoming | |
| 03-15-2016 16:29:06 - up: Massachusetts | |
| 03-15-2016 16:29:10 - up: Connecticut | |
| 03-15-2016 16:29:16 - up: Tennessee | |
| 03-15-2016 16:29:18 - up: Kansas | |
| 03-15-2016 16:29:24 - up: Kansas | |
| 03-15-2016 16:29:30 - up: Rhode Island |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ logstash -f logstash/examples/custom_filter.conf | |
| >> Settings: Default pipeline workers: 4 | |
| >> Logstash startup completed | |
| >> 03-15-2016 16:28:11 - up: Guam | |
| { | |
| "message" => "03-15-2016 16:28:11 - up: Guam", | |
| "@version" => "1", | |
| "@timestamp" => "2016-03-15T15:06:37.148Z", | |
| "host" => "Christos-MacBook-Pro.local", | |
| "date" => "03-15-2016", | |
| "time" => "16:28:11", | |
| "status" => "up", | |
| "state" => "Guam" | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment