Skip to content

Instantly share code, notes, and snippets.

@hamiltont
Last active August 29, 2015 14:06
Show Gist options
  • Save hamiltont/0bfc70ca5f4996dd773a to your computer and use it in GitHub Desktop.
Save hamiltont/0bfc70ca5f4996dd773a to your computer and use it in GitHub Desktop.
input {
file {
type => "couchpotato"
path => [ "/logs/couchpotato/CouchPotato.log" ]
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
# Get rid of any color codes
# NOT WORKING
#mutate {
# gsub => [ "message", "\\e\[0m", ""]
#}
grok {
match => [ "message", "%{MONTHNUM}-%{MONTHDAY} %{TIME} %{LOGLEVEL} \\e\[0m\[%{DATA:module}\] %{GREEDYDATA:couch_message}\\e\[0m" ]
}
}
output {
stdout { codec => rubydebug }
}
@hamiltont
Copy link
Author

Log example:

 09-04 19:54:28 INFO \e[0m[  couchpotato.core.loader] Loaded media_movie_providers_userscript: reddit\e[0m

@untergeek
Copy link

Removing grok, this is what I get:

$ bin/logstash -f c.conf
09-04 19:54:28 INFO \e[0m[  couchpotato.core.loader] Loaded media_movie_providers_userscript: reddit\e[0m
{
       "message" => "09-04 19:54:28 INFO [  couchpotato.core.loader] Loaded media_movie_providers_userscript: reddit",
      "@version" => "1",
    "@timestamp" => "2014-09-05T20:07:06.762Z",
          "host" => "aironaut.untergeek.net"
}

This says to me that the space after the [ matters.

So I changed your grok rule a little:

pattern => [ "%{MONTHNUM}-%{MONTHDAY} %{TIME} %{LOGLEVEL} \[%{SPACE}%{NOTSPACE:module}\] %{GREEDYDATA:couch_message}" ]

And, voila!

09-04 19:54:28 INFO \e[0m[  couchpotato.core.loader] Loaded media_movie_providers_userscript: reddit\e[0m
{
          "message" => "09-04 19:54:28 INFO [  couchpotato.core.loader] Loaded media_movie_providers_userscript: reddit",
         "@version" => "1",
       "@timestamp" => "2014-09-05T20:10:23.816Z",
             "host" => "aironaut.untergeek.net",
           "module" => "couchpotato.core.loader",
    "couch_message" => "Loaded media_movie_providers_userscript: reddit"
}

I presume you may want to save the LOGLEVEL and time and such, but didn't put those in for capture.

@untergeek
Copy link

Full config (updated to catch space or not):

input { stdin {} }

filter {
  # Get rid of any color codes
  mutate {
    gsub => [ "message", "\\e\[0m", ""]
  }
  grok {
    pattern => [ "%{MONTHNUM}-%{MONTHDAY} %{TIME} %{LOGLEVEL} \[(?:%{SPACE})%{NOTSPACE:module}\] %{GREEDYDATA:couch_message}" ]
  }
}

output { stdout { codec => rubydebug } }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment