-
-
Save pcolazurdo/4538923 to your computer and use it in GitHub Desktop.
| java -cp . logstash.runner agent -f logstash.conf -- web --backend elasticsearch:///?local |
| //Add this line to patterns/grok-patterns | |
| TELEFENGINX %{IPORHOST:site} %{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{URIPATHPARAM:request}(?: HTTP/%{NUMBE:httpversion})?|-)" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{QS:forwardedfor} | |
| input { | |
| file { | |
| type => nginx_web | |
| path => ["/var/log/nginx/*"] | |
| exclude => ["*.gz"] | |
| } | |
| } | |
| filter { | |
| grok { | |
| type => "nginx_web" | |
| patterns_dir => [ "/tmp/logstash/patterns" ] | |
| pattern => "%{TELEFENGINX}" | |
| } | |
| } | |
| output { | |
| elasticsearch { embedded => true } | |
| } |
| /// The following is the sample nginx.conf that defines the log used by the grok pattern defined above | |
| log_format combined_with_server '$http_host $remote_addr - $remote_user [$time_local] "$request" ' | |
| '$status $body_bytes_sent "$http_referer" ' | |
| '"$http_user_agent" "$http_x_forwarded_for"'; |
This does not seem to work anymore. This is the error I get when running with this configuration:
[2016-12-02T10:16:56,423][ERROR][logstash.agent ] fetched an invalid config {:config=>"input {\n file {\n type => nginx_web\n path => [\"/Users/justusblumer/Downloads/access_logs_curved/access-curved.log.*\"]\n exclude => [\"*.gz\"]\n }\n}\n\nfilter {\n grok {\n type => \"nginx_web\"\n patterns_dir => [ \"/tmp/logstash/patterns\" ]\n pattern => \"%{TELEFENGINX}\"\n }\n}\n\noutput {\n elasticsearch { embedded => true }\n}\n", :reason=>"The setting typein plugingrokis obsolete and is no longer available. You can achieve this same behavior with the new conditionals, like:if [type] == "sometype" { grok { ... } }. If you have any questions about this, you are invited to visit https://discuss.elastic.co/c/logstash and ask."}
@KIVagant I encountered the same problem when running this gork. There appears to be a typo, and the "R" was forgotten. It should read:
HTTP/%{NUMBER:httpversion}