Skip to content

Instantly share code, notes, and snippets.

@LexxFedoroff
Created January 26, 2016 07:15
Show Gist options
  • Save LexxFedoroff/587dc4eddf2e2416220a to your computer and use it in GitHub Desktop.
Save LexxFedoroff/587dc4eddf2e2416220a to your computer and use it in GitHub Desktop.
Logstash config for Cassandra logs
input {
file {
path => "/var/log/cassandra/system.log"
#start_position => "beginning"
}
#stdin { }
}
filter {
multiline {
pattern => "^(INFO|WARN|ERROR)"
what => "previous"
negate=> true
}
grok {
match => [ "message", "%{LOGLEVEL:level} %{DATA} %{TIMESTAMP_ISO8601:logdate} %{DATA} - %{GREEDYDATA:text}" ]
}
date {
match => [ "logdate", "yyyy-MM-dd HH:mm:ss,SSSS" ]
}
mutate {
add_field => {
"subsystem" => "cassandra"
}
}
}
output {
#stdout { codec => rubydebug }
elasticsearch {
hosts => ["elastic.local:9200"]
index => "cassandra-%{+YYYY.MM.dd}"
}
}
@LexxFedoroff
Copy link
Author

edit /etc/default/logstash and set LS_OPTS="-w 1"

@coder-sreeraj
Copy link

Getting Error :

{:type=>"filter", :name=>"multiline", :path=>"logstash/filters/multiline", :error_message=>"NameError", :error_class=>NameError, :error_backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:226:in namespace_lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:162:in legacy_lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:138:in lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:180:in lookup_pipeline_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in lookup'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:103:in plugin'", "(eval):12:in initialize'", "org/jruby/RubyKernel.java:1079:in eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:75:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:165:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:286:in create_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:95:in register_pipeline'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:313:in execute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:204:in run'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in `(root)'"]}
[2017-11-01T06:09:07,990][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Couldn't find any filter plugin named 'multiline'. Are you sure this is correct? Trying to load the multiline filter plugin resulted in this error: Problems loading the requested plugin named multiline of type filter. Error: NameError NameError"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment