Skip to content

Instantly share code, notes, and snippets.

@irbux
Last active November 2, 2016 14:22
Show Gist options
  • Save irbux/5efab7577cd4673120cf1f0accc04068 to your computer and use it in GitHub Desktop.
Save irbux/5efab7577cd4673120cf1f0accc04068 to your computer and use it in GitHub Desktop.
Quickstart ELK on local machine.
1) Download Kibana ( good versions for communication with elasticsearch 2.#, it is https://www.elastic.co/downloads/past-releases/kibana-4-3-3 or https://www.elastic.co/downloads/past-releases/kibana-4-4-2 )
2) Download Logstash we can use logstash 5.0.0 because version of logstash does not depend from version Kibana or ElasticSearch. You can download package from brew or archive from official elastic site.
3) Next step is a bind Kibana with elasticsearch. After you unpacked Kibana archive,
$ cd /kibana-4.3.3-darwin-x64/config
$ atom kibana.yml
And uncomment this line and save.
# The Elasticsearch instance to use for all your queries.
elasticsearch.url: "http://localhost:9200"
4) Run ElasticSearch and open new Terminal tab and paste:
curl -XPUT 'localhost:9200/.kibana?pretty' -d'
{
"index.mapper.dynamic": true
}
Case #1, if use gem 'logstasher':
a) add to Gemfile
gem 'logstasher'
run bundle
b) add to application.rb or <environment>.rb
config.logstasher.enabled = true
config.logstasher.controller_enabled = true
config.logstasher.job_enabled = true
config.logstasher.log_controller_parameters = true
config.logstasher.backtrace = true
config.logstasher.logger_path = 'log/logstasher.log'
c) create file logstash.conf with content below:
input {
file {
type => "rails logs"
path => "{PATH to RAils root }/log/logstasher.log"
codec => json {
charset => "UTF-8"
}
}
}
output {
# Print each event to stdout.
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["localhost:9200"]
}
}
d) for running Logstash - type in Terminal:
logstash -f /"path to your file"/logstash.conf
e) for running Kibana:
$ cd /kibana-4.3.3-darwin-x64/bin/
$ ./kibana
Open in browser http://0.0.0.0:5601/app/kibana#/discover
Note !
In my case I can not to collect and and logging controllers parameters, despite that I marked in application.rb config.logstasher.log_controller_parameters = true. Also I've tried to create in confing/initializers/logstasher.rb with custom fields :
if LogStasher.enabled?
LogStasher.add_custom_fields do |fields|
fields[:params] = request.filtered_parameters.to_json
end
end
but this decision also didn't help, in general all working good.
Case #2, if use gem 'logstash-logger':
a) add to Gemfile
gem 'logstash-logger'
gem 'lograge'
gem 'logstash-event'
run bundle
b) add to application.rb or <environment>.rb
config.logger = ActiveSupport::TaggedLogging.new(LogStashLogger.new('localhost', 5228))
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Logstash.new
config.logstash.buffer_max_items = 200
c) create file logstash.conf with content below:
input {
udp { # tried with tcp also
host => "localhost"
port => "5228"
}
}
filter {
json {
source => "message"
}
}
output {
stdout { }
elasticsearch {
hosts => ["localhost:9200"]
}
file {
path => "{PATH to RAils root }/log/logstash.log"
}
}
d) for running Logstash - type in Terminal:
logstash -f /"path to your file"/logstash.conf
e) for running Kibana:
$ cd /kibana-4.3.3-darwin-x64/bin/
$ ./kibana
Open in browser http://0.0.0.0:5601/app/kibana#/discover
Note !! This case isn't so good, I didn't figured out how to show logs in Self Rails console,
and why some logs like example on action POST doesn't logged.
NOTE !!!
For use Kibana on production server need to set up nginx for external access through credentials.
For serving already created log-files or files which were created during the time Logstash was stopped,
need to use additional local lightweight server Filebeat from elastic.co
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment