Last active
November 2, 2016 14:22
-
-
Save irbux/5efab7577cd4673120cf1f0accc04068 to your computer and use it in GitHub Desktop.
Quickstart ELK on local machine.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1) Download Kibana ( good versions for communication with elasticsearch 2.#, it is https://www.elastic.co/downloads/past-releases/kibana-4-3-3 or https://www.elastic.co/downloads/past-releases/kibana-4-4-2 ) | |
2) Download Logstash we can use logstash 5.0.0 because version of logstash does not depend from version Kibana or ElasticSearch. You can download package from brew or archive from official elastic site. | |
3) Next step is a bind Kibana with elasticsearch. After you unpacked Kibana archive, | |
$ cd /kibana-4.3.3-darwin-x64/config | |
$ atom kibana.yml | |
And uncomment this line and save. | |
# The Elasticsearch instance to use for all your queries. | |
elasticsearch.url: "http://localhost:9200" | |
4) Run ElasticSearch and open new Terminal tab and paste: | |
curl -XPUT 'localhost:9200/.kibana?pretty' -d' | |
{ | |
"index.mapper.dynamic": true | |
} | |
Case #1, if use gem 'logstasher': | |
a) add to Gemfile | |
gem 'logstasher' | |
run bundle | |
b) add to application.rb or <environment>.rb | |
config.logstasher.enabled = true | |
config.logstasher.controller_enabled = true | |
config.logstasher.job_enabled = true | |
config.logstasher.log_controller_parameters = true | |
config.logstasher.backtrace = true | |
config.logstasher.logger_path = 'log/logstasher.log' | |
c) create file logstash.conf with content below: | |
input { | |
file { | |
type => "rails logs" | |
path => "{PATH to RAils root }/log/logstasher.log" | |
codec => json { | |
charset => "UTF-8" | |
} | |
} | |
} | |
output { | |
# Print each event to stdout. | |
stdout { | |
codec => rubydebug | |
} | |
elasticsearch { | |
hosts => ["localhost:9200"] | |
} | |
} | |
d) for running Logstash - type in Terminal: | |
logstash -f /"path to your file"/logstash.conf | |
e) for running Kibana: | |
$ cd /kibana-4.3.3-darwin-x64/bin/ | |
$ ./kibana | |
Open in browser http://0.0.0.0:5601/app/kibana#/discover | |
Note ! | |
In my case I can not to collect and and logging controllers parameters, despite that I marked in application.rb config.logstasher.log_controller_parameters = true. Also I've tried to create in confing/initializers/logstasher.rb with custom fields : | |
if LogStasher.enabled? | |
LogStasher.add_custom_fields do |fields| | |
fields[:params] = request.filtered_parameters.to_json | |
end | |
end | |
but this decision also didn't help, in general all working good. | |
Case #2, if use gem 'logstash-logger': | |
a) add to Gemfile | |
gem 'logstash-logger' | |
gem 'lograge' | |
gem 'logstash-event' | |
run bundle | |
b) add to application.rb or <environment>.rb | |
config.logger = ActiveSupport::TaggedLogging.new(LogStashLogger.new('localhost', 5228)) | |
config.lograge.enabled = true | |
config.lograge.formatter = Lograge::Formatters::Logstash.new | |
config.logstash.buffer_max_items = 200 | |
c) create file logstash.conf with content below: | |
input { | |
udp { # tried with tcp also | |
host => "localhost" | |
port => "5228" | |
} | |
} | |
filter { | |
json { | |
source => "message" | |
} | |
} | |
output { | |
stdout { } | |
elasticsearch { | |
hosts => ["localhost:9200"] | |
} | |
file { | |
path => "{PATH to RAils root }/log/logstash.log" | |
} | |
} | |
d) for running Logstash - type in Terminal: | |
logstash -f /"path to your file"/logstash.conf | |
e) for running Kibana: | |
$ cd /kibana-4.3.3-darwin-x64/bin/ | |
$ ./kibana | |
Open in browser http://0.0.0.0:5601/app/kibana#/discover | |
Note !! This case isn't so good, I didn't figured out how to show logs in Self Rails console, | |
and why some logs like example on action POST doesn't logged. | |
NOTE !!! | |
For use Kibana on production server need to set up nginx for external access through credentials. | |
For serving already created log-files or files which were created during the time Logstash was stopped, | |
need to use additional local lightweight server Filebeat from elastic.co |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment