Last active
December 24, 2015 13:49
-
-
Save nofxx/6807671 to your computer and use it in GitHub Desktop.
Centralized Ruby/Rails/MongoDB/Redis/Sidekiq/Elasticsearch/Haproxy/PG/OS -> syslog-ng -> logstash -> elasticsearch -> kibana =D
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
# Logstash conf! | |
# | |
input { | |
tcp { | |
host => "0.0.0.0" | |
port => 514 | |
type => syslog | |
} | |
udp { | |
host => "0.0.0.0" | |
port => 514 | |
type => syslog | |
} | |
} | |
filter { | |
grok { | |
match => { "message" => "(?:<%{INT:priority}>)?%{SYSLOGBASE2} (?:\s?%{LOGLEVEL:log_level} )?(?:\s?%{WORD:log_format}: )?%{GREEDYDATA:syslog_message}" } | |
} | |
syslog_pri { } | |
if ("_grokparsefailure" in [tags]) { | |
mutate { | |
replace => ["@message", "TOPARSE: %{message}"] | |
} | |
} else if [log_format] == "json" { | |
mutate { | |
gsub => ["syslog_message", "@timestamp", "syslog_timestamp"] | |
} | |
json { | |
source => "syslog_message" | |
} | |
mutate { | |
replace => ["@message", "%{message}"] | |
} | |
} else { | |
mutate { | |
replace => [ "@message", "%{syslog_message}" ] | |
} | |
} | |
mutate { | |
remove_field => [ | |
"syslog_hostname", "syslog_message", "syslog_timestamp", | |
"syslog_severity_code", "syslog_facility_code", | |
"message" #facility_label", "severity_label" | |
] | |
} | |
} | |
output { | |
stdout { | |
debug => true | |
} | |
elasticsearch { | |
# bind_host => "foo" | |
host => "localhost" | |
embedded => false | |
} | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'log4r' | |
require 'log4r/yamlconfigurator' | |
require 'log4r/outputter/syslogoutputter' | |
module App | |
class Application < Rails::Application | |
log4r_config= YAML.load_file(File.join(File.dirname(__FILE__),"log4r.yml")) | |
Log4r::YamlConfigurator.decode_yaml( log4r_config['log4r_config'] ) | |
config.logger = Log4r::Logger['rails'] | |
config.log_level = Log4r::DEBUG | |
Mongoid.logger = Log4r::Logger['mongoid'] | |
Moped.logger = Log4r::Logger['mongoid'] | |
... | |
# Lograge | |
config.lograge.enabled = true | |
config.lograge.formatter = Lograge::Formatters::Logstash.new |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
destination logstash { udp("logs.xxx.com"); }; | |
log { source(src); destination(logstash); }; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Syslog parameters correctly parsed: LOGLEVEL (INFO|WARN...), LOGTITLE (logtitle: some log...)