logstash.md

/etc/logstash/conf.d

The key is that it is ok to have many logstash processes running with different input and output filters. And we need json in and json out for UDP. The non UDP/JSON versions do not seem to work with extra fields/MDC. The output encoder moving from logstash and into kibana was wrong so it would black hole all of our logs from logback.

# cat 50-udp.conf 

input {
    udp {
        port => 5001
        codec => json
    }
}


# cat 30-output.conf 
output {
  elasticsearch {
    hosts => ["localhost"]
    sniffing => true
    codec => json
  }
  stdout { codec => rubydebug }
}

The output/elasticsearch has to have the json codec or nothing happens.

service logstash stop
service logstash start
# Or just
service logstash restart

I got ELK to work with logback with the above and then using this docker image with those files applied.

elk {
       containerName "elk-df"
       image "sebp/elk:es234_l234_k452"
       portMapping(container: 5601, host: 5601)
       portMapping(container: 9200, host: 9200)
       portMapping(container: 5044, host: 5044)
       portMapping(container: 5000, host: 5000)
       portMapping(container: "5001/udp", host: 5001)
       portMapping(container: 5001, host: 5001)
   }

output {
  elasticsearch {
    hosts => ["localhost"]
    sniffing => true
    codec => json
  }
  stdout { codec => rubydebug }
}