Route Docker Logs to ELK Stack

Instructions.md

• With Docker 1.8.0 shipped new log-driver for GELF via UDP, this means that the logs from Docker Container(s) can be shipped directly to the ELK stack for further analysis.
• This tutorial will illustrate how to use the GELF log-driver with Docker engine.
• Step 1: Setup ELK Stack:
  • docker run -d --name es elasticsearch
  • docker run -d --name logstash --link es:elasticsearch logstash -v /tmp/logstash.conf:/config-dir/logstash.conf logstash logstash -f /config-dir/logstash.conf
  • Note the config for Logstash can be found at this link
  • docker run --link es:elasticsearch -d kibana
• Once the ELK stack is up now let's fire up our nginx container which ships its logs to ELK stack.

• LOGSTASH_ADDRESS=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' logstash)
• docker run -d --net=host --log-driver=gelf --log-opt gelf-address=udp://$LOGSTASH_ADDRESS:12201 --log-opt gelf-tag="fe" nginx
• All logs from the nginx container will be shipped to our ELK stack for slicing and dicing.
• To verify that logs are being passed in visit http://<kibana-container-ip>:5601 follow through the setup and you should see the logs in Kibana.

Is there any way to increase the size of logs logstash can take? Logstash is breaking apart my logs into multiple messages which then fails to parse.

@AlecBruns see logstash-plugins/logstash-input-gelf#37 and moby/moby#22920 and moby/moby#22979

You can't use docker logging if you want to parse multiple lines.

I found that this does not work when using docker-compose has anyone else had luck? https://forums.docker.com/t/docker-loading-in-stack/33051

Can I use this to send logs directly in ELK and keep them in the journal of the host?

Here's a working verision which only requires docker-compose.
https://github.com/amalic/nginxelk

Please let me know what you think.