ELK setup on ubuntu

ELK

2 node Cluster:

    master node:  elasticsearch , logstash , kibana , niginx
    data node: elasticsearch worker


Installation:

  - all the above services are install using standard ubuntu package manager


Configuration:

  - Both the nodes are working as cluster
  - Cluster configuration defined in /etc/elasticsearch/elasticsearch.yml on both nodes
  - Data and logs directories on data node : /usr/share/elasticsearch/data , /usr/she/elasticsearch/logs
  - Kibana is configured to contact elasticsearch on localhost:9200
  - Ngnix standard configuration location , modified config file to proxy incomming requests to kibana
  - Logstash configuration: /usr/share/logstash/config ( main logstash config) 
    and /usr/shre/logstash/conf.d/ ( ingest filters)
  - Logstash is configred to use elasticsearch on localhost:9200
  - To see all the listening ports: ss -tlnu 
  - Curator need to be installed using package manager and configured to clean up old indices 
  

Network:

  - Master and Data node communicate over private network
  - Kibana is accessable using master public ip with username and password 
  
  
  
Curator:

  - Curator is available as a standard command on master node
  - The config files are in / root
  - To clean up old indices, run:   curator ./delete_index.yml --config ./curator.yml
  - it will clean up indices older than 90 days