deploy-prometheus.md

Deploy Prometheus

1. Fresh OC Cluster
2. On local machine (where you will be running ansible):

git clone https://github.com/openshift/openshift-ansible
cd openshift-ansible
# cherry-pick Zohar's PR which adds prometheus role
git remote add zgalor https://github.com/zgalor/openshift-ansible
git fetch zgalor
git checkout zgalor/prometheus-role

3. Create hosts file in your working directory with this content:

[OSEv3:children]
masters

[masters]
<ip of your openshift master>

4. Optional: If you wish Prometheus to create alerts, create the following alerting rules file:

groups:
- name: example-rules
  interval: 30s # defaults to global interval
  rules:
  - alert: Node Down
    expr: up{job="kubernetes-nodes"} == 0
    annotations:
      miqTarget: "ContainerNode"
      severity: "HIGH"
      url: "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
      message: "{{$labels.instance}} is down"
      description: "{{$labels.instance}} is down_d"
  # Uncomment the following alert rule if you want to see trivial alerts 
  # - alert: Dummy Alert
  #   expr: up{job="kubernetes-nodes"} > 0
  #   annotations:
  #     miqTarget: "ContainerNode"
  #     severity: "HIGH"
  #     url: "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
  #     message: "{{$labels.instance}} is up! That's probably a good thing"
  #     description: "{{$labels.instance}} is down_d"

5. You're all set! Run the following command:

ansible-playbook -i <path-to-hosts-file> -vvv --extra-vars \
 "openshift_prometheus_namespace=ANY_NAMESPACE_YOU_WANT \
  openshift_prometheus_additional_rules_file=<path-to-prometheus-alert-rules.yml>" \
  </path/to/openshift-ansible-root>/playbooks/byo/openshift-cluster/openshift-prometheus.yml