alertmanager alerts to slack for prometheus-operator

values.yaml

.......some config.....
alertmanager:

  ## Deploy alertmanager
  ##
  enabled: true

  ## Service account for Alertmanager to use.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
  ##
  serviceAccount:
    create: true
    name: ""

  ## Configure pod disruption budgets for Alertmanager
  ## ref: https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
  ## This configuration is immutable once created and will require the PDB to be deleted to be changed
  ## https://github.com/kubernetes/kubernetes/issues/45398
  ##
  podDisruptionBudget:
    enabled: false
    minAvailable: 1
    maxUnavailable: ""

  ## Alertmanager configuration directives
  ## ref: https://prometheus.io/docs/alerting/configuration/#configuration-file
  ##      https://prometheus.io/webtools/alerting/routing-tree-editor/
  ##
  config:
    global:
      resolve_timeout: 5m
      slack_api_url: "<slack_url>"
    route:
      group_by: ['job']
      group_wait: 30s
      group_interval: 5m
      repeat_interval: 12h
      #receiver: 'slack'
      routes:
      - match:
          alertname: DeadMansSwitch
        receiver: 'null'
      - match:
        receiver: 'slack'
        continue: true
    receivers:
    - name: 'null'
    - name: 'slack'
      slack_configs:
      - channel: '#alertmanager-live'
        send_resolved: false
        title: '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] Monitoring Event Notification'
        text: >-
          {{ range .Alerts }}
            *Alert:* {{ .Annotations.summary }} - `{{ .Labels.severity }}`
            *Description:* {{ .Annotations.description }}
            *Graph:* <{{ .GeneratorURL }}|:chart_with_upwards_trend:> *Runbook:* <{{ .Annotations.runbook }}|:spiral_note_pad:>
            *Details:*
            {{ range .Labels.SortedPairs }} • *{{ .Name }}:* `{{ .Value }}`
            {{ end }}
          {{ end }}
.......some config.....

Thanks for sharing, just copy paste and adjust slack_url, it really works. Nice work.

Hey @bayucandra can you share the entire yaml with the ".......some config....."? How do I use this? is it a deployment or stateful or service yaml?

I have deployed Prometheus Operator in my cluster

Hi @pushp1997 its a portion of values.yaml file which you will find under your helm chart repo ( I hope you have used helm to install Prometheus Operator, in which case only you need this values.yaml file).

Does this work with Kube-prometheus https://github.com/prometheus-operator/kube-prometheus

I got the same working on Kube-prometheus

Hey @anoopl where in kube-prometheus manifests is this change applied to make it work? I have the same situation...

@dnaranjor on values.yaml you can see alertmanger: under that you can add something like below
config:
global:
resolve_timeout: 5m

@anoopl, thing is, I'm not using the Helm chart thingy, I applied manually the YAMLs under /manifest folder to my cluster, and I can't see that vaules.yaml file aywhere... :-(

@dnaranjor You will have values.yaml only when you install it with Helm.
This configs should be some where else in manifest, I am not sure, have not tried the manual installation

great! was so helpful :-)

how to test it

I got the same working on Kube-prometheus

doesnt work for me @anoopl

Thanks for sharing, just copy paste and adjust slack_url, it really works. Nice work.

u create an app and recieve for ex @app right ? @bayucandra

It works well. thanks, man

Thank you thank you thank you

Thank you!