Prometheus Operator CR Testing Manifests

prometheus-cr-test.md

Prometheus Operator Custom Resource (CR) Testing Manifests

Ever searched for a simple setup to test the exposure of a /metrics endpoint and tried to add this endpoint to Prometheus using a ServiceMonitor CR? Well, use the manifests below and you are good to go :).

Creation

---
apiVersion: v1
kind: Namespace
metadata:
  name: testing
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: runasany
spec:
  allowPrivilegeEscalation: false
  fsGroup:
    rule: RunAsAny
  requiredDropCapabilities:
  - ALL
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - configMap
  - emptyDir
  - projected
  - secret
  - downwardAPI
  - persistentVolumeClaim
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: runasany
rules:
- apiGroups:
  - extensions
  resourceNames:
  - runasany
  resources:
  - podsecuritypolicies
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp-runasany-all-serviceaccounts
  namespace: testing
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: runasany
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:testing
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: prometheus-example-app
  name: prometheus-example-app
  namespace: testing
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: prometheus-example-app
  template:
    metadata:
      labels:
        app.kubernetes.io/name: prometheus-example-app
    spec:
      containers:
      - name: prometheus-example-app
        image: quay.io/brancz/prometheus-example-app:v0.3.0
        ports:
        - name: web
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: testing-service
  namespace: testing
  labels:
    service: prometheus-example-app-svc
spec:
  selector:
    app.kubernetes.io/name: prometheus-example-app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: test-servicemonitor
  namespace: testing
spec:
  selector:
    matchLabels:
      service: prometheus-example-app-svc
  endpoints:
  - targetPort: 8080
    path: /metrics
    interval: 5s
---
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: test-prometheusrule
  namespace: testing
spec:
  groups:
  - name: testing-rules
    rules:
    - alert: TestingRule1
      expr: http_requests_total{code="404",method="get"} > 3
      for: 1h
      labels:
        severity: warning

Testing

kubectl run --generator=run-pod/v1 --rm -it --image busybox:1.32.0 test-client -n testing -- sh

wget -qO - http://testing-service:8080/
wget -qO - http://testing-service:8080/err

Removal

kubectl delete -n testing pod test-client
kubectl delete -n testing prometheusrule test-prometheusrule
kubectl delete -n testing servicemonitor test-servicemonitor
kubectl delete -n testing service testing-service
kubectl delete -n testing deployment prometheus-example-app 
kubectl delete -n testing rolebinding psp-runasany-all-serviceaccounts
kubectl delete -n testing clusterrole runasany 
kubectl delete -n testing psp runasany
kubectl delete -n testing namespace testing