Skip to content

Instantly share code, notes, and snippets.

@rikatz
Created November 5, 2018 18:21
Show Gist options
  • Save rikatz/f723a2b98937d6f7350bede44e30f5d8 to your computer and use it in GitHub Desktop.
Save rikatz/f723a2b98937d6f7350bede44e30f5d8 to your computer and use it in GitHub Desktop.
Kubernetes Events Export with Metricbeat + Logstash
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-config
namespace: kube-system
labels:
k8s-app: metricbeat
data:
metricbeat.yml: |-
metricbeat.config.modules:
path: /etc/metricbeat/modules.d/*.yml
reload.enabled: false
output.logstash:
hosts: ['${LOGSTASH_HOST:127.0.0.1}:${LOGSTASH_PORT:5044}']
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-modules
namespace: kube-system
labels:
k8s-app: metricbeat
data:
kubernetes.yml: |-
- module: kubernetes
metricsets:
- event
in_cluster: true
---
apiVersion: v1
kind: ConfigMap
metadata:
name: logstash
namespace: kube-system
labels:
k8s-app: logstash
data:
logstash.conf: |-
input {
beats {
port => "5044"
}
}
output {
stdout { codec => rubydebug }
}
---
# Deploy singleton instance in the whole cluster for some unique data sources, like kube-state-metrics
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: metricbeat
namespace: kube-system
labels:
k8s-app: metricbeat
spec:
template:
metadata:
labels:
k8s-app: metricbeat
spec:
serviceAccountName: metricbeat
containers:
- name: logstash
image: docker.elastic.co/logstash/logstash:6.4.2
volumeMounts:
- name: logstash
mountPath: /usr/share/logstash/pipeline
readOnly: true
env:
- name: XPACK_MONITORING_ENABLED
value: "false"
- name: metricbeat
image: docker.elastic.co/beats/metricbeat:6.4.2
args: [
"-c", "/etc/metricbeat.yml",
"-e",
]
env:
- name: LOGSTASH_HOST
value: "127.0.0.1"
- name: LOGSTASH_PORT
value: "5044"
securityContext:
runAsUser: 0
resources:
limits:
memory: 1000Mi
volumeMounts:
- name: config
mountPath: /etc/metricbeat.yml
readOnly: true
subPath: metricbeat.yml
- name: modules
mountPath: /etc/metricbeat/modules.d
readOnly: true
volumes:
- name: config
configMap:
defaultMode: 0600
name: metricbeat-config
- name: modules
configMap:
defaultMode: 0600
name: metricbeat-modules
- name: logstash
configMap:
defaultMode: 0644
name: logstash
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: metricbeat
subjects:
- kind: ServiceAccount
name: metricbeat
namespace: kube-system
roleRef:
kind: ClusterRole
name: metricbeat
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: metricbeat
labels:
k8s-app: metricbeat
rules:
- apiGroups: [""]
resources:
- nodes
- namespaces
- events
- pods
verbs: ["get", "list", "watch"]
- apiGroups: ["extensions"]
resources:
- replicasets
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources:
- statefulsets
- deployments
verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: metricbeat
namespace: kube-system
labels:
k8s-app: metricbeat
@rikatz
Copy link
Author

rikatz commented Nov 5, 2018

This GIST/Deploy will connect into Kubernetes API Server (with in_cluster options), map the events and export them to Logstash.

I'm not sending them directly to Elasticsearch, as I do intend to send them to Graylog, to each users/namespace stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment