vyta

Example of using Azure Monitor to scrape Prometheus metrics from NGINX

NGINX Ingress Controller w/ Helm

Enable metrics endpoint:

helm install stable/nginx-ingress --name my-release --set controller.metrics.enabled=true

Custom VNET Notes

subnet used for your aks cluster should not overlap with any virtual networks in the environment. This creates a new subnet to be used only by AKS.
service-cidr:
- Must not be within the virtual network IP address range of your cluster
- Must not overlap with any other virtual networks with which the cluster virtual network peers
- Must not overlap with any on-premises IPs
dns-service-ip must be within service cidr AND must not use the first address, which is used for the kubernetes.default.svc.cluster.local address
docker-bridge-address must not be within vnet address space AND must not overlap with any other address ranges in use.

FYI

subnet used for your aks cluster should not overlap with any virtual networks in the environment. This creates a new subnet to be used only by AKS.
service prinicipal must have at least network contributor role to vnet rg
service-cidr:
- Must not be within the virtual network IP address range of your cluster
- Must not overlap with any other virtual networks with which the cluster virtual network peers
- Must not overlap with any on-premises IPs
- Must not be within the ranges 169.254.0.0/16, 172.30.0.0/16, or 172.31.0.0/16
dns-service-ip must be within service cidr AND must not use the first address, which is used for the kubernetes.default.svc.cluster.local address

Observation

Approximately 10% increase in CPU usage when running containerized application vs running the application directly on the host

Troubleshooting

Tools & Setup:

Windows Performance Recorder: built-in tool to troubleshoot performance related issues on Windows.

Alerts with Prometheus

The helm installation makes it particularly simple to get going: https://github.com/kubernetes/charts/tree/master/stable/prometheus.

Now you just have to worry about

Creating the alert
Creating the alert rules

Creating the alert

What kind of alert do you want and where do you want it to go? This is defined in the configMap for alertManager, which can be found in the alertmanagerFiles.alertmanager.yml portion of the values.yaml file of the helm chart.

	---
	kind: Cluster
	apiVersion: kind.sigs.k8s.io/v1alpha3
	nodes:
	- role: control-plane
	- role: worker
	- role: worker