jewzaam

Setup Prometheus to scrape external node-exporter

This came out of an ansible role I was creating to monitor some external hosts that run node-exporter. I've since moved away from exposting node-exporter and am using prometheus pushgateway instead. This gist exists to capture what I learned.

How does it work?

Deploy the endpoint, service, and servicemonitor in the openshift-monitoring namespace. You should be able to deploy to another project but you'll maybe have problems with reading data. A couple of notes on that at the end

Ansible snippet to make this happen (I have not tested this). Assumes the host it runs on has ability to run oc commands against openshift-monitoring.

jewzaam

import boto3

ec = boto3.client('ec2')

def lambda_handler(event, context):
    # https://serverlesscode.com/post/lambda-schedule-ebs-snapshot-backups/
    reservations = ec.describe_instances(
            Filters=[
                {'Name': 'tag-key', 'Values': ['openshiftClusterID']},
            ]

jewzaam

$ go run test.go 
2019/03/01 08:51:34 PASS: test=openshift-nmalik, expected=true, matched=true
2019/03/01 08:51:34 FAIL: test=nmalik-openshift-test, expected=false, matched=true
2019/03/01 08:51:34 PASS: test=nmalik-test, expected=false, matched=false

jewzaam

https://github.com/openshift/hive

# login to the target cluster with oc
go get sigs.k8s.io/kustomize
go get sigs.k8s.io/controller-tools
go get github.com/openshift/hive
cd $GOPATH/src/github.com/openshift/hive
make deploy

jewzaam

#!/bin/bash

CLUSTER_NAME=$1
PASSWORD=$2

if [ "$CLUSTER_NAME" == "" ] || [ "$PASSWORD" == "" ];
then
    echo "Usage: $0 <Cluster Name> <\$USER password>"
    exit 1
fi

jewzaam

htpasswd -c -b htpasswd $USER $PASSWORD
oc delete secret htpass-secret -n openshift-config
oc create secret generic htpass-secret --from-file=htpasswd=htpasswd -n openshift-config

git clone [email protected]:openshift/online.git --depth=1 -b prod
oc delete secret -n openshift-config oauth-templates

oc create secret generic oauth-templates -n openshift-config \
    --from-file=login.html=online/ansible/roles/oso_custom_templates/files/dedicated/login.html \
    --from-file=providers.html=online/ansible/roles/oso_custom_templates/files/dedicated/provider-selection.html \

jewzaam

#!/bin/bash

CLUSTER_NAME=$1
PASSWORD=$2

IDENTITY_ID=aos-sre
IDENTITY_NAME="Red Hat SRE Test Auth"

if [ "$CLUSTER_NAME" == "" ];
then

jewzaam

apiVersion: hive.openshift.io/v1alpha1
kind: SelectorSyncSet
metadata:
  generation: 1
  name: osd-oauth-templates
spec:
  clusterDeploymentSelector:
    matchLabels:
      api.openshift.com/managed: "true"
  resourceApplyMode: sync

jewzaam

#!/bin/bash

# /api
GROUP=""
for VERSION in `oc get --raw /api | jq -r .versions[] | sort`;
do
    echo "- apiGroups:"
    echo "  - $GROUP"

    for RESOURCE in `oc get --raw /api/${VERSION} | jq -r .resources[].name | sort`;

jewzaam

PASSWORD=
CLUSTER_NAME=
CLUSTER_NAMESPACE=

touch htpasswd
htpasswd -b htpasswd customer-admin $PASSWORD
htpasswd -b htpasswd customer-user $PASSWORD

oc create secret generic htpasswd-secret --from-file=htpasswd=htpasswd -n openshift-config --dry-run -o yaml > htpasswd-secret.yaml