Skip to content

Instantly share code, notes, and snippets.

@wallrj
Created September 6, 2024 16:27
Show Gist options
  • Save wallrj/6f6cb31281f21c8a63513bfba5aeb092 to your computer and use it in GitHub Desktop.
Save wallrj/6f6cb31281f21c8a63513bfba5aeb092 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash
#
# https://github.com/cert-manager/cert-manager/pull/7259
#
# Creates a Kind cluster and installs cert-manager from master and then from the
# bug fix branch containing the fix.
#
# The logs are printed in each case to demonstrate that there are rapid
# repeating errors in the master branch, and only 2-3 errors in the bug fix
# branch.
#
# The STS error is simulated by configuring a Route53 ClusterIssuer to use the
# Kubernetes JWT tokens for the cert-manager ServiceAccount, which will not be
# recognized by AWS, because no matching Identity Provider has been configured
# in AWS.
#
# Before:
# E0906 16:12:10.319958 1 controller.go:158] "re-queuing item due to error
# processing" err="error instantiating route53 challenge solver: unable to
# assume role with web identity: operation error STS:
# AssumeRoleWithWebIdentity, https response error StatusCode: 400, RequestID:
# bfc826c9-7b53-4738-90a3-b937a8b39723, InvalidIdentityToken: No OpenIDConnect
# provider found in your account for
# https://kubernetes.default.svc.cluster.local"
# logger="cert-manager.controller"
#
# After:
# E0906 16:14:19.434185 1 controller.go:158] "re-queuing item due to error
# processing" err="error instantiating route53 challenge solver: unable to
# assume role with web identity: operation error STS:
# AssumeRoleWithWebIdentity, https response error StatusCode: 400, RequestID:
# <REDACTED>, InvalidIdentityToken: No OpenIDConnect provider found in your
# account for https://kubernetes.default.svc.cluster.local"
# logger="cert-manager.controller"
set -o errexit
set -o nounset
set -o pipefail
set -o xtrace
# Create cluster
kind create cluster || true
# Deploy cert-manager (master)
helm uninstall -n cert-manager cert-manager || true
git fetch origin master
git checkout FETCH_HEAD
export KO_REGISTRY=ttl.sh/$(uuidgen)
make -j4 ko-deploy-certmanager
# Deploy deliberately misconfigured Route53 issuer
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: [email protected]
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- dns01:
route53:
region: us-west-2
role: arn:aws:iam::000000000000000:role/cert-manager-acme-dns01-route53
auth:
kubernetes:
serviceAccountRef:
name: cert-manager
---
# certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: www
spec:
secretName: www-tls
revisionHistoryLimit: 1
privateKey:
rotationPolicy: Always
commonName: 75b2296e-65a9-4c25-954b-d0b283c6bfd2.com
dnsNames:
- 75b2296e-65a9-4c25-954b-d0b283c6bfd2.com
usages:
- digital signature
- key encipherment
- server auth
issuerRef:
name: letsencrypt-staging
kind: ClusterIssuer
EOF
# Wait for 10 seconds and observe the many errors
sleep 10
kubectl logs -n cert-manager deploy/cert-manager | grep "error instantiating route53 challenge solver" | tee /dev/stderr | uniq | wc -l
# Deploy cert-manager (wallrj:5486-redact-amz-request-id)
helm uninstall -n cert-manager cert-manager || true
git fetch origin pull/7259/head
git checkout FETCH_HEAD
make -j4 ko-deploy-certmanager
# Wait for 10 seconds and observe only one error
sleep 10
kubectl logs -n cert-manager deploy/cert-manager | grep "error instantiating route53 challenge solver" | tee /dev/stderr | uniq | wc -l
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment