This is a short guide explaining how to deploy and manage custom certificates via openshift-ansible. These custom certificates will be served for public facing UI/API.
1. Set up
NOTE: I'm running ansible from the first master so I'm creating the certificates in /root/ on my first master. The certificates will be copied to the masters (within /etc/origin/master/named_certificates/) by ansible. I could create them on my local system and run ansible from my local system instead.
1.1 Create CA Certificate
openssl genrsa -out rootCA.key 2048
openssl req -x509 -new -nodes -key rootCA.key -days 1024 -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=public-example.com/[email protected]/' -out rootCA.pem
1.2 Create master.public-example.com.crt
openssl genrsa -out master.public-example.com.key 2048
openssl req -new -key master.public-example.com.key -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=master.public-example.com/[email protected]/' -out master.public-example.com.csr
openssl x509 -req -in master.public-example.com.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out master.public-example.com.crt -days 500
1.3 Create wildcard-flibberty-jibbet.com.crt
openssl genrsa -out wildcard-flibberty-jibbet.com.key 2048
openssl req -new -key wildcard-flibberty-jibbet.com.key -subj '/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=OpenShift/CN=*.flibberty-jibbet.com/[email protected]/' -out wildcard-flibberty-jibbet.com.csr
openssl x509 -req -in wildcard-flibberty-jibbet.com.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out wildcard-flibberty-jibbet.com.crt -days 500
1.4 Modify /etc/hosts
192.168.122.224 master.public-example.com master.flibberty-jibbet.com
2. Install using master.public-example.com certificate
Create an HA OpenShift cluster and specify the openshift_master_named_certificates inventory variable. Modify paths to match the certificate paths on the system where you will be running ansible and where the certificates were created.
openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
2.1 Verify Configuration for master.public-example.com
2.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/
# ls /etc/origin/master/named_certificates/
master.public-example.com.crt master.public-example.com.key
2.1.2 Verify master configuration
servingInfo:
bindAddress: 0.0.0.0:8443
bindNetwork: tcp4
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 500
requestTimeoutSeconds: 3600
namedCertificates:
- certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
names:
- "master.public-example.com"
2.1.3 Ensure API serves the default certificate
# curl -vk https://192.168.122.224:8443
* About to connect() to 192.168.122.224 port 8443 (#0)
* Trying 192.168.122.224...
* Connected to 192.168.122.224 (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=172.30.0.1
* start date: Nov 17 00:53:36 2015 GMT
* expire date: Nov 16 00:53:37 2017 GMT
* common name: 172.30.0.1
* issuer: CN=openshift-signer@1447721613
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.122.224:8443
> Accept: */*
2.1.4 Ensure API serves certificate for master.public-example.com
# curl -vk https://master.public-example.com:8443
* About to connect() to master.public-example.com port 8443 (#0)
* Trying 192.168.122.224...
* Connected to master.public-example.com (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: [email protected],CN=master.public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
* start date: Nov 17 00:22:26 2015 GMT
* expire date: Mar 31 00:22:26 2017 GMT
* common name: master.public-example.com
* issuer: [email protected],CN=public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: master.public-example.com:8443
> Accept: */*
3. Re-run ansible with wildcard-flibberty-jibbet.com certificate
When we re-run ansible with different openshift_master_named_certificates the original certificates should remain in place and they should also remain configured.
Modify host inventory, commenting the original openshift_master_named_certificates variable and add a new variable for wildcard-flibberty-jibbet.com.{crt,key}.
#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]
3.1 Verify Configuration for wildcard-flibberty-jibbet.com
3.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/
Both sets of certificates will exist on the filesystem.
# ls /etc/origin/master/named_certificates/
master.public-example.com.crt master.public-example.com.key wildcard-flibberty-jibbet.com.crt wildcard-flibberty-jibbet.com.key
3.1.2 Verify master configuration
Both certificates will be configured.
servingInfo:
bindAddress: 0.0.0.0:8443
bindNetwork: tcp4
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 500
requestTimeoutSeconds: 3600
namedCertificates:
- certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
names:
- "master.public-example.com"
- certFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.crt
keyFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.key
names:
- "*.flibberty-jibbet.com"
3.1.3 Ensure API serves the default certificate
# curl -vk https://192.168.122.224:8443
* About to connect() to 192.168.122.224 port 8443 (#0)
* Trying 192.168.122.224...
* Connected to 192.168.122.224 (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=172.30.0.1
* start date: Nov 17 00:53:36 2015 GMT
* expire date: Nov 16 00:53:37 2017 GMT
* common name: 172.30.0.1
* issuer: CN=openshift-signer@1447721613
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.122.224:8443
> Accept: */*
3.1.4 Ensure API serves certificate for master.flibberty-jibbet.com
[root@master4 ~]# curl -vk https://master.flibberty-jibbet.com:8443
* About to connect() to master.flibberty-jibbet.com port 8443 (#0)
* Trying 192.168.122.224...
* Connected to master.flibberty-jibbet.com (192.168.122.224) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: [email protected],CN=*.flibberty-jibbet.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
* start date: Nov 17 00:22:37 2015 GMT
* expire date: Mar 31 00:22:37 2017 GMT
* common name: *.flibberty-jibbet.com
* issuer: [email protected],CN=public-example.com,OU=OpenShift,O="Red Hat, Inc.",L=Raleigh,ST=North Carolina,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: master.flibberty-jibbet.com:8443
> Accept: */*
4. Re-run ansible with no certificates
If we have previously ran ansible with openshift_master_named_certificates and then re-run with no variable set, the original configuration and certificates should remain in place.
Comment all previous openshift_master_named_certificates inventory variables.
#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]
4.1 Verify Configuration with no certificates
4.1.1 Ensure certificates exist in /etc/origin/master/named_certificates/
Both sets of certificates will exist on the filesystem.
# ls /etc/origin/master/named_certificates/
master.public-example.com.crt master.public-example.com.key wildcard-flibberty-jibbet.com.crt wildcard-flibberty-jibbet.com.key
4.1.2 Verify master configuration
Both certificates will be configured.
servingInfo:
bindAddress: 0.0.0.0:8443
bindNetwork: tcp4
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 500
requestTimeoutSeconds: 3600
namedCertificates:
- certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
names:
- "master.public-example.com"
- certFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.crt
keyFile: /etc/origin/master/named_certificates/wildcard-flibberty-jibbet.com.key
names:
- "*.flibberty-jibbet.com"
5. Re-run ansible and overwrite configured certificates
Modify host inventory, uncommenting the openshift_master_named_certificate inventory variable for master.public-example.com, commenting the openshift_master_named_certificates inventory variable for wildcard-flibberty-jibbet.com and add openshift_master_overwrite_named_certificates=true.
openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]
openshift_master_overwrite_named_certificates=true
5.1 Verify Configuration with overwritten certificates
5.1.1 Ensure correct certificates exist in /etc/origin/master/named_certificates/
wildcard-flibbery-jibbet.{crt,key} should be absent.
# ls /etc/origin/master/named_certificates/
master.public-example.com.crt master.public-example.com.key
5.1.2 Verify master configuration
Similarly, wildcard-flibbery-jibbet.{crt,key} should be absent from master configuration.
servingInfo:
bindAddress: 0.0.0.0:8443
bindNetwork: tcp4
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 500
requestTimeoutSeconds: 3600
namedCertificates:
- certFile: /etc/origin/master/named_certificates/master.public-example.com.crt
keyFile: /etc/origin/master/named_certificates/master.public-example.com.key
names:
- "master.public-example.com"
6. Re-run ansible and clear certificates
Modify host inventory, setting openshift_master_overwrite_named_certificates=true. If this variable is set and no openshift_master_named_certificates are configured, all certificates and configuration will be cleared.
#openshift_master_named_certificates=[{"certfile": "/root/master.public-example.com.crt", "keyfile": "/root/master.public-example.com.key"}]
#openshift_master_named_certificates=[{"certfile": "/root/wildcard-flibberty-jibbet.com.crt", "keyfile": "/root/wildcard-flibberty-jibbet.com.key"}]
openshift_master_overwrite_named_certificates=true
6.1 Verify Configuration with cleared certificates
6.1.1 Ensure no certificates exist in /etc/origin/master/named_certificates/
The named_certificates directory should be absent.
# ls /etc/origin/master/named_certificates/
ls: cannot access /etc/origin/master/named_certificates/: No such file or directory
6.1.2 Verify master configuration
No certificates will be configured.
servingInfo:
bindAddress: 0.0.0.0:8443
bindNetwork: tcp4
certFile: master.server.crt
clientCA: ca.crt
keyFile: master.server.key
maxRequestsInFlight: 500
requestTimeoutSeconds: 3600