Fixing untrusted self-signed Chef Server 12 certificates

gistfile1.md

Scenario: You've started up a brand new Chef Server using version 12, and you have installed Chef 12 on your local system. You log into the Management Console to create a user and organization (or do this with the command-line chef-server-ctl commands), and you're ready to rock with this knife.rb:

node_name                'jtimberman'
client_key               'jtimberman.pem'
validation_client_name   'tester-validator'
validation_key           'tester-validator.pem'
chef_server_url          'https://ip-172-31-0-111.us-west-2.compute.internal/organizations/tester'

However, when you try to check things out with knife:

% knife client list
ERROR: SSL Validation failure connecting to host: ip-172-31-8-24.us-west-2.compute.internal - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
ERROR: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

This is because Chef client 12 has SSL verification enabled by default for all requests. Since the certificate generated by the Chef Server 12 installation is self-signed, there isn't a signing CA that can be verified, and this fails. Never fear intrepid user, for you can get the SSL certificate from the server and store it as a "trusted" certificate. To find out how, use knife ssl check.

Connecting to host ip-172-31-0-111.us-west-2.compute.internal:443
ERROR: The SSL certificate of ip-172-31-0-111.us-west-2.compute.internal could not be verified
Certificate issuer data: /C=US/ST=WA/L=Seattle/O=YouCorp/OU=Operations/CN=ip-172-31-0-111.us-west-2.compute.internal/emailAddress=you@example.com

Configuration Info:

OpenSSL Configuration:
* Version: OpenSSL 1.0.1j 15 Oct 2014
* Certificate file: /opt/chefdk/embedded/ssl/cert.pem
* Certificate directory: /opt/chefdk/embedded/ssl/certs
Chef SSL Configuration:
* ssl_ca_path: nil
* ssl_ca_file: nil
* trusted_certs_dir: "/Users/jtimberman/Downloads/chef-repo/.chef/trusted_certs"

TO FIX THIS ERROR:

If the server you are connecting to uses a self-signed certificate, you must
configure chef to trust that server's certificate.

By default, the certificate is stored in the following location on the host
where your chef-server runs:

  /var/opt/chef-server/nginx/ca/SERVER_HOSTNAME.crt

Copy that file to your trusted_certs_dir (currently: /Users/jtimberman/Downloads/chef-repo/.chef/trusted_certs)
using SSH/SCP or some other secure method, then re-run this command to confirm
that the server's certificate is now trusted.

(note, this chef-server location is incorrect, it's /var/opt/opscode)

There is a fetch plugin for knife too. Let's download the certificate to the automatically preconfigured trusted certificate location mentioned in the output above.

% knife ssl fetch
WARNING: Certificates from ip-172-31-8-24.us-west-2.compute.internal will be fetched and placed in your trusted_cert
directory (/Users/jtimberman/Downloads/chef-repo/.chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.

Adding certificate for ip-172-31-8-24.us-west-2.compute.internal in /Users/jtimberman/Downloads/chef-repo/.chef/trusted_certs/ip-172-31-8-24_us-west-2_compute_internal.crt

The certificate should be verified that what was downloaded is in fact the same as the certificate on the Chef Server. For example, I compared SHA256 checksums:

% ssh ubuntu@ip-172-31-0-111.us-west-2.compute.internal sudo sha256sum /var/opt/opscode/nginx/ca/ip-172-31-0-111.us-west-2.compute.internal.crt
043728b55144861ed43a426c67addca357a5889158886aee50685cf1422b5ebf  /var/opt/opscode/nginx/ca/ip-172-31-0-111.us-west-2.compute.internal.crt
% gsha256sum .chef/trusted_certs/ip-172-31-0-111_us-west-2_compute_internal.crt
043728b55144861ed43a426c67addca357a5889158886aee50685cf1422b5ebf  .chef/trusted_certs/ip-172-31-0-111_us-west-2_compute_internal.crt

Now check knife client list again.

% knife client list
tester-validator

Addendum: to do this on a client node, you can use knife with the client config.

% knife ssl fetch -c /etc/chef/client.rb
WARNING: Certificates from api.opscode.com will be fetched and placed in your trusted_cert
directory (/etc/chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.

Adding certificate for *.opscode.com in /etc/chef/trusted_certs/wildcard_opscode_com.crt
Adding certificate for DigiCert Secure Server CA in /etc/chef/trusted_certs/DigiCert_Secure_Server_CA.crt
root@cask:~# ls /etc/chef/trusted_certs/
DigiCert_Secure_Server_CA.crt  wildcard_opscode_com.crt

Note that this particular node is using Hosted Chef, which has a valid certificate, but we see it gets everything in place.