`CERTIFICATE_VERIFY_FAILED` in Python, on a Mac, on a computer that uses Cisco Umbrella

README.md

CERTIFICATE_VERIFY_FAILED in Python, on a Mac, on a computer that uses Cisco Umbrella

If you've ever tried to initiate an HTTPS request in Python on a Mac, perhaps usign the popular requests library, you've probably run into an exception that looks something like this:

requests.exceptions.SSLError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/03-11-2020.csv (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))

These issues are complicated by the fact that many users have multiple versions of Python installed and that there may be multiple pieces of software handling certificates.

This gist describes a particularly obscure version of this issue for people on computers that are also running Cisco Umbrella.

Is this for me?

This issue/fix is only relevent if:

• You're running OS X
• You have Python installed through Homebrew or pyenv
• Your computer is running Cisco Umbrella

If this isn't you, a fix like this one will likely work.

What is Cisco Umbrella?

I'm not really sure. I do know that it seems to mess with the SSL certificate for at least some domains. On my work comptuer, when I view a file on the raw.githubusercontent.com domain, I see GitHub's certificate. However, when I view the same file on Firefox, when I click to see the certificate information, I see that the certificate is verified by Cisco and a message that says:

Mozilla does not recognize this certificate issuer. It may have been added from an operating system or by an administrator.

Checking whether this is the problem

Try downloading the file with wget:

wget https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/03-11-2020.csv

You'll see a message like:

ERROR: cannot verify raw.githubusercontent.com's certificate, issued by 'CN=Cisco Umbrella Secondary SubCA chi-SG,O=Cisco':
  Unable to locally verify the issuer's authority.
To connect to raw.githubusercontent.com insecurely, use `--no-check-certificate'.

The fix

This is hacky and I don't have time to figure out a full workaround. This will get things going in your particular script, however.

Download the Cisco certificate

There's probably another way to find this certificate, maybe through the Keychain app, but this is what worked for me.

First, open a URL in Firefox that validates using the Umbrella cert.

Then click on the lock icon by the URL.

Click the ">" icon in the popup.

Then click "More Information".

A "Page Info" window will appear.

Click the "View Certificate" button.

The certificate will open in a browser tab.

Click on "Cisco Umbrella Root CA"

Scroll down and click the "PEM (cert)" link next to the "Download" label.

Save the file somewhere (I would put it in my project directory).

Set the REQUESTS_CA_BUNDLE environment variable in your script

You could set this environment variable in a number of other ways, but this way will ensure it's only used by your script.

Add these lines near the top of your script before making any HTTPS requests:

import os

# Update the path with the path of wherever you put the certificate PEM file.
os.environ['REQUESTS_CA_BUNDLE'] = '/path/to/raw-githubusercontent-com.pem'

A systemwide fix

I haven't tried this, but this is what I would try next.

brew info openssl gives me this information:

A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

I would put the Umbrella PEM I downloaded from Firefox in /usr/local/etc/openssl/certs and then run /usr/local/opt/openssl/bin/c_rehash.

References

• unable to get local issuer certificate for files.pythonhosted.org · Issue #7663 · pypa/pip: This might be the most comprehensive report of this issue. See this comment for a potential systemwide (for Python at least) fix.