Overcoming ssl module in Python is not available error when using MacOS Homebrew's pyenv and openssl
I use MacOS Homebrew and it's versions of pyenv and openssl. When I wanted to install a new Python package, I ran into an unexpected error:
❯ pip install llama-cpp-python
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/llama-cpp-python/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/llama-cpp-python/
^CERROR: Operation cancelled by user
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.Searching around a bit, no one had a concise, reliable solve that worked for me ... until I got to @burhan's one-liner for testing your python's openssl install, which kicked out a wealth of info, including the version of openssl it was looking for:
> python -c "import ssl; print(ssl.OPENSSL_VERSION)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/purp/.pyenv/versions/3.7.16/lib/python3.7/ssl.py", line 98, in <module>
import _ssl # if we can't import it, let the error propagate
ImportError: dlopen(/Users/purp/.pyenv/versions/3.7.16/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so, 0x0002): Library not loaded: /opt/homebrew/opt/[email protected]/lib/libssl.1.1.dylib
Referenced from: <E7910C8B-B99E-3C4A-AF10-A1B1AA530B91> /Users/purp/.pyenv/versions/3.7.16/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so
Reason: tried: '/opt/homebrew/opt/[email protected]/lib/libssl.1.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/[email protected]/lib/libssl.1.1.dylib' (no such file), '/opt/homebrew/opt/[email protected]/lib/libssl.1.1.dylib' (no such file), '/usr/local/lib/libssl.1.1.dylib' (no such file), '/usr/lib/libssl.1.1.dylib' (no such file, not in dyld cache)Somehow, the homebrew [email protected] package, which is what my python 3.7 build was built with, had been uninstalled. I'm guessing it was during a brew clean step where no homebrew package relied on it and, because homebrew doesn't know about external build dependencies, it just removed it.
Reinstalling fixed the issue, with justfiable deprecation warnings:
> brew install [email protected]
Warning: [email protected] has been deprecated because it is not supported upstream!
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/1.1/manifests/1.1.1w
######################################################################### 100.0%
==> Fetching [email protected]
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/1.1/blobs/sha256:38619f
######################################################################### 100.0%
==> Pouring [email protected]_sonoma.bottle.tar.gz
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
/opt/homebrew/etc/[email protected]/certs
and run
/opt/homebrew/opt/[email protected]/bin/c_rehash
[email protected] is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.
If you need to have [email protected] first in your PATH, run:
echo 'export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH"' >> ~/.zshrc
For compilers to find [email protected] you may need to set:
export LDFLAGS="-L/opt/homebrew/opt/[email protected]/lib"
export CPPFLAGS="-I/opt/homebrew/opt/[email protected]/include"
For pkg-config to find [email protected] you may need to set:
export PKG_CONFIG_PATH="/opt/homebrew/opt/[email protected]/lib/pkgconfig"
==> Summary
🍺 /opt/homebrew/Cellar/[email protected]/1.1.1w: 8,102 files, 18MB
==> Running `brew cleanup [email protected]`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).I'll be updating my pyenv built pythons (and likely some pip'd in packages) to openssl3 soon.