Skip to content

Instantly share code, notes, and snippets.

@yatsu

yatsu/pyenv-python38-macos.sh

Created January 2, 2021 11:35
Show Gist options
  • Save yatsu/9ae2dc633511d6d97efccd4fcfa8c8b1 to your computer and use it in GitHub Desktop.
Save yatsu/9ae2dc633511d6d97efccd4fcfa8c8b1 to your computer and use it in GitHub Desktop.
Download ZIP
Install Python 3.8.6 with pyenv on macOS Big Sur (11)
Raw
pyenv-python38-macos.sh
#!/bin/sh
# 1) Install Xcode 12
# 2) Install command line tools: `xcode-select --install`
# 3) Install Homebrew
# 4) brew install zlib bzip2 xz
CFLAGS="-I$(brew --prefix zlib)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix xz)/include" \
LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib -L$(brew --prefix xz)/lib" \
pyenv install 3.8.6
@kailichou
Copy link

Thanks for your quick reply.

I still run into a WARNING after installing 3.8.6

bash-3.2$ pyenv install 3.8.6
python-build: use [email protected] from homebrew
python-build: use readline from homebrew
Downloading Python-3.8.6.tar.xz...
-> https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz
Installing Python-3.8.6...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
**WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?**
Installed Python-3.8.6 to /Users/kailichou/.pyenv/versions/3.8.6

What I did,

  1. xcode-select installed and up-to-date
  2. brew and dependencies (bzip2, zlib, readline, xz) have installed
  3. uninstall pyenv and reinstall pyenv
  4. install 3.8.6 and got a WARNING

Many recommendations are to install bzip2-devel, libbz2-dev... for Linux system, which brew doesn't have those formulae for MacOS.

What happened during installing pyenv
git issue, does this cause my bzip2 compiler problem?
Many thanks.

bash-3.2$ brew install pyenv
Updating Homebrew...
fatal: Unable to create '/usr/local/Homebrew/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
fatal: Unable to create '/usr/local/Homebrew/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
==> Downloading https://mirrors.ustc.edu.cn/homebrew-bottles/bottles/pyenv-1.2.2
Already downloaded: /Users/kailichou/Library/Caches/Homebrew/downloads/d824644ecebd72fcb903b1e8a2fecf162c94a7ce960f0977a782176dd387b559--pyenv-1.2.21.big_sur.bottle.tar.gz
==> Pouring pyenv-1.2.21.big_sur.bottle.tar.gz
🍺  /usr/local/Cellar/pyenv/1.2.21: 716 files, 2.6MB

@yatsu
Copy link
Author

yatsu commented Jan 4, 2021

Check bzip2 is correctly installed.

$ ls $(brew --prefix bzip2)/lib
libbz2.a

libbz2.a is the library Python should link to. -devel package is not required in macOS/Homebrew.

Next, make sure the arguments -I$(brew --prefix bzip2)/include and -L$(brew --prefix bzip2)/lib (don't mix up -I and -L, include and lib).

pyenv install -v shows the compile log. You can check if bz2 module was successfully installed.

$ LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install -v 3.8.5 2>&1 | grep bz2
building '_bz2' extension
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/opt/brew/opt/readline/include -I/opt/brew/opt/zlib/include -I/usr/local/include -I/private/var/folders/nb/dw5p3r0j17q4y5r5wkhpnr0r0000gn/T/python-build.20210104122251.50610/Python-3.8.5/Include -I/private/var/folders/nb/dw5p3r0j17q4y5r5wkhpnr0r0000gn/T/python-build.20210104122251.50610/Python-3.8.5 -c /private/var/folders/nb/dw5p3r0j17q4y5r5wkhpnr0r0000gn/T/python-build.20210104122251.50610/Python-3.8.5/Modules/_bz2module.c -o build/temp.macosx-11.1-x86_64-3.8/private/var/folders/nb/dw5p3r0j17q4y5r5wkhpnr0r0000gn/T/python-build.20210104122251.50610/Python-3.8.5/Modules/_bz2module.o
clang -bundle -undefined dynamic_lookup -L/opt/brew/opt/readline/lib -L/opt/brew/opt/readline/lib -L/Users/yatsu/.pyenv/versions/3.8.5/lib -L/opt/brew/opt/zlib/lib -L/opt/brew/opt/bzip2/lib -L/opt/brew/opt/readline/lib -L/opt/brew/opt/readline/lib -L/Users/yatsu/.pyenv/versions/3.8.5/lib -L/opt/brew/opt/zlib/lib -L/opt/brew/opt/bzip2/lib -L/opt/brew/opt/readline/lib -L/opt/brew/opt/readline/lib -L/Users/yatsu/.pyenv/versions/3.8.5/lib -L/opt/brew/opt/zlib/lib -L/opt/brew/opt/bzip2/lib -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/opt/brew/opt/readline/include -I/opt/brew/opt/readline/include -I/Users/yatsu/.pyenv/versions/3.8.5/include -I/opt/brew/opt/zlib/include build/temp.macosx-11.1-x86_64-3.8/private/var/folders/nb/dw5p3r0j17q4y5r5wkhpnr0r0000gn/T/python-build.20210104122251.50610/Python-3.8.5/Modules/_bz2module.o -L/opt/brew/opt/readline/lib -L/Users/yatsu/.pyenv/versions/3.8.5/lib -L/opt/brew/opt/zlib/lib -L/opt/brew/opt/bzip2/lib -L/usr/local/lib -lbz2 -o build/lib.macosx-11.1-x86_64-3.8/_bz2.cpython-38-darwin.so -Wl,-search_paths_first
/opt/brew/bin/ginstall -c -m 644 ./Lib/bz2.py /Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8
/opt/brew/bin/ginstall -c -m 644 ./Lib/test/test_bz2.py /Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/test
/opt/brew/bin/ginstall -c -m 644 ./Lib/encodings/bz2_codec.py /Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/encodings
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/test/test_bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/encodings/bz2_codec.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/test/test_bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/encodings/bz2_codec.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/test/test_bz2.py'...
Compiling '/Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/encodings/bz2_codec.py'...
copying build/lib.macosx-11.1-x86_64-3.8/_bz2.cpython-38-darwin.so -> /Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/lib-dynload
changing mode of /Users/yatsu/.pyenv/versions/3.8.5/lib/python3.8/lib-dynload/_bz2.cpython-38-darwin.so to 755

@kailichou
Copy link

kailichou commented Jan 6, 2021 •
edited
Loading

I took another way.

OS: Big Sur 11.1

What I tried

  • Remove brew and its packages
  • and reinstall brew, pyenv, its dependencies python3.8.5
  • without brew install bzip2 or brew install openblas

Result

  • no more Polyfit issue with numpy
  • or no more _bz2 module issue with pandas

I have no idea what's changed, but it worked.

Thanks for your help anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment