Multiple python versions

Reasoning and use cases

There are often scenarios when we would need to install multiple versions of python.

On on our development machine, we might want to work on many projects, some using different python versions, or maybe we're working on a package that we want to support on multiple python distributions, and want to test them all with tox.

On a production system we may require a python version other than the version pre-installed on the system, or available as packaged distributions.

Step By Step Guide

They say a step-by-step example is worth a thousands words (really!). The following section aims to show a simple way to install Python 3.5.1 on CentOS 7 while at the same time cleaning up dependencies as soon as these are not required anymore.

Dependencies

Before we get to installation, lets install our minimum dependencies:

$ yum install -y git gcc openssl-devel

We use git to fetch pyenv, gcc to compile python and openssl-devel for the SSL C extension (required for example to send an HTTP request to an https:// url).

Installation

An easy way to download (and compile) multiple versions of python and then switching between them is using pyenv. The preferred method of installation is pyenv-installer (which is why we require git).

$ yum install -y git
$ curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash

$ cat >> ~/.bash_profile
export PATH="/home/user/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
^d

$ . ~/.bash_profile

We're now ready to install a new python version, and switch versions at ease:

$ pyenv install --list | grep 3.5
3.3.5
3.5.0
3.5-dev
3.5.1
...
$ pyenv install 3.5.1
Downloading Python-3.5.1.tar.xz...
-> https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tar.xz
Installing Python-3.5.1...
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib?
Installed Python-3.5.1 to /home/user/.pyenv/versions/3.5.1

Since we installed openssl-devel, the python distribution includes SSL support. However, it excludes other C extensions as seen in the warnings above, namely the bz2, readline and sqlite3 C extenisons.

If you find that you need the extensions after the fact, you can reinstall python at any time.

# get readline develop libraries
$ yum install readline-devel

# force install our target python version
$ pyenv install -f 3.5.1

Warning

The -f operator will force install a version, which may be a destructive operation.

Swiching python versions

$ pyenv versions
* system (set by /home/user/.pyenv/version)
  3.5.1
$ pyenv shell 3.5.1
$ python -V
Python 3.5.1

Cleanup

After installing all required python versions, we can clean up our dependencies.

On our CentOS machine the cleanup is done by yum automatically after we adding the following option to /etc/yum.conf: clean_requirements_on_remove=1.

$ cat >> /etc/yum.conf
clean_requirements_on_remove=1
^D

$ # remove manually installed dependencies (and any unused libs they depend on)
$ yum remove -y git gcc openssl-devel

Dependency cleanup on other systems

Installing python on Linux/OSX often requires compiling from source, which also means that dependencies need to be installed. On a production machine we would prefer to remove these dependencies after installation.

The cleanup of installation dependencies can usually be handled automatically by the system package manager, such as yum on CentOS or apt-get on Ubuntu/Debian. In our CentOS example above we used yum to auto remove all requirements when not used by any other packages. With apt-get one could use the --auto-remove command line argument like so:

$ apt-get remove --auto-remove packagename

asfaltboy/py-env-guide.rst