Sane automatic python + virtualenv

auto-python-venv.md

Automatic python version plus venv

Goal

For easy editor integration and command-line usage, we'd like to be able to specify a Python version per project, with its own virtualenv to isolate its libraries from those of other projects.

We're willing to change $PATH globally once, but not per project. And we'd like to avoid having to run every python command invocation in a special subshell created by a shell wrapper. Instead, simply invoking "python" or "pip" etc. should do the right thing, based on the directory in which it is invoked.

It turns out this is possible!

Method

Install pyenv and pyenv-virtualenv.

Add the following to your shell startup file:

if command -v pyenv >/dev/null 2>&1; then
    eval "$(pyenv init -)"
    if command -v pyenv-virtualenv >/dev/null 2>&1; then
        eval "$(pyenv virtualenv-init -)"
    fi
fi

Among other things, this will add the ~/.pyenv/shims to your $PATH. Make sure that the $PATH your editor sees also contains that directory at the beginning, so that the python it sees is the shim wrapper script in that dir.

In your project, create a virtualenv with your chosen python version:

pyenv install 3.6.8  # if you haven't used pyenv with this version before
pyenv virtualenv 3.6.8 my-virtual-env-3.6.8

Then tell pyenv to use that "virtualised" python in this project:

pyenv local my-virtual-env-3.6.8

This writes a .python-version file which contains "my-virtual-env-3.6.8".

Result

Now, whenever you run python or pip etc. inside a project directory, that command will be run with the virtualenv and Python version corresponding to any .python-version file in the project's root directory. This means that your editor needs no special per-project configuration in order to run the correct python/pip scripts, and you don't need to "activate" virtual envs when you switch from working on one project to another.

Note that you should still read a little about pyenv, including the usage of commands like pyenv rehash, but in the simple case, the above is all you need.

I've actually switched to using nix + direnv instead of virtualenv/pyenv, and it is an approach which extends to other languages too. I recorded a screencast about this: https://www.youtube.com/watch?v=TbIHRHy7_JM&feature=youtu.be