Skip to content

Instantly share code, notes, and snippets.

@dmtucker

dmtucker/pytoolchain.md

Created February 13, 2019 08:11
Show Gist options
  • Save dmtucker/3c9d5e2c389409aa807f95621bfdd242 to your computer and use it in GitHub Desktop.
Save dmtucker/3c9d5e2c389409aa807f95621bfdd242 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
pytoolchain.md
  • setuptools builds distributions from source code.
  • pip (un)installs distributions (optionally, from PyPI).
    • twine publishes distributions to PyPI.
  • pytest tests installed distributions.
@tucked
Copy link

tucked commented Mar 15, 2019

anyone confused over all the py-things in the typical toolchain? we've been discussing this in a PM, but it's probably generally applicable, so i'll share it here. it is my opinion on what goes where:

requirements.txt: replaced by Tox deps or extras_require (in setup.py) for libs, replaced by Pipfile (eventually) for apps
Pipfile: used for apps, mostly with pipenv (since pip doesn't support it yet)
pipenv: used for apps. kennethreitz says otherwise... he's probably wrong
pip/twine: used for managing dists, might get merged eventually
pyproject.toml: replaces setup_requires, allows for build systems that aren't setuptools (e.g. flit, poetry), will probably replace setup.cfg
virtualenv: probably replaced by python3 -m venv which is stdlib
pyenv: manage multiple pythons on a system (so you're not locked into whatever the OS supports, which tends to be ancient)
pipx: manage global packages (without sudo) in a way that uncomplicates pip install --user. this is perfect for python packages that aren't necessarily packaged for OSes (e.g. ducttape cli)

simplifying... this is the toolchain i use / my view of the world:

  • pyenv as necessary
    as a user / in production:
  • pipx and pipenv (they use pip under-the hood, but i generally don't need to know that)
    as a dev:
  • pipenv for apps (note it's presence in dev and prod -- like docker, another deployment tool)
  • tox + pytest for testing (tox manages virtualenvs, including a dev env: https://tox.readthedocs.io/en/latest/example/devenv.html)
  • pyproject.toml (always, starting now) + setuptools (for now... i want to look into flit) for build

note that pip and python3 -m venv/virtualenv is pretty much never used directly.

@tucked
Copy link

tucked commented Mar 2, 2020

https://xkcd.com/1987/

@tucked
Copy link

tucked commented Mar 2, 2020

no global pip, instead (these both dont work)

mkdir -p ~/.local/pipx/venvs
sudo apt install python3-venv
python3 -m venv ~/.local/pipx/venvs/pipx
~/.local/pipx/venvs/pipx/bin/pip install pipx
ln -snf ~/.local/pipx/venvs/pipx/bin/pipx ~/.local/bin/pipx
~/.local/bin/pipx ensurepath
pipx upgrade pipx

use global python for pipx not pyenv versions (in case you decide to remove them)

  • pipx first, then pyenv

pipx install pipenv tox

@tucked
Copy link

tucked commented Mar 2, 2020

related? https://dev.to/codemouse92/dead-simple-python-virtual-environments-and-pip-5b56

@tucked
Copy link

tucked commented Mar 2, 2020

pypa/packaging.python.org#396

@tucked
Copy link

tucked commented Mar 2, 2020

CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment