Author: Claude (Anthropic)
Date: 2025-10-27
Three parallel histories of Python package management from 1998 to 2025, presented from different perspectives: a generous account focusing on people and progress, a subtly negative account emphasizing accumulated difficulties, and a neutral technical timeline.
distutils (1998-2000): In September 1998, Greg Ward noticed that every Python library with C extensions had its own fragile Makefile cribbed from Python's standard library installation. At the 1998 International Python Conference, Ward organized a session called "Building Extensions Considered Painful" where he, Greg Stein, Barry Warsaw, and others hammered out what would become the Distribution Utilities. They chose the python setup.py interface specifically to avoid creating a new DSL - why not use Python's full power? The name "Distribution Utilities" was deliberate; Perl already had ExtUtils, and the Python community wanted to be clear this was for all Python code, not just extensions. Ward later admitted distutils predated unit testing culture in Python, which explains some of its limitations. By 2000, distutils shipped with Python 1.6, giving the ecosystem its first standard way to install code.
setuptools (2004): Phillip Eby saw what distutils couldn't do and built setuptools to add dependency management, automatic installation, and the Egg format. This was ambitious infrastructure work that made PyPI actually useful. Eby was solving real problems: before setuptools, there was no way to declare "my package needs these other packages," which severely limited what Python projects could achieve. The feature set was extensive because the problems were extensive - namespace packages for large projects, optional dependencies for flexibility, version negotiation for compatibility. For several years, setuptools was simply the way serious Python projects managed dependencies.
pip (2008): Ian Bicking created pip (originally "pyinstall") after years of frustration with easy_install. Bicking had already created virtualenv in 2007, demonstrating his sustained focus on improving the Python development experience. The name "pip" came from community suggestions on his blog - "pip installs packages" was both recursive and clear. Pip could actually uninstall packages, had better error messages, and felt more like a proper package manager. By 2011, the Python Packaging Authority formed around Bicking's work, with Carl Meyer, Brian Rosner, and Jannis Leidel taking the lead. Former pip maintainer Jannis Leidel would later praise pipenv as fitting his brain better than manual pip calls - high praise from someone who knew pip's internals intimately.
virtualenv (2007): Also Bicking's creation, virtualenv solved the critical problem of conflicting dependencies between projects. Before virtualenv, installing packages globally meant project A and project B would fight over package versions. Virtualenv's insight was elegantly simple: give each project its own Python environment. This became foundational - almost every subsequent tool built on or incorporated virtualenv's approach. The Python community enthusiastically adopted it because it solved a problem everyone had experienced.
requirements.txt (emerged ~2010): As pip usage grew, the community converged on requirements.txt as a simple way to specify dependencies. You could now clone a repository and run pip install -r requirements.txt to get the same packages. This wasn't specified in any PEP - it emerged organically from people sharing their workflows and blog posts about best practices. The simplicity was the point: just a list of packages in a text file anyone could read and edit.
wheel (2012-2013): Daniel Holth's PEP 427 introduced the wheel format to replace eggs with something simpler and more standard. Holth explicitly designed wheels to be unpackable with basic zip tools if needed, though specialized installers would work better. The format separated build-time from install-time, amortizing compilation costs across many installations. Holth drew inspiration from other package formats while learning from egg's complications. By 2013, wheels began appearing on PyPI, and by 2015, pip would preferentially install wheels when available, making Python package installation noticeably faster for everyone.
pipenv (2017): Kenneth Reitz, already famous for creating the elegant requests library, turned his attention to the packaging workflow in January 2017. Reitz believed in the "for humans" philosophy - tools should fit how humans think, not the other way around. Pipenv aimed to unify pip, virtualenv, and Pipfile into a single coherent experience. When Python.org officially recommended pipenv in November 2017, it felt like validation that the Python community was ready to move beyond the fragmented pip/virtualenv/requirements.txt workflow. Reitz gave talks at PyCon 2018 about pipenv's vision for the future.
poetry (2018): Sébastien Eustace built poetry after frustration with existing tools that required multiple separate components. He wanted "something reliable and intuitive that the community could use and enjoy." Poetry's use of the PubGrub algorithm for dependency resolution was technically sophisticated - the same algorithm Dart's pub package manager uses. Poetry quickly gained adoption precisely because Eustace had thought carefully about the full workflow: dependency declaration, lock files, building, and publishing all in one tool. Major frameworks like FastAPI and tools like Kedro adopted poetry for their dependency management, showing enterprise-grade confidence in Eustace's design.
pip's dependency resolver (2020): After years of work, pip 20.3 finally shipped a proper backtracking dependency resolver. The old resolver's "first wins" strategy had been a known limitation for years - pip would install incompatible packages and shrug. The new resolver fixed this fundamental flaw. The pip team knew this would break some workflows and carefully staged the rollout with warnings starting in pip 20.2. The resolver was more correct even if it took longer - and for production systems, correctness matters more than speed.
pyproject.toml (2016-2020): Brett Cannon, Donald Stufft, and others worked through PEP 518 (2016) and PEP 621 (2020) to standardize project metadata in pyproject.toml. This finally solved the chicken-and-egg problem: how do you specify build dependencies when you need build tools to read that specification? The choice of TOML over JSON or YAML was carefully reasoned - TOML is human-editable, unambiguous, and already proven by Rust's Cargo. Multiple build backends (setuptools, flit, hatchling) now support pyproject.toml, giving projects real choice in tooling.
uv (2024): Charlie Marsh and the Astral team, having already revolutionized Python linting with Ruff, released uv in February 2024. Written in Rust for performance, uv delivers 10-100x speedups while maintaining compatibility with existing workflows. The team's philosophy has been "meet users where they are" - you can use uv as a drop-in pip replacement without changing anything else. The Astral team actively engages with the community, responding to GitHub issues rapidly and shipping updates frequently. In August 2024, they extended uv to handle project management, tool installation, and Python version management - a genuinely unified experience. The venture backing gives the team resources to work full-time on developer tooling, which shows in the pace of improvement.
distutils (1998-2000): Mr. Ward's Distribution Utilities emerged from a 1998 conference session, which is to say it bore all the characteristics of committee-designed software produced in an afternoon. The choice to use Python itself for the setup.py interface seemed clever at the time, though it rather committed the ecosystem to executing arbitrary code for the seemingly simple task of installing a package. This would prove consequential. By 2000, when distutils entered the standard library, the Python community found itself in possession of a tool that could install packages but not manage dependencies - a feature set that might charitably be described as "foundational" or, less charitably, as "incomplete."
setuptools (2004): Mr. Eby's setuptools attempted to remedy distutils' limitations through the time-honored engineering practice of adding features until the problem goes away. The Egg format appeared, introducing to Python the concept of zipped packages that one could not easily inspect. The automatic dependency resolution installed whatever versions happened to be newest at installation time, which is to say it resolved dependencies in roughly the same way one might resolve dinner plans by eating whatever is in the refrigerator without checking expiration dates. The inability to uninstall packages was particularly memorable; one simply deleted files and hoped. For several years this represented the state of the art.
pip (2008): Mr. Bicking's pip improved upon easy_install by introducing the ability to uninstall packages, a feature that in any reasonable ecosystem would have been present from the start. The dependency resolution algorithm operated by installing packages sequentially and hoping they would prove compatible - an approach that might be termed "optimistic." When package A required version 1.0 of package C and package B required version 2.0, pip would install whichever it encountered last, producing environments that were inconsistent in ways that would only reveal themselves during execution, preferably in production. This behavior persisted for twelve years.
virtualenv (2007): Also Mr. Bicking's work, virtualenv solved the problem of global package pollution by requiring developers to manually activate and deactivate environments before installing anything. The activation scripts were shell-specific, being different for bash, zsh, and Windows, which meant cross-platform projects accumulated multiple activation scripts. One might forget which environment was currently active, leading to packages installed in unintended locations. This proved so common that the ecosystem developed tools specifically to help remember which virtualenv was active, which is to say, tools to work around the usability characteristics of virtualenv itself.
requirements.txt (~2010): The community converged on requirements.txt as a solution to dependency specification, which had the merit of being a text file that humans could read. It had the drawback of being merely a list of packages without any mechanism to ensure the dependencies of those packages were consistent. When one specified requests==2.0.0, one received that exact version, but its dependencies might vary, leading to the phenomenon whereby the same requirements.txt file produced different environments over time. The phrase "works on my machine" became rather less humorous when one considered how often it was accurate.
wheel (2012-2013): Mr. Holth's wheel format replaced eggs with something simpler, which is to say it replaced eggs with something that still required platform-specific builds but at least used a standardized archive format. The wheel format addressed the problem of slow installations by creating the problem of combinatorial platform-specific builds. PyPI began hosting multiple wheel files per package version, organized by Python version, ABI, and platform. The ecosystem migrated slowly, such that for several years one might find wheels for some packages and not others, making installation time unpredictable in novel ways.
pipenv (2017): Mr. Reitz's pipenv aimed to unify pip and virtualenv into a single tool, which proved to be rather more ambitious than initially apparent. The dependency resolver, while technically correct, required sufficient time to operate that users would begin the installation process and then attend to other matters, occasionally returning to find it still resolving. The official endorsement by Python.org in November 2017 lent pipenv a certain authority, though this authority did not extend to making the resolver faster. The community tried pipenv with considerable enthusiasm and then, after some months, quietly returned to their previous workflows, rarely discussing the matter in detail.
poetry (2018): M. Eustace's poetry arrived with modern dependency resolution and a clean command-line interface. It introduced its own lock file format and workflow, which is to say it solved the fragmentation problem by adding another option. The tool was genuinely better than its predecessors in several respects, though this created the curious situation where the ecosystem now had three main approaches (pip, pipenv, poetry), each with advocates who found the others unsuitable. Major projects adopted poetry, demonstrating that it was serious tooling, though one might observe that "major projects adopted poetry" and "major projects continued using pip" were both accurate statements about the ecosystem simultaneously.
pip's dependency resolver (2020): After more than a decade, pip 20.3 acquired a proper dependency resolver that would refuse to install incompatible packages, which broke numerous workflows that had relied on pip's previous tolerance for inconsistency. The resolver employed backtracking, occasionally spending considerable time downloading and evaluating different package versions before determining that no compatible set existed - or, more frustratingly, before finding one. The pip developers staged a careful rollout with warnings and an opt-out flag, which rather suggests they anticipated the difficulties. The new resolver was more correct, which is a different thing from saying it was more convenient.
pyproject.toml (2016-2020): PEP 518 and PEP 621 standardized project metadata in pyproject.toml, solving the chicken-and-egg problem of specifying build dependencies. Multiple build backends now read pyproject.toml, though they read different subsets of it, and setuptools still occasionally required setup.cfg or setup.py for certain features. The migration from setup.py proceeded gradually, which is to say that five years after PEP 518, the ecosystem still contained both approaches, with different tools expecting different configurations. The standardization was welcome, though one might observe that complete standardization remained elusive.
uv (2024): Astral's uv demonstrated that one could achieve dramatic speedups in Python package management by writing it in Rust, which rather raises the question of why Python tooling should be written in Python at all. The tool is genuinely fast and well-designed. It is also venture-backed, which introduces questions about sustainability that the Python community will answer in due course. The fact that uv can replace pip, virtualenv, pipx, poetry, and pyenv simultaneously is either a testament to excellent engineering or an indictment of the preceding twenty-five years of fragmentation, depending on one's perspective. Astral is rapidly iterating, which is encouraging, though rapid iteration is easier at the beginning than after ten million users depend on stable behavior.
distutils (1998-2000)
- Started at 1998 International Python Conference session "Building Extensions Considered Painful"
- Created by Greg Ward with input from Greg Stein and Barry Warsaw
- Included in Python 1.6 (2000)
- Provided
setup.pyinterface using Python code - Enabled package installation but no dependency management
- Included C extension compilation support
setuptools (2004)
- Created by Phillip Eby
- Introduced Egg format (zipped packages)
- Added automatic dependency fetching and installation
- Provided
easy_installcommand-line tool - Could not uninstall packages cleanly
- Installed dependencies at installation time without version locking
- Included namespace packages, optional dependencies, and various advanced features
pip (2008)
- Created by Ian Bicking as "pyinstall", renamed to "pip" (recursive: pip installs packages)
- Drop-in replacement for easy_install
- Could uninstall packages
- Naive dependency resolution: installed packages sequentially in order encountered
- "Last one wins" for conflicting requirements
- Installed from PyPI by default
virtualenv (2007)
- Created by Ian Bicking
- Created isolated Python environments
- Prevented global site-packages pollution
- Required manual activation via shell-specific scripts (bash, Windows batch)
- Each virtualenv linked to specific Python installation
requirements.txt (~2010)
- Community convention, not PEP-specified
- Plain text file listing package dependencies
- Format: one package per line with optional version specifiers
- Used with
pip install -r requirements.txt - No transitive dependency locking
- Specified direct dependencies only
wheel (PEP 427, September 2012, accepted February 2013)
- Created by Daniel Holth
- Binary distribution format replacing eggs
- ZIP archive with .whl extension
- Platform-specific builds (Python version, ABI, platform tags)
- Faster installation than source distributions
- Separated build-time from install-time
- pip preferentially installs wheels when available (since pip 6.0, 2014)
pipenv (January 2017)
- Created by Kenneth Reitz
- Unified pip + virtualenv + lockfile management
- Introduced Pipfile and Pipfile.lock
- Used dependency resolver with backtracking
- Official PyPA recommendation (November 2017)
- Performance issues with dependency resolution for complex projects
poetry (April 2018)
- Created by Sébastien Eustace
- Used PubGrub dependency resolution algorithm
- Unified dependency management, building, and publishing
- Introduced pyproject.toml-based configuration
- Generated poetry.lock file
- Integrated virtual environment management
pip dependency resolver (pip 20.3, October-November 2020)
- Introduced proper backtracking dependency resolver
- Replaced "first specification wins" algorithm
- Could detect and reject incompatible dependency combinations
- Staged rollout: warnings in 20.2, default in 20.3
- Included
--use-deprecated=legacy-resolveropt-out flag - Slower resolution for complex dependency graphs
pyproject.toml standardization (PEP 518 May 2016, PEP 621 November 2020)
- PEP 518: Defined
[build-system]table for build dependencies - PEP 621: Defined
[project]table for package metadata - TOML format chosen over JSON, YAML, INI
- Supported by multiple build backends: setuptools, flit, hatchling, poetry-core
- Replaced setup.py for build configuration
- Gradual ecosystem migration from setup.py
uv (February 2024)
- Created by Astral (Charlie Marsh and team)
- Written in Rust
- 10-100x faster than pip for common operations
- Drop-in replacement for pip, pip-tools, virtualenv, pipx
- Extends to project management, tool installation, Python version management
- Uses PubGrub-inspired dependency resolution
- Single static binary
- Venture-backed company (Astral)