Created
March 12, 2020 13:48
-
-
Save pfmoore/36c3d3ace6ec38ce42515ff540411634 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lifecycle of an InstallRequirement | |
================================== | |
Three types of InstallRequirement: | |
* A specifier, something like "pip>=19.0" or just "pip". | |
* A location, which could be: | |
- A URL - https://example.com/wheels/foo-1.0-py3-none-any.whl | |
- A directory - D:\Wheels\foo-1.0-py3-none-any.whl | |
- A VCS URL - git+https://github.com/pypa/pip | |
* An editable, something like "-e ." | |
There are also lots of other terms, like "direct". We'll come to them... | |
Creating an InstallRequirement | |
============================== | |
Lots of exciting parsing goes on, but it all ends up at one of the | |
constructors in pip._internal.req.constructors: | |
* install_req_from_editable | |
* install_req_from_line | |
Constructing the resolver uses make_req_from_req_string. TODO: go back | |
to this path and check. For now, ignore it for the sake of my sanity. | |
line: | |
return InstallRequirement( | |
parts.requirement, | |
* comes_from, | |
* link=parts.link, | |
markers=parts.markers, | |
* use_pep517=use_pep517, | |
* isolated=isolated, | |
* install_options=options.get("install_options", []) if options else [], | |
* global_options=options.get("global_options", []) if options else [], | |
* hash_options=options.get("hashes", {}) if options else {}, | |
* wheel_cache=wheel_cache, | |
* constraint=constraint, | |
* extras=parts.extras, | |
) | |
editable: | |
return InstallRequirement( | |
parts.requirement, | |
* comes_from, | |
source_dir=source_dir, | |
editable=True, | |
* link=parts.link, | |
* constraint=constraint, | |
* use_pep517=use_pep517, | |
* isolated=isolated, | |
* install_options=options.get("install_options", []) if options else [], | |
* global_options=options.get("global_options", []) if options else [], | |
* hash_options=options.get("hashes", {}) if options else {}, | |
* wheel_cache=wheel_cache, | |
* extras=parts.extras, | |
) | |
Note that the first argument is a Requirement (but not an InstallRequirement!) | |
This is from pip._vendor.packaging.requirements. | |
So we are now at the constructor of the class. | |
Most stuff just bunged into attributes. | |
* source_dir is normpath(abspath()) | |
* if req and req.url (PEP 508 URL req) then set link to req.url if not given explicitly | |
* local_file_path is from link, if it's a file | |
* extras come from req if not set | |
* markers come from req if not set | |
Plus a chunk of (mutable) state: | |
* satisfied_by | |
* should_reinstall | |
* _temp_build_dir | |
Set in ensure_build_location, which is used in ensure_has_source_dir | |
to set source_dir if unset. Which in turn is called in prepare. | |
* install_succeeded | |
* prepared | |
Set in resolver, *not* in operations.prepare!!! | |
Only used by resolver...!!! | |
* is_direct | |
Set in req_command... | |
Means "Was specified by the user in the command line or requirement/constraint file" | |
* successfully_downloaded | |
* build_env | |
Set in pip._internal.distributions.sdist | |
* metadata_directory | |
Set by prepare_metadata() from _generate_metadata() | |
* pyproject_requires | |
* requirements_to_check | |
* pep517_backend | |
* use_pep517 | |
These 4 set by load_pyproject_toml from pip._internal.distributions.sdist | |
Install Command | |
=============== | |
Question - what is a requirement tracker?? | |
get_requirements - from args. Note check_supported_wheels, true unless --target. | |
* Parse constraint files. Reqs have constraint=true | |
* Add rqs from command line | |
* Add editable reqs | |
* Add reqs from requirement files | |
All these reqs are is_direct=True | |
Question - what state are requirements in at this point?? | |
Answer - freshly built, no significant (lifecycle) methods called. | |
resolve. | |
Question - what state are requirements in at this point?? | |
Go on to build and install steps here. | |
Resolve Process | |
=============== | |
_resolve_one | |
Set prepared | |
_get_abstract_dist_for, get_pkg_resources_distribution | |
** What is an "unnamed" requirement (quotes from comment in the code - hence the question!)? Do we care? | |
It's an artifact of the resolver and reuqirement sets. | |
* Some requirements (e.g., ".") don't have a defined name until we get the metadata. | |
* It's possible to have unnamed requirements as *dependencies* (foo depends on ".")?? Maybe? | |
* Need to rescan to get the name. Not sure if rescanning rather than preparing now is | |
just because the existing resolver sucks, or if there's a better reason... | |
_get_abstract_dist_for | |
Editable reqs get prepared. That's it for them. Simples. | |
req.ensure_has_source_dir | |
update_editable | |
_get_prepared_distribution | |
_check_skip_installed. Sets satisfied_by. | |
*This is the code that makes installed requirements!* | |
So it's not until this point that we know if this is an install or an upgrade. | |
populate_link | |
Calls the finder to find the requirement (if .link not set) | |
Checks for a cached wheel and uses that if possible | |
prepare_linked_requirement | |
req.ensure_has_source_dir | |
unpack_url | |
_get_prepared_distribution | |
<< req.archive() on VCS source dir >> - only of interest because it's the place req.archive is used. | |
I'm going to say at this point that I consider "prepared" as the "complete" point for the resolver, because preparing | |
returns an abstract_dist, which can be converted into a pkg_resources distribution. | |
But if you *lose* that abstract_dist, you can't get it back without preparing again... | |
_get_prepared_distribution(req,tracker,finder,build_isolation) | |
make_distribution_for_install_requirement | |
tracker.track(req) | |
prepare_distribution_metadata(finder, build_isolation) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment