A proposal for a new output format for pip

README.md

Reworking the output of pip install, download, wheel

Current Output

Collecting Flask
  Downloading Flask-0.12.2-py2.py3-none-any.whl (83kB)
    100% |████████████████████████████████| 92kB 296kB/s
Collecting itsdangerous>=0.21 (from Flask)
  Downloading itsdangerous-0.24.tar.gz (46kB)
    100% |████████████████████████████████| 51kB 385kB/s
Collecting Jinja2>=2.4 (from Flask)
  Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
    100% |████████████████████████████████| 348kB 568kB/s
Collecting click>=2.0 (from Flask)
  Downloading click-6.7-py2.py3-none-any.whl (71kB)
    100% |████████████████████████████████| 71kB 609kB/s
Collecting Werkzeug>=0.7 (from Flask)
  Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
    100% |████████████████████████████████| 317kB 518kB/s
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask)
  Downloading MarkupSafe-1.0.tar.gz
Building wheels for collected packages: itsdangerous, MarkupSafe
  Running setup.py bdist_wheel for itsdangerous ... done
  Stored in directory: /Users/pradyunsg/Library/Caches/pip/wheels/fc/a8/66/24d655233c757e178d45dea2de22a04c6d92766abfb741129a
  Running setup.py bdist_wheel for MarkupSafe ... done
  Stored in directory: /Users/pradyunsg/Library/Caches/pip/wheels/88/a7/30/e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57
Successfully built itsdangerous MarkupSafe
Installing collected packages: itsdangerous, MarkupSafe, Jinja2, click, Werkzeug, Flask
Successfully installed Flask-0.12.2 Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 itsdangerous-0.24

Collecting Flask
  Using cached Flask-0.12.2-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask)
  Using cached click-6.7-py2.py3-none-any.whl
Collecting Jinja2>=2.4 (from Flask)
  Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask)
  Using cached Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from Flask)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask)
Installing collected packages: click, MarkupSafe, Jinja2, Werkzeug, itsdangerous, Flask
Successfully installed Flask-0.12.2 Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 itsdangerous-0.24

General Notes

I feel that using a PixelBar for progress is nicer because the current progress bar - IncrementalBar shines like a tubelight on my light-on-dark terminal. On the other hand, the PixelBar looks much dimmer with equivalent progress reporting resolution.

I'm gonna use PixelBar in the proposals below because I like it more. The logic/structure won't change based on the progress bar theme anyway, so, there's that.

|████████████████████████████████|

|⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿|

There's a lot of text in the non-verbose output - it does contain a lot of information but a fair amount of it is not needed often.

Further, I'm gonna introduce a debug output variant; this allows for the default output to be more compact, since there's a verbose mode that can show more things while not being overwhelming.

0: Vanishing Progress Bars and updating lines

pip install Flask

Instead of having a top level "Collecting <package>", I propose:

Fetching packages:
  Flask {spinner}

The spinner stays there until the version of the package that would be used is determined. Once that happens, the line is re-written to mention what's happening.

Fetching packages:
  Flask == 0.12.2 - downloading wheel
    |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇  | 80kB/83kB 1.31kB/s

Once the file gets downloaded, the progress bar line is cleared and the requirement line gets updated accordingly; the file was downloaded, it was a wheel/sdist and what its size is.

Fetching packages:
  Flask == 0.12.2 - downloaded wheel (83kB)

Notice how square brackets are used to show the parent package. This is distinct from parenthesis used for size of files and looks nice as well.

Fetching packages:
  Flask == 0.12.2 - downloaded wheel (83kB)
  click == 6.7 [>= 2.0 from Flask] - using cached wheel

Once everything is fetched, pip might need to build wheels.

Fetching packages:
  Flask == 0.12.2 - downloaded wheel (83kB)
  click == 6.7 [>= 2.0 from Flask] - using cached wheel
  Jinja2 == 2.9.6 [from Flask] - downloaded wheel (340kB)
  itsdangerous == 0.24 [>= 0.21 from Flask] - downloaded sdist (46kB)
  Werkzeug == 0.12.2 [>= 0.7 from Flask] - downloaded wheel (312kB)
  MarkupSafe == 1.0 [>= 0.23 from Jinja2] - using cached wheel
Building wheels from sdists:
  itsdangerous == 0.24 - building {spinner}

Once a wheel is built, the line is rewritten and we proceed to the next package.

Fetching packages:
  Flask == 0.12.2 - downloaded sdist (83kB)
  click == 6.7 [>= 2.0 from Flask] - using cached wheel
  Jinja2 == 2.9.6 [from Flask] - downloaded wheel (340kB)
  itsdangerous == 0.24 [>= 0.21 from Flask] - downloaded sdist (46kB)
  Werkzeug == 0.12.2 [>= 0.7 from Flask] - downloaded wheel (312kB)
  MarkupSafe == 1.0 [>= 0.23 from Jinja2] - using cached wheel
Building wheels from sdists:
  itsdangerous == 0.24 - built and cached

Doing a similar "spinner then message" for installations, we end up with a final output that looks like:

Fetching packages:
  Flask == 0.12.2 - downloaded sdist (83kB)
  click == 6.7 [>= 2.0 from Flask] - using cached wheel
  Jinja2 == 2.9.6 [from Flask] - downloaded wheel (340kB)
  itsdangerous == 0.24 [>= 0.21 from Flask] - downloaded sdist (46kB)
  Werkzeug == 0.12.2 [>= 0.7 from Flask] - downloaded wheel (312kB)
  MarkupSafe == 1.0 [>= 0.23 from Jinja2] - using cached wheel
Building wheels from sdists:
  itsdangerous == 0.24 - built and cached
Installing packages:
  Flask == 0.12.2 - done
  click == 6.7 - done
  Jinja2 == 2.9.6 - done
  itsdangerous == 0.24 - done
  Werkzeug == 0.12.2 - done
  MarkupSafe == 1.0 - done
Successfully installed all packages.

Compare this to the output at the start of this document. :)

This output format has the nice benefit that the "download", "wheel" and "install" commands now visibly do certain steps of the process. "download" would just fetch packages, "wheel" would fetch and build wheels from sdists if needed and "install" would fetch, build wheels and install the built wheels.

We could use colours to highlight the some parts of the output, if needed. I don't feel the need to do so though. I'd like to though. :P

Sidenote - with extras, the square brackets for parent packages still looks fine to me; it could possibly use some improvement though...

  jupyter-client>=4.1 [from qtconsole; extra == "qtconsole"->ipython[qtconsole]] - downloaded wheel (84kB)

1: Show the final decisions and results

pip install --verbose Flask

Instead of having status being shown on a single line that gets updated, pip would print what it decided to do and show what it's doing.

Fetching packages:
  Flask
    Choosing version {spinner}

Fetching packages:
  Flask
    Chose 12.0.2 out of 23 versions available
    Downloading Flask-0.12.2-py2.py3-none-any.whl
      |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇  | 80kB/83kB 1.31kB/s

Fetching packages:
  Flask
    Chose 12.0.2 out of 23 versions available
    Downloaded Flask-0.12.2-py2.py3-none-any.whl (83kB)

Fetching packages:
  Flask
    Chose 12.0.2 out of 23 versions available
    Downloaded Flask-0.12.2-py2.py3-none-any.whl (83kB)
  click
    Chose 6.7 out of 23 versions available [>= 2.0 from Flask]
    Using cached click-6.7-py2.py3-none-any.whl
  

Fetching packages:
  Flask
    Chose 12.0.2 out of 23 versions available
    Downloaded Flask-0.12.2-py2.py3-none-any.whl (83kB)
  click
    Chose 6.7 out of 23 versions available [>= 2.0 from Flask]
    Using cached click-6.7-py2.py3-none-any.whl
  Jinja2 
    Chose 2.9.6 out of 12 versions available [from Flask]
    Downloaded Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
  itsdangerous
    Chose 0.24 out of n versions available [>= 0.21 from Flask]
    Downloaded itsdangerous-0.24.tar.gz (46kB)
  Werkzeug
    Chose 0.12.2 from n versions available [>= 0.7 from Flask]
    Downloaded Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
  MarkupSafe
    Chose 1.0 from n versions available [>=0.23 from Jinja2 >= 2.4 -> Flask]
    Downloaded MarkupSafe-1.0.tar.gz (100kB)
Building wheels from sdists:
  itsdangerous == 0.24
    Built and cached itsdangerous-0.24-py2.py3-none-any.whl
Installing packages:
  Flask == 0.12.2 - done
  click == 6.7 - done
  Jinja2 == 2.9.6 - done
  itsdangerous == 0.24 - done
  Werkzeug == 0.12.2 - done
  MarkupSafe == 1.0 - done
Successfully installed all packages.

2: Debug mode - show all the information

pip install --debug Flask or pip install --verbose --verbose Flask

This would be equivalent to the current --verbose output. Out of the 2 spellings, I have a preference for the former given that the only time this information is useful is when debugging why something is happening.

The proposed output would be a slightly re-structured version of the current output, with more use of indentation than the current verbose output. The thing it adds is some output when installing the packages (since that is the only place where adding verbosity currently doesn't really make things print more stuff)

I still have to go through this properly; there's a lot that would get printed here. :)

• Have the same top-level structure as --verbose
• Use indentation generously
  • Nice when copying to a text editor that allows collapsing indented blocks
• Show subprocess commands and their output indented under that
• Show all links and paths being touched
• Show reasons for decisions, if any
• more?

How about:

from tqdm import tqdm
with tqdm(total=total_bytes, ascii=True, leave=verbose > 1, miniters=1, unit='B', unit_divisor=1024, unit_scale=True,
          desc="Downloading {thing}", disable=verbose <= 0) as t:
    download(thing, callback=t.update)