Charlie Haley cshaley
cshaley
/ conda_forge_lister.py
Last active
August 31, 2017 14:02
List all conda forge packages and files on anaconda.org
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Goal is to list all conda forge packages and package files | |
| # that are hosted on anaconda.org | |
| # This program takes about 5 minutes to run locally. | |
| # 75% of time is URL fetching | |
| # 25% of time is pandas read_html (this could be cut significantly with custom code) | |
| import requests | |
| import time | |
| from joblib import Parallel, delayed |