Last active
July 16, 2018 07:44
-
-
Save santiagobasulto/141cb786aa789892a3dd14cabc473482 to your computer and use it in GitHub Desktop.
Despite multiple efforts, I couldn't figure out how to evaluate generators in parallel. Ideas are welcome.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_repo_stars(org, repo): | |
| url = 'https://api.github.com/repos/{org}/{repo}'.format( | |
| org=org, repo=repo) | |
| print("GET ", url) | |
| resp = requests.get(url) | |
| return resp.json()['stargazers_count'] | |
| params = [ | |
| ('requests', 'requests'), | |
| ('requests', 'httpbin'), | |
| ('django', 'django'), | |
| ('Lukasa', 'hyper'), | |
| ] | |
| generator = (get_repo_stars(org, repo) for org, repo in params) | |
| def parallel(generator): | |
| def _next(): | |
| try: | |
| val = next(generator) | |
| print(val) | |
| except StopIteration: | |
| pass | |
| threads = [threading.Thread(target=_next) for _ in range(2)] | |
| [t.start() for t in threads] | |
| [t.join() for t in threads] | |
| parallel(generator) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Generators are always synchronous, but you can make the request async (and therefore do it concurrently).