Skip to content

Instantly share code, notes, and snippets.

@a3linux
Created April 25, 2018 09:04
Show Gist options
  • Select an option

  • Save a3linux/3295a95fbcd067c30782362ea3419052 to your computer and use it in GitHub Desktop.

Select an option

Save a3linux/3295a95fbcd067c30782362ea3419052 to your computer and use it in GitHub Desktop.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import asyncio
import aiohttp
import async_timeout
URLs = [
'https://aiohttp.readthedocs.org/',
'https://www.baidu.com/',
'https://www.google.com/',
'https://www.mozilla.org/',
'https://www.twitter.com/',
'https://www.ebay.com/',
'https://www.autodesk.com/',
'https://www.taobao.com/',
'https://www.jd.com/',
]
async def get_site(url):
async with aiohttp.ClientSession() as sess:
async with async_timeout.timeout(10):
async with sess.get(url) as res:
html = await res.text()
print("Length %s: %s" % (url, len(html)))
loop = asyncio.get_event_loop()
tasks = [get_site(site) for site in URLs]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment