client-task-pool.py

client-task-pool.py

# client-task-pool.py

from aiohttp import ClientSession, TCPConnector
import asyncio
import sys
from pypeln.task import TaskPool

limit = 1000


async def fetch(url, session):
    async with session.get(url) as response:
        return await response.read()


async def _main(url, total_requests):
    connector = TCPConnector(limit=None)
    async with ClientSession(connector=connector) as session, TaskPool.create(limit) as tasks:

        for i in range(total_requests):
            await tasks.put(fetch(url.format(i), session))


url = "http://localhost:8080/{}"
loop = asyncio.get_event_loop()
loop.run_until_complete(_main(url, int(sys.argv[1])))

Updated gist, TaskPool is now a dataclass that has a create constructor:
https://github.com/cgarciae/pypeln/blob/5909376b30fe25fd869e49e4e46b7782d48f1be2/pypeln/task/worker.py#L193

@chitzinwin

@cgarciae I found the error messages when I ran this script:
TypeError: 'coroutine' object is not callable
sys:1: RuntimeWarning: coroutine 'fetch' was never awaited

I can run this script but I would like to ask you @cgarciae something. How should I write the code to handler the error and kill the remained task?

I can run this script but I would like to ask you @cgarciae something. How should I write the code to handler the error and kill the remained task?

l know its been ages but how did you manage to get it to run past the TypeError? l'm experiencing the same