Skip to content

Instantly share code, notes, and snippets.

@Bertus-W
Bertus-W / middleware.py
Created July 28, 2024 16:17
Scrapy Playwright RAM Management
import asyncio
from asyncio import Lock
from playwright.async_api import async_playwright
from scrapy import signals
from scrapy.http import HtmlResponse
class PlaywrightMiddleware:
def __init__(self, crawler):