A Python-Selenium script to bulk take screenshots of webpage using headless Chrome by reading a text file full of URLs Tutorial: https://www.ilovefreesoftware.com/26/tutorial/how-to-take-full-page-screenshot-in-bulk-from-multiple-urls.html

bulk_webpage_screenshots.py

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.by import By
from tqdm import tqdm
import time

lines = []
Links_File = r''
OP_DIR = r''
i = 1

S = lambda X: driver.execute_script('return document.body.scrollHeight') + X

with open(Links_File, "r") as f:
   lines = f.readlines()
lines = [line.rstrip() for line in lines]

options = webdriver.ChromeOptions()
options.headless = True
options.add_argument('--log-level=3')
driver = webdriver.Chrome(options=options)
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.4103.97 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
for link in tqdm(lines, ncols=65):
   try:
        driver.get(link)
        time.sleep(5)
        driver.set_window_size(1024,S(0))  # May need manual adjustment
        driver.find_element(By.TAG_NAME,"body").screenshot(f'{OP_DIR}\{i}.png')
        i = i + 1
   except WebDriverException:
        print(link)
        continue       
driver.quit()

Great script thank you, do you have a workaround to accept all the cookie banner and avoid that some screenshot are grayed?

You can do it in two ways.

1. Inject JavaScript based on website you are taking screenshot of.
2. Load Chrome with an extension installed that will block the cookie and other popups. Try with https://adlock.com/ or https://crumbs.org/en/
   You will need CRX file of any of these extensions that you can get using this: https://chrome.google.com/webstore/detail/get-crx/dijpllakibenlejkbajahncialkbdkjc

Now, you can load the extension using CRX like this:

options.add_extension('pathToCRX')

Add this after line 18 and update PATH to the CRX file of the extension.

Thank you following yours suggestions worked fine!

You can do it in two ways.

1. Inject JavaScript based on website you are taking screenshot of.
2. Load Chrome with an extension installed that will block the cookie and other popups. Try with https://adlock.com/ or https://crumbs.org/en/
   You will need CRX file of any of these extensions that you can get using this: https://chrome.google.com/webstore/detail/get-crx/dijpllakibenlejkbajahncialkbdkjc

Now, you can load the extension using CRX like this:

options.add_argument('pathToCRX')

Add this after line 18 and update PATH to the CRX file of the extension.

Thank you following yours suggestions worked fine!