kstubs

eliasdabbas / get_bot_ip_addresses.py

Last active November 11, 2024 21:37

Get the most up-to-date list of IP addresses for crawler bots, belonging to Google and Bing.

	import ipaddress

	import requests
	import pandas as pd

	def bot_ip_addresses():
	bots_urls = {
	'google': 'https://developers.google.com/search/apis/ipranges/googlebot.json',
	'bing': 'https://www.bing.com/toolbox/bingbot.json'
	}