Created
October 24, 2021 21:10
-
-
Save dcts/a1b689b88e61fe350a446a5799209c9b to your computer and use it in GitHub Desktop.
Scrape opensea floor prices with python cloudscraper package
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import cloudscraper | |
import json | |
def filter_typename(dict): | |
return dict["__typename"] == "AssetQuantityType" | |
def filter_quantityInEth_exists(dict): | |
if "quantityInEth" in dict: | |
return True | |
else: | |
return False | |
def get_floor_price_in_eth(dict): | |
return float(dict["quantity"]) / 1000000000000000000 | |
def get_floor_prices(slug): | |
scraper = cloudscraper.create_scraper( | |
browser={ | |
'browser': 'chrome', | |
'platform': 'android', | |
'desktop': False | |
} | |
) | |
url = "https://opensea.io/collection/{}?search[sortAscending]=true&search[sortBy]=PRICE&search[toggles][0]=BUY_NOW".format(slug); | |
html = scraper.get(url).text | |
json_string = html.split("</script>",2)[0].split("window.__wired__=",2)[1] | |
data = json.loads(json_string) | |
data_values = data["records"].values() # get all values type... | |
data_list = [*data_values] # convert to list =~ array in js | |
data_list = list(filter(filter_typename, data_list)) | |
data_list = list(filter(filter_quantityInEth_exists, data_list)) | |
data_list = list(map(get_floor_price_in_eth, data_list)) | |
return data_list | |
# scraping floor prices from opensea | |
print("RUNNING FOR cool-cats-nft") | |
print(get_floor_prices("cool-cats-nft")) | |
print("RUNNING FOR treeverse") | |
print(get_floor_prices("treeverse")) | |
This works locally, cloudflare is bypassed. Unfortunately when deployed to any cloud it stopps working. I tried
take a look at https://rapidapi.com/restyler/api/scrapeninja it seems to work fine with opensea, at least with api.opensea graphql requests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Oh ok, well, you for sure can get it through scraping, probably I would do the following:
https://opensea.io/collection/sneaky-vampire-syndicate?tab=activity
Good luck with the challenge! 🚀💪