Skip to content

Instantly share code, notes, and snippets.

@scrapehero
Created January 17, 2019 05:55
Show Gist options
  • Save scrapehero/f9eafcf27e794fb2bf43fd34403ad270 to your computer and use it in GitHub Desktop.
Save scrapehero/f9eafcf27e794fb2bf43fd34403ad270 to your computer and use it in GitHub Desktop.
Sitemap to extract product data and pricing from Walmart using web scraper.io extension
{
"_id": "walmart",
"startUrl": [
"https://www.walmart.com/browse/home-improvement/electrical/1072864_1067619?povid=1072864+%7C+2018-05-02+%7C+Flyout_Electrical"
],
"selectors": [
{
"id": "product",
"type": "SelectorElementClick",
"parentSelectors": [
"_root"
],
"selector": "li.Grid-col",
"multiple": true,
"delay": "2000",
"clickElementSelector": "button.elc-icon.paginator-btn-next",
"clickType": "clickMore",
"discardInitialElements": false,
"clickElementUniquenessType": "uniqueText"
},
{
"id": "name",
"type": "SelectorText",
"parentSelectors": [
"product"
],
"selector": "a.product-title-link span",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "price",
"type": "SelectorText",
"parentSelectors": [
"product"
],
"selector": "div.price-main-block span.price",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "rating",
"type": "SelectorText",
"parentSelectors": [
"product"
],
"selector": "span.seo-avg-rating",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "number_of_reviews",
"type": "SelectorText",
"parentSelectors": [
"product"
],
"selector": "span.stars-reviews-count",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "image",
"type": "SelectorImage",
"parentSelectors": [
"product"
],
"selector": "img",
"multiple": false,
"delay": 0
},
{
"id": "shipping",
"type": "SelectorText",
"parentSelectors": [
"product"
],
"selector": "div.ShippingMessage-container",
"multiple": false,
"regex": "",
"delay": 0
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment