Skip to content

Instantly share code, notes, and snippets.

@adrianhajdin
Created July 9, 2021 08:21
Show Gist options
  • Save adrianhajdin/13a69e174448b112785523128d34139c to your computer and use it in GitHub Desktop.
Save adrianhajdin/13a69e174448b112785523128d34139c to your computer and use it in GitHub Desktop.
Amazon Scraper API
const express = require('express');
const request = require('request-promise');
const PORT = process.env.PORT || 5000;
const app = express();
app.use(express.json());
const returnScraperApiUrl = (apiKey) => `http://api.scraperapi.com?api_key=${apiKey}&autoparse=true`;
// Welcome route
app.get('/', async (req, res) => {
res.send('Welcome to Amazon Scraper API!');
});
// Get product details
app.get('/products/:productId', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/dp/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get product reviews
app.get('/products/:productId/reviews', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/product-reviews/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get product offers
app.get('/products/:productId/offers', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/gp/offer-listing/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get search results
app.get('/search/:searchQuery', async (req, res) => {
const { searchQuery } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/s?k=${searchQuery}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
app.listen(PORT, () => console.log(`Server Running on Port: ${PORT}`));
`${baseUrl}&url=https://www.amazon.com/dp/${productId}`)
`${baseUrl}&url=https://www.amazon.com/product-reviews/${productId}`);
`${baseUrl}&url=https://www.amazon.com/gp/offer-listing/${productId}`);
`${baseUrl}&url=https://www.amazon.com/s?k=${searchQuery}`);
@mwaikar
Copy link

mwaikar commented Jul 10, 2021

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

@akbc1221
Copy link

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

It is working for amazon india domain as well

@mwaikar
Copy link

mwaikar commented Jul 11, 2021

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

It is working for amazon india domain as well

Yes, i tried it now again and it's working.

@darbdenral
Copy link

darbdenral commented Jul 24, 2021

I don't get price for the product using the Get product details

@danju4rizzl
Copy link

Same here i don't get product details

@moustaphahamra
Copy link

Hello there, amazing project by the way.
But I have a problem when testing the api endpoints whatever i do it will give me statusCodeError 404.
Message: Amazon.com page NOt Found.
Where do you think the problem is.
And thank you.

@Nicxzmiller
Copy link

@moustaphahamra Do you have a link to your code so I can help you check it out?
You can paste it here so I can check it out

@buddhi-ashen
Copy link

I get product details, offers, reviews everything. but when I click open app button in Heroku it says application error

@devesh-21-hub
Copy link

Great, thank you!

@YakovTamam
Copy link

hi thank you about your guide in youtube its amazing,
I do just like you and i get error 404 can you help me with that

@downinahill
Copy link

image

i get this error

@JamesCartar
Copy link

when i try with ebay, it doesn't return object of data. it's return the css file

@mansi-done
Copy link

hey, thanks for the amazing project.
However, the scrapping only works for amazon, any idea how we can use it to get JSON objects from other sites?

@priyankasaini69
Copy link

"access-control-allow-credentials":

"true"
"access-control-allow-origin":

"*"
"connection":

"keep-alive"
"content-length":

"2"
"content-type":

"application/json; charset=utf-8"
"date":

"Fri, 01 Jul 2022 16:16:52 GMT"
"etag":

"W/"2-vyGp6PvFo4RvsFtPoIWeCReyIC8""
"server":

"RapidAPI-1.2.8"
"via":

"1.1 vegur"
"x-powered-by":

"Express"
"x-rapidapi-region":

"AWS - ap-southeast-1"
"x-rapidapi-version":

"1.2.8"
Response Body
{0 items
}

I'm getting 0 items under response body in all Endpoints cases.
How can i solveit?

@alihamzarao6
Copy link

alihamzarao6 commented Jan 28, 2023

2

Getting this error. Anyone knows about it? @buddhi-ashen @adrianhajdin @Nicxzmiller

@Divyansh-singh08
Copy link

hi i am getting error
can u tell how can i fix this...
api

@Divyansh-singh08
Copy link

@adrianhajdin plz look by above code and help me here

@DevGod100
Copy link

hi i am getting error can u tell how can i fix this... api

Me too, can someone answer this issue please. @buddhi-ashen @adrianhajdin @Nicxzmiller

Does it have to do with the request being deprecated?

@neelammkw
Copy link

can we upload it any other platform now on heroku have to insert credit card info pls help

@1-Ujjwal
Copy link

1-Ujjwal commented Sep 4, 2024

can you tell the other hosting platform because now heroku is not available for free

@JokerHin
Copy link

can you tell the other hosting platform because now heroku is not available for free

maybe u can use vercel or netlify, both are free btw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment