Skip to content

Instantly share code, notes, and snippets.

@adrianhajdin
Created July 9, 2021 08:21
Show Gist options
  • Select an option

  • Save adrianhajdin/13a69e174448b112785523128d34139c to your computer and use it in GitHub Desktop.

Select an option

Save adrianhajdin/13a69e174448b112785523128d34139c to your computer and use it in GitHub Desktop.
Amazon Scraper API
const express = require('express');
const request = require('request-promise');
const PORT = process.env.PORT || 5000;
const app = express();
app.use(express.json());
const returnScraperApiUrl = (apiKey) => `http://api.scraperapi.com?api_key=${apiKey}&autoparse=true`;
// Welcome route
app.get('/', async (req, res) => {
res.send('Welcome to Amazon Scraper API!');
});
// Get product details
app.get('/products/:productId', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/dp/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get product reviews
app.get('/products/:productId/reviews', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/product-reviews/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get product offers
app.get('/products/:productId/offers', async (req, res) => {
const { productId } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/gp/offer-listing/${productId}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
// Get search results
app.get('/search/:searchQuery', async (req, res) => {
const { searchQuery } = req.params;
const { api_key } = req.query;
try {
const response = await request(`${returnScraperApiUrl(api_key)}&url=https://www.amazon.com/s?k=${searchQuery}`);
res.json(JSON.parse(response));
} catch (error) {
res.json(error);
}
});
app.listen(PORT, () => console.log(`Server Running on Port: ${PORT}`));
`${baseUrl}&url=https://www.amazon.com/dp/${productId}`)
`${baseUrl}&url=https://www.amazon.com/product-reviews/${productId}`);
`${baseUrl}&url=https://www.amazon.com/gp/offer-listing/${productId}`);
`${baseUrl}&url=https://www.amazon.com/s?k=${searchQuery}`);
@mwaikar
Copy link
Copy Markdown

mwaikar commented Jul 10, 2021

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

@akbc1221
Copy link
Copy Markdown

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

It is working for amazon india domain as well

@mwaikar
Copy link
Copy Markdown

mwaikar commented Jul 11, 2021

If I try using amazon.in instead of amazon.com ,then it does not return any information regarding a product. Any idea why would that be ?

It is working for amazon india domain as well

Yes, i tried it now again and it's working.

@darbdenral
Copy link
Copy Markdown

darbdenral commented Jul 24, 2021

I don't get price for the product using the Get product details

@danju4rizzl
Copy link
Copy Markdown

Same here i don't get product details

@moustaphahamra
Copy link
Copy Markdown

Hello there, amazing project by the way.
But I have a problem when testing the api endpoints whatever i do it will give me statusCodeError 404.
Message: Amazon.com page NOt Found.
Where do you think the problem is.
And thank you.

@Nicxzmiller
Copy link
Copy Markdown

@moustaphahamra Do you have a link to your code so I can help you check it out?
You can paste it here so I can check it out

@buddhi-ashen
Copy link
Copy Markdown

I get product details, offers, reviews everything. but when I click open app button in Heroku it says application error

@Nathan-Yennupiak
Copy link
Copy Markdown

I keep getting Application error when I deploy to heroku and try to open it

@devesh-21-hub
Copy link
Copy Markdown

Great, thank you!

@YakovTamam
Copy link
Copy Markdown

hi thank you about your guide in youtube its amazing,
I do just like you and i get error 404 can you help me with that

@downinahill
Copy link
Copy Markdown

image

i get this error

@JamesCartar
Copy link
Copy Markdown

when i try with ebay, it doesn't return object of data. it's return the css file

@mansi-done
Copy link
Copy Markdown

hey, thanks for the amazing project.
However, the scrapping only works for amazon, any idea how we can use it to get JSON objects from other sites?

@priyankasaini69
Copy link
Copy Markdown

"access-control-allow-credentials":

"true"
"access-control-allow-origin":

"*"
"connection":

"keep-alive"
"content-length":

"2"
"content-type":

"application/json; charset=utf-8"
"date":

"Fri, 01 Jul 2022 16:16:52 GMT"
"etag":

"W/"2-vyGp6PvFo4RvsFtPoIWeCReyIC8""
"server":

"RapidAPI-1.2.8"
"via":

"1.1 vegur"
"x-powered-by":

"Express"
"x-rapidapi-region":

"AWS - ap-southeast-1"
"x-rapidapi-version":

"1.2.8"
Response Body
{0 items
}

I'm getting 0 items under response body in all Endpoints cases.
How can i solveit?

@alihamzarao6
Copy link
Copy Markdown

alihamzarao6 commented Jan 28, 2023

2

Getting this error. Anyone knows about it? @buddhi-ashen @adrianhajdin @Nicxzmiller

@Divyansh-singh08
Copy link
Copy Markdown

hi i am getting error
can u tell how can i fix this...
api

@Divyansh-singh08
Copy link
Copy Markdown

@adrianhajdin plz look by above code and help me here

@DevGod100
Copy link
Copy Markdown

hi i am getting error can u tell how can i fix this... api

Me too, can someone answer this issue please. @buddhi-ashen @adrianhajdin @Nicxzmiller

Does it have to do with the request being deprecated?

@neelammkw
Copy link
Copy Markdown

can we upload it any other platform now on heroku have to insert credit card info pls help

@1-Ujjwal
Copy link
Copy Markdown

1-Ujjwal commented Sep 4, 2024

can you tell the other hosting platform because now heroku is not available for free

@JokerHin
Copy link
Copy Markdown

can you tell the other hosting platform because now heroku is not available for free

maybe u can use vercel or netlify, both are free btw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment