Last active
July 16, 2019 17:19
-
-
Save vishwasnavadak/bf461449edd7683b1535de60d7815e13 to your computer and use it in GitHub Desktop.
Scraping the web with Nodejs. Code for the blog post -> https://vishwas.tech/blog/2019/07/12/scraping-with-nodejs.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
const axios = require("axios"); | |
const cheerio = require("cheerio"); | |
const scrape = async url => { | |
const { data } = await axios.get(url).catch(err => console.log(err)); | |
const $ = cheerio.load(data); | |
const result = $(".skill-col").text(); | |
return result; | |
}; | |
(async ()=> { | |
console.log(await scrape("https://vishwas.tech/")) | |
} )() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
const Nightmare = require("nightmare"); | |
const nightmare = Nightmare({ show: true }); | |
nightmare | |
.goto("https://duckduckgo.com") | |
.type("#search_form_input_homepage", "how to scrape using nightmarejs") | |
.click("#search_button_homepage") | |
.wait("#r1-0 a.result__a") | |
.evaluate(() => document.querySelector("#r1-0 a.result__a").href) | |
.end() | |
.then(console.log) | |
.catch(error => { | |
console.error("Search failed:", error); | |
}); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment