Skip to content

Instantly share code, notes, and snippets.

@DZuz14
Last active March 21, 2020 16:15
Show Gist options
  • Save DZuz14/f83467d023b52a6cceb259f863ba8ee3 to your computer and use it in GitHub Desktop.
Save DZuz14/f83467d023b52a6cceb259f863ba8ee3 to your computer and use it in GitHub Desktop.
Web Scraping With Node.js & Puppeteer 2
const puppeteer = require('puppeteer');
const fs = require('fs')
async function main() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://some-website.com');
// Return an array of all the link text and assign it to sideBarLinks
const sidebarLinks = await page.evaluate(() => {
const links = document.querySelectorAll('ul.sidebar li a');
return Array.from(links).map(a => a.innerText().trim());
});
// You could use the synchronous or asychronous writeFile methods. Up to you.
fs.writeFileSync('./sidebar-links.json', JSON.stringify(sidebarLinks));
await browser.close()
}
main();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment