Skip to content

Instantly share code, notes, and snippets.

@elog08
Created April 17, 2018 02:22
Show Gist options
  • Save elog08/b274c1c0d404ac351bc1ac72aee5976d to your computer and use it in GitHub Desktop.
Save elog08/b274c1c0d404ac351bc1ac72aee5976d to your computer and use it in GitHub Desktop.
Scrape with Puppeteer
const puppeteer = require('puppeteer')
const script = require('./script');
const { writeFileSync } = require("fs");
function save(raw) {
writeFileSync('results.json', JSON.stringify(raw));
}
const URL = 'https://www.quora.com/search?q=meaning%20of%20life&type=answer';
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
page.on('console', msg => console.log(msg.text()));
await page.goto(URL);
const threads = await page.evaluate(script);
save(threads);
await browser.close();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment