Skip to content

Instantly share code, notes, and snippets.

@rvanweerd
Forked from quinn-joyce/web-scraper-node-js.md
Created December 31, 2021 12:55
Show Gist options
  • Select an option

  • Save rvanweerd/885a36946445b7afa7f76f22c06fb375 to your computer and use it in GitHub Desktop.

Select an option

Save rvanweerd/885a36946445b7afa7f76f22c06fb375 to your computer and use it in GitHub Desktop.
Node web scraper with axios and cheerio

Node JS Web Scraper

I this is part of the first node web scraper I created with axios and cheerio. I took out all of the logic, since I only wanted to showcase how a basic setup for a nodejs web scraper would look.

const cheerio = require('cheerio'),
      axios = require('axios'),
      url = `<url goes here>`;
      
axios.get(url)
    .then((response) => {
        let $ = cheerio.load(response.data);
        $('a').each(function (i, e) {
          let links = $(e).attr('href');      
          console.log(links);
      })
    }).catch(function (e) {
    console.log(e);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment