Skip to content

Instantly share code, notes, and snippets.

@KustomDeveloper
Last active August 28, 2020 16:21
Show Gist options
  • Select an option

  • Save KustomDeveloper/8c031bf806993eb65260b7eadae87ae7 to your computer and use it in GitHub Desktop.

Select an option

Save KustomDeveloper/8c031bf806993eb65260b7eadae87ae7 to your computer and use it in GitHub Desktop.
Puppeteer script to check if a site has a noindex tag
import {puppeteer} from 'puppeteer';
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('http://devstaging1.jsabeta.com/');
const meta = await page.$eval("head > meta[name='robots']", element => element.content);
console.log(meta);
meta.includes("noindex") ? console.log('Send Alert') : console.log('Site Okay');
await browser.close();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment