Skip to content

Instantly share code, notes, and snippets.

@jerryan999
Created December 10, 2020 06:18
Show Gist options
  • Save jerryan999/a42853261c14339b98c522a9fad2441c to your computer and use it in GitHub Desktop.
Save jerryan999/a42853261c14339b98c522a9fad2441c to your computer and use it in GitHub Desktop.
crawl_by_puppeteer using abuyun proxy
const puppeteer = require("puppeteer");
const crypto = require('crypto');
// 代理服务器
const proxyHost = "http-dyn.abuyun.com";
const proxyPort = 9020;
const proxyServer = "http://" + proxyHost + ":" + proxyPort;
// 隧道验证信息
const proxyUser = "";
const proxyPass = "";
(async () => {
const browser = await puppeteer.launch({
headless: false,
args: ["--proxy-server=http://forward.xdaili.cn:80", '--no-sandbox']
});
const page = await browser.newPage();
await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36');
await page.authenticate({ username : proxyUser, password : proxyPass});
try {
await page.goto("https://arh.antoinevastel.com/bots/areyouheadless")
await page.goto("https://www.myip.com/");
}
catch(err) {
console.log(err);
}
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment