Skip to content

Instantly share code, notes, and snippets.

View ps-jessejjohnson's full-sized avatar

Jesse Johnson ps-jessejjohnson

View GitHub Profile
@ps-jessejjohnson
ps-jessejjohnson / set-user-agent.js
Created May 1, 2020 00:11
Set User-Agent header with Puppeteer
// Intellectual genuineness of
// Anthony McNamara (aka Big Red)
// Great article regarding approach
// https://medium.com/@filipvitas/how-to-set-user-agent-header-with-puppeteer-js-and-not-fail-28c7a02165da
Object.defineProperty(navigator, 'Mozilla/5.0 (compatible; PriceSpider0.2; +https://www.pricespider.com/)', {
get: function() { return setPlat },
set: function(a) { }
});
@ps-jessejjohnson
ps-jessejjohnson / docker-compose.yaml
Last active July 14, 2024 16:31
RabbitMQ and Redis via Docker Compose
version: '3.6'
services:
rabbit:
image: rabbitmq:3-management
volumes:
- ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
ports:
- '8080:15672'
redis:
@ps-jessejjohnson
ps-jessejjohnson / bs4_tree_construct.py
Created October 27, 2020 22:33 — forked from wildmichael/bs4_tree_construct.py
Small utility to easily create a document tree with BeautifulSoup4
from bs4 import BeautifulSoup, Tag
class Attribute(object):
def __init__(self, attrname, attrvalue):
'Initializes a new attribute.'
self._attrname = attrname
self._attrvalue = attrvalue
@property
def name(self):
@ps-jessejjohnson
ps-jessejjohnson / launch.json
Created December 19, 2020 07:11
Debugging Ruby with VS Code
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug Test File (rspec)",
"type": "Ruby",
"request": "launch",
"cwd": "${workspaceRoot}",
"program": "/Users/jessejohnson/.rvm/gems/ruby-2.7.0@oris-intel-bot/bin/rspec",
"args": ["-fd", "${relativeFile}"],
@ps-jessejjohnson
ps-jessejjohnson / kill.sh
Last active September 28, 2022 17:17
Conflux (aka Conslidated Crawler/Pipeline) Scripts
#!/usr/local/bin/bash
pgrep -f rosetta | xargs kill
pgrep -f node | xargs kill -9
pgrep -f rake | xargs kill -9
sleep 10
for proc in node rake rosetta ;do
echo Checking for $proc processes
pgrep -f $proc
done
const pw = require('playwright');
const UserAgent = require('user-agents');
const uuid = require('uuid');
const tmp = require('tmp-promise');
const UINT32_MAX = (2 ** 32) - 1;
const WEBGL_RENDERERS = ['ANGLE (NVIDIA Quadro 2000M Direct3D11 vs_5_0 ps_5_0)', 'ANGLE (NVIDIA Quadro K420 Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (NVIDIA Quadro 2000M Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (NVIDIA Quadro K2000M Direct3D11 vs_5_0 ps_5_0)', 'ANGLE (Intel(R) HD Graphics Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (Intel(R) HD Graphics Family Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (ATI Radeon HD 3800 Series Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (Intel(R) HD Graphics 4000 Direct3D11 vs_5_0 ps_5_0)', 'ANGLE (Intel(R) HD Graphics 4000 Direct3D11 vs_5_0 ps_5_0)', 'ANGLE (AMD Radeon R9 200 Series Direct3D11 vs_5_0 ps_5_0)', 'ANGLE (Intel(R) HD Graphics Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (Intel(R) HD Graphics Family Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (Intel(R) HD Graphics Direct3D9Ex vs_3_0 ps_3_0)', 'ANGLE (Intel(R) HD Graphics Family Direct3D
@ps-jessejjohnson
ps-jessejjohnson / description.md
Created January 14, 2024 15:44 — forked from dgozman/description.md
Playwright network settled helper

Network Settled

This helper waits for requests initiated by the action to finish. It is similar to waitUntil: 'networkidle' option, but is designed to work with any action, like click.

Example usage:

const waitForNetworkSettled = require('./networkSettled');

// Wait for network settled after navigation,
@ps-jessejjohnson
ps-jessejjohnson / index.js
Last active March 24, 2024 01:25
Nodejs (Somewhat) Random TLS Fingerprint
import { DEFAULT_CIPHERS } from 'tls';
import https from 'https';
const shuffler = (a, b) => 0.5 - Math.random();
const defaultCiphers = DEFAULT_CIPHERS.split(':');
const shuffledCiphers = [
...defaultCiphers.slice(0, 2).sort(shuffler),
...defaultCiphers.slice(3).sort(shuffler)
].join(':');
@ps-jessejjohnson
ps-jessejjohnson / index.js
Created March 24, 2024 00:58
Nodejs HTTP Tunnel
import { createServer, request } from 'node:http';
import { connect } from 'node:net';
import { URL } from 'node:url';
// Create an HTTP tunneling proxy
const proxy = createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('okay');
});
proxy.on('connect', (req, clientSocket, head) => {
@ps-jessejjohnson
ps-jessejjohnson / squidanonymousproxy.md
Created April 19, 2024 06:09 — forked from RobinDev/squidanonymousproxy.md
Install a SQUID anonymous proxy
  1. Install SQUID
apt-get install squid
  1. Create an user
htpasswd -md /etc/squid3/users myuserlogin`