quantumproxies

# Watch any webpage for content changes (pricing pages, docs, ToS, competitors).
# Extracts CONTENT as markdown before hashing, so ads/timestamps/session junk
# don't cause false positives like raw-HTML diffing does.
#
# Extraction: https://quantumproxies.io/extract-api (key: app.quantumproxies.io/api-keys)
#
#   QP_API_KEY=qp_live_... python page-change-monitor.py https://example.com/pricing
#   (cron it hourly; exit code 1 = page changed, so `&& notify` just works)

import hashlib, json, os, pathlib, sys

quantumproxies

# Watch any webpage for content changes (pricing pages, docs, ToS, competitors).
# Extracts CONTENT as markdown before hashing, so ads/timestamps/session junk
# don't cause false positives like raw-HTML diffing does.
#
# Extraction: https://quantumproxies.io/extract-api (key: app.quantumproxies.io/api-keys)
#
#   QP_API_KEY=qp_live_... python page-change-monitor.py https://example.com/pricing
#   (cron it hourly; exit code 1 = page changed, so `&& notify` just works)

import hashlib, json, os, pathlib, sys

quantumproxies

# SEO content research: fetch Google's top-5 for a keyword, extract each
# page's heading outline (H2/H3), and print them side by side — what you
# must cover to compete for that query.
#
# SERP + extraction: https://quantumproxies.io/serp-api +
# https://quantumproxies.io/extract-api (one key for both:
# app.quantumproxies.io/api-keys)
#
#   QP_API_KEY=qp_live_... python competitor-outline-analyzer.py "best vpn 2026"

quantumproxies

# SEO content research: fetch Google's top-5 for a keyword, extract each
# page's heading outline (H2/H3), and print them side by side — what you
# must cover to compete for that query.
#
# SERP + extraction: https://quantumproxies.io/serp-api +
# https://quantumproxies.io/extract-api (one key for both:
# app.quantumproxies.io/api-keys)
#
#   QP_API_KEY=qp_live_... python competitor-outline-analyzer.py "best vpn 2026"

quantumproxies

"""
Rotating proxies in Python: retry, backoff and 429 handling that actually works.

The naive version of this — pick a proxy, send the request, catch the exception —
fails in production for three reasons that are easy to miss:

  1. A 429 is NOT a proxy failure. Rotating the exit IP on every 429 burns your
     pool for nothing: the block often lives on the *account* or the fingerprint,
     not the IP. Back off first, rotate only if backing off doesn't help.
  2. A connection error is not the same as a bad response. Retrying a 500 with

quantumproxies

"""
Sticky proxy sessions: keeping the same exit IP across a multi-step flow.

The problem this solves: you log in through proxy exit A, then the next request
goes out of exit B, and the site logs you straight back out. Anything with state
— login, cart, checkout, paginated search behind a session cookie — breaks the
moment the IP changes mid-flow.

The fix is a *sticky session*: a token in the proxy username that pins your
traffic to one exit for a while. Two things people get wrong:

quantumproxies

"""
Playwright behind a proxy — including the leak checks nobody runs until it's too late.

Setting `proxy=` in Playwright is three lines. What bites you afterwards:

  1. **WebRTC leaks your real IP.** Your HTTP traffic goes through the proxy and
     the site still sees your home address via a STUN candidate. Chromium needs
     an explicit policy flag; there is no Playwright option for it.
  2. **The timezone and locale don't follow the exit.** A "US residential" IP
     reporting Europe/Rome and it-IT is a louder signal than the IP itself.

quantumproxies

curl + proxy: the cheatsheet I wish I'd had

Every flag here is one I've had to look up under pressure. Ordered by how often that happened.

The basics

# HTTP proxy with auth
curl -x http://USER:PASS@gate.example.com:8080 https://api.ipify.org

quantumproxies

"""
Fix 403 Forbidden in web scraping — the escalation ladder that actually works.

A 403 is the server saying "I know what you are". Escalate in this order and
stop at the first rung that works — each rung costs more than the last:

  1. Send real browser headers (UA alone is not enough — order matters less
     than presence: Accept, Accept-Language, Sec-Fetch-*).
  2. Switch from datacenter to residential exits — most 403 walls are just
     IP-reputation lists.

quantumproxies

"""
Fix 429 Too Many Requests — backoff that respects Retry-After, then rotate.

Two mistakes cause most 429 loops:

  1. Ignoring the Retry-After header. If the server tells you when to come
     back, guessing an exponential delay instead is self-harm.
  2. Rotating the IP on the FIRST 429. Many rate limits key on your API key,
     account or fingerprint — rotating burns pool bandwidth for nothing.
     Rotate only when backoff on the same exit fails twice.