Skip to content

Instantly share code, notes, and snippets.

@owlwang
Forked from laixintao/decent_request.py
Created December 21, 2023 13:29
Show Gist options
  • Save owlwang/1530169692d3bac00808cd950bb9aea6 to your computer and use it in GitHub Desktop.
Save owlwang/1530169692d3bac00808cd950bb9aea6 to your computer and use it in GitHub Desktop.
Send HTTP requests using python-requests with timeout, tcp reuse(session) and retry.
from requests.adapters import HTTPAdapter, Retry
from requests import Session
retries = Retry(
total=5, backoff_factor=1, status_forcelist=[502, 503, 504]
)
session = Session() # reuse tcp connection
session.mount("http://", HTTPAdapter(max_retries=retries))
session.mount("https://", HTTPAdapter(max_retries=retries))
session.get("https://example.com", timeout=5) # seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment