Skip to content

Instantly share code, notes, and snippets.

@dazzag24
Last active March 2, 2020 13:00
Show Gist options
  • Save dazzag24/3bb76399f33c5d030a7d3d7cb52021ef to your computer and use it in GitHub Desktop.
Save dazzag24/3bb76399f33c5d030a7d3d7cb52021ef to your computer and use it in GitHub Desktop.
import requests
from requests.packages.urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter
s = requests.Session()
# https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html
# backoff_factor (float) –
# A backoff factor to apply between attempts after the second try (most errors are resolved immediately
# by a second try without a delay). urllib3 will sleep for:
#
# {backoff factor} * (2 ** ({number of total retries} - 1))
# seconds. If the backoff_factor is 0.1, then sleep() will sleep for [0.0s, 0.2s, 0.4s, …] between retries.
# It will never be longer than Retry.BACKOFF_MAX.
#
# By default, backoff is disabled (set to 0).
#
# See https://findwork.dev/blog/advanced-usage-python-requests-timeouts-retries-hooks/
# for a good writeup of Python requests retries and timeouts
#
retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])
s.mount('https://', HTTPAdapter(max_retries=retries))
r = s.get('https://www.example.com')
if r.status_code == 200:
print("Success: Downloaded data")
else:
print("Error: failed to download data")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment