More succinct resilient session

resilientsession.py

from requests import Session
import time

class ResilientSession(Session):
    """
    This class is supposed to retry requests that return temporary errors.

    At this moment it supports: 502, 503, 504
    """

    def request(self, method, url, **kwargs):
        counter = 0

        while True:
            counter += 1

            r = super(ResilientSession, self).request(method, url, **kwargs)

            if r.status_code in [ 502, 503, 504 ]:
                delay = 10 * counter
                logging.warn("Got recoverable error [%s] from %s %s, retry #%s in %ss" % (r.status_code, method, url, counter, delay))
                time.sleep(delay)
                continue

            return r

if __name__ == '__main__':
    s = ResilientSession()
    s.get('http://httpstat.us/502')

In the Session class, all HTTP methods are channelled through the request method, so adding the retry there makes it work for all HTTP methods.

I have changed the behaviour slightly. Exceptions are not caught, so the controlling process chooses how to deal with them. Also the log message is slightly different (no HTTP in front of status_code).

I think you need to add the exit clause for the while loop... at least I had to do it to get it working:

...
                elif r.status_code==200:
                    break
                else:
                    raise ConnectionError

The last else is to cover for all other status codes

Argh, I have the return r in the wrong place. Thanks @sladkovm!