Created
September 24, 2013 15:53
-
-
Save rdhyee/6686948 to your computer and use it in GitHub Desktop.
A proxying interface to the Evernote Python SDK as one approach to dealing with rate limiting: simple retry after suggested wait.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"metadata": { | |
"name": "" | |
}, | |
"nbformat": 3, | |
"nbformat_minor": 0, | |
"worksheets": [ | |
{ | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": {}, | |
"source": [ | |
"My notes for background to problem -- implementing rate limiting into my use of the Evernote API ( http://dev.evernote.com/doc/articles/rate_limits.php )\n", | |
"\n", | |
"https://www.evernote.com/shard/s1/sh/5327734c-fdfd-4d1c-9af4-8a32ad7a2e51/2d234b29342f3b8c8c4ec3d180c8861d\n", | |
"\n", | |
"Basic approach to explore:\n", | |
"\n", | |
"* proxying the existing EvernoteClient object or client\n", | |
"* learning from Python decorators about how to do a retry\n", | |
"* down the road, look at using `APScheduler` or `celery` to do more sophisticated handling of asynchronous retries.\n", | |
"\n", | |
"I was considering using a class decorator: http://my.safaribooksonline.com/book/programming/python/9780768687040/classes-and-object-oriented-programming/ch07lev1sec17" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"collapsed": false, | |
"input": [ | |
"import evernote\n", | |
"\n", | |
"from time import sleep\n", | |
"from evernote.edam.error.ttypes import (EDAMSystemException, EDAMErrorCode)\n", | |
"\n", | |
"def evernote_rate_limit(f):\n", | |
" def f2(*args, **kwargs):\n", | |
" try:\n", | |
" return f(*args, **kwargs)\n", | |
" except EDAMSystemException, e:\n", | |
" if e.errorCode == EDAMErrorCode.RATE_LIMIT_REACHED:\n", | |
" sleep(e.rateLimitDuration)\n", | |
" return f(*args, **kwargs)\n", | |
" \n", | |
" return f2\n", | |
"\n", | |
"\n", | |
"class RateLimitingEvernoteProxy(object):\n", | |
" __slots__ = [\"_obj\"]\n", | |
" def __init__(self, obj):\n", | |
" object.__setattr__(self, \"_obj\", obj)\n", | |
" \n", | |
" def __getattribute__(self, name):\n", | |
" return evernote_rate_limit(getattr(object.__getattribute__(self, \"_obj\"), name))" | |
], | |
"language": "python", | |
"metadata": {}, | |
"outputs": [], | |
"prompt_number": 1 | |
}, | |
{ | |
"cell_type": "code", | |
"collapsed": false, | |
"input": [ | |
"# settings holds the devToken/authToken that can be used to access Evernote account\n", | |
"#http://dev.evernote.com/doc/articles/authentication.php#devtoken\n", | |
"# settings.authToken\n", | |
"\n", | |
"import settings\n", | |
"\n", | |
"\n", | |
"from evernote.api.client import EvernoteClient\n", | |
"\n", | |
"dev_token = settings.authToken\n", | |
"\n", | |
"client = RateLimitingEvernoteProxy(EvernoteClient(token=dev_token, sandbox=False))\n", | |
"\n", | |
"userStore = client.get_user_store()\n", | |
"user = userStore.getUser()\n", | |
"print user.username" | |
], | |
"language": "python", | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"output_type": "stream", | |
"stream": "stdout", | |
"text": [ | |
"rdhyee\n" | |
] | |
} | |
], | |
"prompt_number": 2 | |
}, | |
{ | |
"cell_type": "code", | |
"collapsed": false, | |
"input": [ | |
"type(client)" | |
], | |
"language": "python", | |
"metadata": {}, | |
"outputs": [ | |
{ | |
"metadata": {}, | |
"output_type": "pyout", | |
"prompt_number": 3, | |
"text": [ | |
"__main__.RateLimitingEvernoteProxy" | |
] | |
} | |
], | |
"prompt_number": 3 | |
} | |
], | |
"metadata": {} | |
} | |
] | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment