Skip to content

Instantly share code, notes, and snippets.

@rdhyee

rdhyee/bug_report.ipynb

Last active July 19, 2016 22:22
Show Gist options
  • Save rdhyee/404573708a805861edb5 to your computer and use it in GitHub Desktop.
Save rdhyee/404573708a805861edb5 to your computer and use it in GitHub Desktop.
Download ZIP
A demonstration of a bug in pyzotero reported at https://github.com/urschrei/pyzotero/issues/52
Display the source blob
Display the rendered blob
Raw
bug_report.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# bug report summary\n",
"\n",
"I'm running into a `KeyError: u'self'` on https://github.com/urschrei/pyzotero/blob/v1.1.14/pyzotero/zotero.py#L616 (for pyzotero v 1.1.14). I *think* the problem is related to ['self' entry missing from links · Issue #43 · urschrei/pyzotero](https://github.com/urschrei/pyzotero/issues/43). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# setup details\n",
"\n",
"https://github.com/urschrei/pyzotero now deals with [dev:web_api:v3:start [Zotero Documentation]](https://www.zotero.org/support/dev/web_api/v3/start). Currently using https://github.com/urschrei/pyzotero/tree/v1.1.14 in my `myenv` conda environment\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# working code"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"blogPost | M9STMK38 | Endnote vs .... well, everything else \n",
"book | I657XCB8 | Big data, little data, no data: scholarship in the networked world \n",
"blogPost | TAXMWKBZ | Setting up QGIS 2.8 on MacOS X 10.10 Yosemite | Konstantin Greger \n",
"webpage | N3D2GDB7 | Disqus \n",
"journalArticle | AQ92GASF | Machine learning: Trends, perspectives, and prospects \n"
]
}
],
"source": [
"# https://github.com/urschrei/pyzotero#quickstart\n",
"\n",
"from itertools import islice\n",
"\n",
"# user_id, user_key from https://www.zotero.org/settings/keys stored in zoteroconf.py\n",
"from zoteroconf import user_id, user_key\n",
"\n",
"# from itertools import islice\n",
"from pyzotero import zotero\n",
"# getting my own personal library\n",
"zot = zotero.Zotero(user_id, 'user', user_key)\n",
"items = zot.top(limit=5)\n",
"\n",
"# we've retrieved the latest five top-level items in our library\n",
"# we can print each item's item type and ID\n",
"for item in items:\n",
" print('{} | {} | {} '.format(item['data']['itemType'], item['data']['key'], item['data'].get('title')))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## zot.follow works"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[{u'data': {u'abstractNote': u'',\n",
" u'accessDate': u'2015-09-05T11:57:45Z',\n",
" u'blogTitle': u'',\n",
" u'collections': [],\n",
" u'creators': [],\n",
" u'date': u'',\n",
" u'dateAdded': u'2015-09-05T11:57:45Z',\n",
" u'dateModified': u'2015-09-05T11:57:45Z',\n",
" u'extra': u'',\n",
" u'itemType': u'blogPost',\n",
" u'key': u'TAXMWKBZ',\n",
" u'language': u'',\n",
" u'relations': {},\n",
" u'rights': u'',\n",
" u'shortTitle': u'',\n",
" u'tags': [],\n",
" u'title': u'Setting up QGIS 2.8 on MacOS X 10.10 Yosemite | Konstantin Greger',\n",
" u'url': u'http://www.konstantingreger.net/setting-up-qgis-2-8-on-macos-x-10-10-yosemite/',\n",
" u'version': 1140,\n",
" u'websiteType': u''},\n",
" u'key': u'TAXMWKBZ',\n",
" u'library': {u'id': 71,\n",
" u'links': {u'alternate': {u'href': u'https://www.zotero.org/rdhyee',\n",
" u'type': u'text/html'}},\n",
" u'name': u'rdhyee',\n",
" u'type': u'user'},\n",
" u'links': {u'alternate': {u'href': u'https://www.zotero.org/rdhyee/items/TAXMWKBZ',\n",
" u'type': u'text/html'},\n",
" u'self': {u'href': u'https://api.zotero.org/users/71/items/TAXMWKBZ',\n",
" u'type': u'application/json'}},\n",
" u'meta': {u'numChildren': 1},\n",
" u'version': 1140}]"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# zot.follow works\n",
"\n",
"first_item = zot.top(limit=1)\n",
"# now we can start retrieving subsequent items\n",
"next_item = zot.follow()\n",
"third_item = zot.follow()\n",
"\n",
"third_item"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## zot.makeiter fails for me"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"ename": "KeyError",
"evalue": "u'self'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-3-1ef1034902c9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mgen\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mzot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmakeiter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mzot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlimit\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/Users/raymondyee/anaconda/envs/myenv/lib/python2.7/site-packages/pyzotero/zotero.pyc\u001b[0m in \u001b[0;36mmakeiter\u001b[0;34m(self, func)\u001b[0m\n\u001b[1;32m 614\u001b[0m \u001b[0m_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 615\u001b[0m \u001b[0;31m# reset the link. This results in an extra API call, yes\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 616\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlinks\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'next'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlinks\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'self'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 617\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miterfollow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 618\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mKeyError\u001b[0m: u'self'"
]
}
],
"source": [
"gen = zot.makeiter(zot.top(limit=1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# creating zot.all_items to be a lazy generator of all the items\n",
"\n",
"I've not used `zot.everything` since it's is not a lazy function: https://github.com/urschrei/pyzotero/blob/v1.1.14/pyzotero/zotero.py#L619"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## using itertools.chain and zot.iterfollow to create all_items"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"blogPost | M9STMK38 | Endnote vs .... well, everything else \n",
"book | I657XCB8 | Big data, little data, no data: scholarship in the networked world \n",
"blogPost | TAXMWKBZ | Setting up QGIS 2.8 on MacOS X 10.10 Yosemite | Konstantin Greger \n",
"webpage | N3D2GDB7 | Disqus \n",
"journalArticle | AQ92GASF | Machine learning: Trends, perspectives, and prospects \n",
"webpage | MUBN7RKN | CenterForOpenScience/osf.io \n",
"webpage | F9QX9SKT | What Makes Houston the Next Great American City? | Travel | Smithsonian \n",
"webpage | T38MHX9Q | rdhyee/census \n",
"webpage | WV9S2IPH | sunlightlabs/census \n",
"journalArticle | TVAUW8JF | The Tao of open science for ecology \n"
]
}
],
"source": [
"# create an all_items method using zot.iterflow to build a lazy generator for the items\n",
"\n",
"from itertools import chain\n",
"zot = zotero.Zotero(user_id, 'user', user_key)\n",
"\n",
"def all_items(zot, *args, **kw):\n",
" for page in chain([zot.top(*args, **kw)], zot.iterfollow()):\n",
" for item in page:\n",
" yield item\n",
"\n",
"for item in islice(all_items(zot, limit=50),10):\n",
" print('{} | {} | {} '.format(item['data']['itemType'], item['data']['key'], item['data'].get('title')))\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"blogPost | M9STMK38 | Endnote vs .... well, everything else \n",
"book | I657XCB8 | Big data, little data, no data: scholarship in the networked world \n",
"blogPost | TAXMWKBZ | Setting up QGIS 2.8 on MacOS X 10.10 Yosemite | Konstantin Greger \n",
"webpage | N3D2GDB7 | Disqus \n",
"journalArticle | AQ92GASF | Machine learning: Trends, perspectives, and prospects \n",
"webpage | MUBN7RKN | CenterForOpenScience/osf.io \n",
"webpage | F9QX9SKT | What Makes Houston the Next Great American City? | Travel | Smithsonian \n",
"webpage | T38MHX9Q | rdhyee/census \n",
"webpage | WV9S2IPH | sunlightlabs/census \n",
"journalArticle | TVAUW8JF | The Tao of open science for ecology \n"
]
}
],
"source": [
"# one can monkey patch the zot object\n",
"# http://stackoverflow.com/a/2982/7782\n",
"\n",
"from itertools import chain\n",
"import types\n",
"\n",
"def all_items_bound(self, *args, **kw):\n",
" for page in chain([self.top(*args, **kw)], self.iterfollow()):\n",
" for item in page:\n",
" yield item\n",
"\n",
"zot = zotero.Zotero(user_id, 'user', user_key)\n",
"zot.all_items = types.MethodType( all_items_bound, zot )\n",
"\n",
"for item in islice(zot.all_items(limit=50),10):\n",
" print('{} | {} | {} '.format(item['data']['itemType'], item['data']['key'], item['data'].get('title')))\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment