Skip to content

Instantly share code, notes, and snippets.

@jeffwidman
Created February 3, 2016 20:25
Show Gist options
  • Save jeffwidman/a38865b34b9f9f292dd1 to your computer and use it in GitHub Desktop.
Save jeffwidman/a38865b34b9f9f292dd1 to your computer and use it in GitHub Desktop.
github import api returning issue import status url that 404s
# The problem is that from time to time the GitHub issue import API returns a status url for the issue import
# that when I check that status url, I get a 404 response.
# Below is an example of a request/response pair that came back with a status url that 404'd:
# All of these requests use the header = {'Accept': 'application/vnd.github.golden-comet-preview+json'}
# Also I authenticate via my username/personal access token.
# Neither the header nor the auth should pose a problem because it successfuly imports dozens of issues before
# hitting any issues.
# The issue data that's POSTed to the issue import endpoint... this is in python format that's passed directly
# to the python requests library:
# Original issue at https://bitbucket.org/pypa/setuptools/issue/9
{'comments': [{'body': '*Original comment by* **Jason R. Coombs (Bitbucket: '
'[jaraco](http://bitbucket.org/jaraco), GitHub: '
'[jaraco](http://github.com/jaraco))**:\n'
'\n'
'----------------------------------------\n'
'\n'
"I'm inclined to say this shouldn't be fixed. The "
'updated code now prefers a secure download technique, '
'leveraging system downloaders. These measures should '
'be suitable to protect against most vectors that would '
'employ download or extraction attacks. I say most '
"because I can't prove there aren't viable vectors out "
'there.\n'
'\n'
"In other words, I believe we've put reasonable "
'safeguards in place to ensure the content is trusted.\n'
'\n'
'That said, if you believe the potential vulnerability '
'is worth the investment, please do provide a pull '
'request.\n',
'created_at': '2014-02-09T20:07:59Z'}],
'issue': {'body': 'Originally reported by: **Christian Heimes (Bitbucket: '
'[tiran](http://bitbucket.org/tiran), GitHub: '
'[tiran](http://github.com/tiran))**\n'
'\n'
'----------------------------------------\n'
'\n'
"ez_setup.download_setuptools() doesn't limit the amount of "
'data that is downloaded with urllib. An attacker can forge '
'a HTTP response with a large or even infinite file (e.g. '
'netcat < /dev/zero). This can consume lots of memory or '
'occupy lots of disk space on /tmp. src.read() should be '
'limited to a sane value (e.g. 2-5 MB) and raise an error '
'if more data is downloaded.\n'
'\n'
"ez_setup._extractall() should be limited, too. It's open "
'to zip decompression bomb vulnerability.\n'
'\n'
'----------------------------------------\n'
'- Bitbucket: '
'https://bitbucket.org/pypa/setuptools/issue/9\n',
'closed': True,
'created_at': '2013-06-03T14:42:19Z',
'labels': ['major', 'bug'],
'title': 'ez_setup.py download and extraction flooding '
'vulnerability'}}
# The response from the issue import API endpoint--it returns HTTP status code 202:
{'created_at': '2016-02-03T12:04:42-08:00',
'id': 471762,
'import_issues_url': 'https://api.github.com/repos/jeffwidman/t/import/issues',
'repository_url': 'https://api.github.com/repos/jeffwidman/t',
'status': 'pending',
'updated_at': '2016-02-03T12:04:42-08:00',
'url': 'https://api.github.com/repos/jeffwidman/t/import/issues/471762'}
# Checking the status url https://api.github.com/repos/jeffwidman/t/import/issues/471762 returns a HTTP 404
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment