Skip to content

Instantly share code, notes, and snippets.

@pcn
Created February 13, 2013 15:43
Show Gist options
  • Select an option

  • Save pcn/4945507 to your computer and use it in GitHub Desktop.

Select an option

Save pcn/4945507 to your computer and use it in GitHub Desktop.
Stacktrace from a long s3 list or an s3_get with paging handled. I am trying to figure out why s3 is getting me NoneType objects and why it's happening so low down in boto.
Traceback (most recent call last):
File "/home/pn/s3-describe-bucket", line 264, in <module>
main()data/pearsoncourseeventsdump_2013_01_17T17_40_40/62a0e4f7202d558f5222c0f147e8d682_ea33f882-f362-4ca4-a654-f96c41c4a374: 15712 kB
File "/home/pn/s3-describe-bucket", line 249, in main
write_output(generated_key_list, opts)
File "/home/pn/s3-describe-bucket", line 213, in write_output
for key in s3_generator:
File "/home/pn/s3-describe-bucket", line 167, in yield_keys_list
for k in src:
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/k/aws/util.py", line 56, in yield_aws_data
for item in (u for u in data_lookup(response)):
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/k/aws/util.py", line 56, in <genexpr>
for item in (u for u in data_lookup(response)):
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/s3/bucketlistresultset.py", line 30, in bucket_lister
delimiter=delimiter, headers=headers)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/s3/bucket.py", line 360, in get_all_keys
'', headers, **params)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/s3/bucket.py", line 317, in _get_all
query_args=s)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/s3/connection.py", line 470, in make_request
override_num_retries=override_num_retries)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/connection.py", line 913, in make_request
return self._mexe(http_request, sender, override_num_retries)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/connection.py", line 821, in _mexe
self._renew_credentials()
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/connection.py", line 881, in _renew_credentials
self._auth_handler.update_provider(self.provider)
File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/auth.py", line 127, in update_provider
super(HmacAuthV1Handler, self).update_provider(provider) File "/mnt/data/s3-costs/local/lib/python2.7/site-packages/boto/auth.py", line 82, in update_provider
self._hmac = hmac.new(self._provider.secret_key, digestmod=sha)
File "/usr/lib/python2.7/hmac.py", line 133, in new return HMAC(key, msg, digestmod)
File "/usr/lib/python2.7/hmac.py", line 68, in __init__
if len(key) > blocksize:
TypeError: object of type 'NoneType' has no len()
@pcn

pcn commented Feb 13, 2013

Copy link
Copy Markdown
Author

It's possible that this is related to the auth I'm using being an STS, and having it expire. I'll try doing this with an IAM next to see if that eliminates this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment