-
-
Save gleicon/2b8acb9f9c0f22753eaac227ff997b34 to your computer and use it in GitHub Desktop.
from boto3.session import Session | |
from botocore.client import Config | |
from botocore.handlers import set_list_objects_encoding_type_url | |
import boto3 | |
ACCESS_KEY = "xx" | |
SECRET_KEY = "yy" | |
boto3.set_stream_logger('') | |
session = Session(aws_access_key_id=ACCESS_KEY, | |
aws_secret_access_key=SECRET_KEY, | |
region_name="US-CENTRAL1") | |
session.events.unregister('before-parameter-build.s3.ListObjects', | |
set_list_objects_encoding_type_url) | |
s3 = session.resource('s3', endpoint_url='https://storage.googleapis.com', | |
config=Config(signature_version='s3v4')) | |
bucket = s3.Bucket('yourbucket') | |
for f in bucket.objects.all(): | |
print(f.key) |
If you are wondering how to generate the ACCESS KEY and SECRET KEY: https://cloud.google.com/storage/docs/authentication/managing-hmackeys
Has anyone managed to get this working with a ResponseContentDisposition
header? I'm receiving SignatureDoesNotMatch
errors
Edit: Check my comment under(https://gist.github.com/gleicon/2b8acb9f9c0f22753eaac227ff997b34?permalink_comment_id=4721658#gistcomment-4721658) for solution
Try enabling boto3 logging. I'm not sure Google Object Storage support all headers or the same syntax (see the AwsAccessKeyId issue above). Also check if the signature version still right for the type of storage you've created.
I managed to get it working. The comment under https://stackoverflow.com/a/21028609/9553927 helped:
The signed urls worked for me. Although, I've tried to call generate_url() with the parameter response_headers and the value response-content-disposition but I got malformed signed urls. So my solution has been to concatenate '&response-content-disposition=attachment%3B%20filename%3D"{}"'.format(file_name) to the signed url and it worked.
params = {
"Bucket": "xyz",
"Key": blob_name,
}
ten_minutes = 600 # seconds
url = self.s3_client.generate_presigned_url(
"get_object", Params=params, ExpiresIn=ten_minutes
).replace("AWSAccessKeyId", "GoogleAccessId")
url += '&response-content-disposition=attachment;filename="newFileName"'
Awesome, thanks for sharing !
Thanks for a minimum viable solution to the gcp interop issue. (Linking because I didn't understand your solution until reading the issue thread).