Skip to content

Instantly share code, notes, and snippets.

@cowboy-bebug
Created October 23, 2018 20:57
Show Gist options
  • Save cowboy-bebug/23bd56c4d823b338a345adb7001f4769 to your computer and use it in GitHub Desktop.
Save cowboy-bebug/23bd56c4d823b338a345adb7001f4769 to your computer and use it in GitHub Desktop.
import boto3, re, asyncio
async def sync_one_to_s3(path):
source, target = path
bucket = re.sub('^s3://(.+?)/.+$', '\\1', target)
s3 = boto3.resource('s3')
b = s3.Bucket(bucket)
try:
b.upload_file(file, key)
except ClientError as e:
print(e)
else:
return(True)
async def sync_many_to_s3(paths):
# asyncio.gather is new in Python 3.7+
tasks = [asyncio.ensure_future(sync_one_to_s3(path) for path in paths)]
await asyncio.gather(*tasks)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment