Skip to content

Instantly share code, notes, and snippets.

@sminot

sminot/read_feather_from_s3.py

Created September 2, 2018 16:36
Show Gist options
  • Save sminot/a82ad01eb28bd2114bf34fd6c7d99f6e to your computer and use it in GitHub Desktop.
Save sminot/a82ad01eb28bd2114bf34fd6c7d99f6e to your computer and use it in GitHub Desktop.
Download ZIP
Read feather file directly from AWS S3
Raw
read_feather_from_s3.py
import io
import boto3
import pandas as pd
def read_feather_file_from_s3(s3_url):
assert s3_url.startswith("s3://")
bucket_name, key_name = s3_url[5:].split("/", 1)
s3 = boto3.client('s3')
retr = s3.get_object(Bucket=bucket_name, Key=key_name)
return pd.read_feather(io.BytesIO(retr['Body'].read()))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment