Skip to content

Instantly share code, notes, and snippets.

@sminot

sminot/read_gzip_fasta_from_s3.py

Created July 13, 2018 19:37
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sminot/39c3b52d97279c6016f327a48381eca2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sminot/39c3b52d97279c6016f327a48381eca2 to your computer and use it in GitHub Desktop.
Download ZIP
Read a gzipped fasta from s3
Raw
read_gzip_fasta_from_s3.py
def read_fasta_from_s3(bucket_name, key_name, sep="\t"):
s3 = boto3.client('s3')
retr = s3.get_object(Bucket=bucket_name, Key=key_name)
bytestream = io.BytesIO(retr['Body'].read())
got_text = gzip.GzipFile(None, 'rb', fileobj=bytestream).read().decode('utf-8')
for header, seq in SimpleFastaParser(io.StringIO(got_text)):
yield header, seq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment