Skip to content

Instantly share code, notes, and snippets.

View Padigela's full-sized avatar

VardhanS Padigela

3 followers · 23 following
View GitHub Profile
@Padigela
Padigela / load_parquet_s3.py
Created January 3, 2021 08:15 — forked from asmaier/load_parquet_s3.py
Pyspark script for downloading a single parquet file from Amazon S3 via the s3a protocol. It also reads the credentials from the "~/.aws/credentials", so we don't need to hardcode them. See also https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html .
#
# Some constants
#
aws_profile = "your_profile"
aws_region = "your_region"
s3_bucket = "your_bucket"
#
# Reading environment variables from aws credential file
#
@Padigela
Padigela / README.md
Created March 19, 2019 14:06 — forked from leonardofed/README.md
A curated list of AWS resources to prepare for the AWS Certifications


A curated list of AWS resources to prepare for the AWS Certifications

A curated list of awesome AWS resources you need to prepare for the all 5 AWS Certifications. This gist will include: open source repos, blogs & blogposts, ebooks, PDF, whitepapers, video courses, free lecture, slides, sample test and many other resources.