Skip to content

Instantly share code, notes, and snippets.

@tommydangerous

tommydangerous/mage_delta-lake_load_magic_energy.py

Last active May 9, 2023 23:30
Show Gist options
  • Save tommydangerous/98f14bcbef5d73aac7c284d3d0662cba to your computer and use it in GitHub Desktop.
Save tommydangerous/98f14bcbef5d73aac7c284d3d0662cba to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mage_delta-lake_load_magic_energy.py
from deltalake import DeltaTable
@data_loader
def load_data(*args, **kwargs):
dt = DeltaTable(
's3://mage-demo-public/magic-energy/1337',
storage_options={
'AWS_ACCESS_KEY_ID': '...',
'AWS_SECRET_ACCESS_KEY': '...',
},
)
return dt.to_pandas()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment