Skip to content

Instantly share code, notes, and snippets.

@barograf
Created November 4, 2022 09:37
Show Gist options
  • Save barograf/4e19bf7ceafed28d8426830fc460adfb to your computer and use it in GitHub Desktop.
Save barograf/4e19bf7ceafed28d8426830fc460adfb to your computer and use it in GitHub Desktop.
client = boto3.client("glue", region_name=args["REGION"])
paginator = client.get_paginator("get_tables")
page_iterator = paginator.paginate(DatabaseName=args["CATALOG"])
for page in page_iterator:
for table in page["TableList"]:
if not table_valid(table, args):
continue
[db, schema, tbl_name] = table["StorageDescriptor"]["Location"].split(".")
dest_path = f"{args['DEST']}/{tbl_name}"
print(f"Dumping table {tbl_name} to {dest_path}")
# dump code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment