dvu4/Advanced Data Engineering with Databricks notes-Week 3.md

Last active February 25, 2026 01:30

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/dvu4/ba4e73e4738861898da78e76250c79ef.js"></script>
Save dvu4/ba4e73e4738861898da78e76250c79ef to your computer and use it in GitHub Desktop.

Raw

ai_parse_document

@pandas_udf decorator

.cache() - MEM .persist() - MEM/DISK

"_metadata.file_name"

CLUSTER BY AUTO

%sql SELECT LABUSER 3434141241

Select an option