Skip to content

Instantly share code, notes, and snippets.

@paulnicholsen27
Created August 19, 2025 20:30
Show Gist options
  • Select an option

  • Save paulnicholsen27/47197d1bfa615d3b716206601b3c10fe to your computer and use it in GitHub Desktop.

Select an option

Save paulnicholsen27/47197d1bfa615d3b716206601b3c10fe to your computer and use it in GitHub Desktop.
import json
import csv
project_folder = "airflow_extractions/BTRIS_CB_Color_Coding_08192025_100349/"
path_to_action_steps = f"{project_folder}action_steps.json"
with open(path_to_action_steps, "r") as f:
data = json.load(f)
with open("lineage_steps.csv", "w", newline="") as csvfile:
writer = csv.writer(csvfile)
writer.writerow(["step_number", "script_path", "input_files", "output_files", "project_folder"])
for step_number, step_data in data["steps"].items():
script_path = project_folder + step_data.get("script_path", "")
input_files = [project_folder + file for file in step_data.get("input_files", [])]
input_files = " ".join(input_files)
output_files = [project_folder + file for file in step_data.get("output_files", [])]
output_files = " ".join(output_files)
writer.writerow([step_number, script_path, input_files, output_files, project_folder])
@paulnicholsen27
Copy link
Author

Update project folder (keep the trailing slash). Run the code, then copy and paste lineage_steps.csv to https://docs.google.com/spreadsheets/d/1_fJxmG1MR0gOXZ56YleeHGfHzroF6cAlQCeEDCs_5_w/edit?gid=131833823#gid=131833823 (Duplicate the first sheet, select A1, paste, then select "Split text to columns").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment