Skip to content

Instantly share code, notes, and snippets.

@bsidhom
Created October 21, 2022 21:47
Show Gist options
  • Save bsidhom/27d9e45b232e3eaf3c642d98f331d58c to your computer and use it in GitHub Desktop.
Save bsidhom/27d9e45b232e3eaf3c642d98f331d58c to your computer and use it in GitHub Desktop.
Extract Spark eventlog schema from a duckdb shell
create table j as select json(column0) as j from read_csv_auto('eventlog.json', delim='\0');
.mode list
.header off
.once 'schema.json'
select json_group_object(event, structure) from (select j->>'Event' as event, json_group_structure(j) as structure from j group by event);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment