Skip to content

Instantly share code, notes, and snippets.

@GDBSD
Created December 12, 2022 18:54
Show Gist options
  • Select an option

  • Save GDBSD/bfda7c7d45bf81683ffe5f6b2468c096 to your computer and use it in GitHub Desktop.

Select an option

Save GDBSD/bfda7c7d45bf81683ffe5f6b2468c096 to your computer and use it in GitHub Desktop.
Compare BigQuery table schemas
def compare_table_schemas(client, project: str, dataset: str,
table_a: str, table_b: str) -> bool:
"""Compare the schemas of two BigQuery tables. Useful for instance
to confirm that there hasn't been a drift in the schemas for the
production and development tables.
:param client: BigQuery client object
:param project: string - GCP project ID
:param dataset: string - dataset name
:param table_a: string - table name
:param table_b: string - table name
:return: bool - whether the schemas are the same
"""
query_a = f"""
WITH schema AS (
SELECT * FROM {project}.{dataset}.INFORMATION_SCHEMA.COLUMNS
WHERE table_name = f'{table_a}'
SELECT column_name, data_type FROM schema
"""
query_b = f"""
WITH schema AS (
SELECT * FROM {project}.{dataset}.INFORMATION_SCHEMA.COLUMNS
WHERE table_name = f'{table_b}'
SELECT column_name, data_type FROM schema
"""
df_a = client.query(query_a).to_dataframe()
df_b = client.query(query_b).to_dataframe()
schema_1 = df_a.to_dict('records')
schema_2 = df_b.to_dict('records')
return schema_1 == schema_2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment