Skip to content

Instantly share code, notes, and snippets.

@robcowie

robcowie/bigquery_notes.md

Last active June 17, 2019 08:17
Show Gist options
  • Save robcowie/13fdbd3498c83d31f0a7792ad586b7c3 to your computer and use it in GitHub Desktop.
Save robcowie/13fdbd3498c83d31f0a7792ad586b7c3 to your computer and use it in GitHub Desktop.
Download ZIP
Biquery Notes
Raw
bigquery_notes.md

Biqquery Notes

Require a partition filter on an existing table

bq update --require_partition_filter --time_partitioning_field ts -t page_impressions.raw

Copy a table

bq cp source_table destination_table
bq cp --location EU source_table destination_table

Export a table to GCS

bq extract project-id:dataset.table 'gs://skim-dp-tmp/dataset/table/*'

The * is required as the extract writes multiple files.

Loading data

bq load --source_format AVRO page_impressions.raw gs://skim-dp-data/pi_export/000000000000
bq load --source_format AVRO --time_partitioning_field ts --time_partitioning_type DAY page_impressions.raw gs://skim-dp-data/pi_export/000000000001
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment