Sometimes you need to troubleshoot and inspect the details of jobs (load, query etc.) in BigQuery. Inspecting the job history in the BigQuery web UI will only show the jobs that you have run. This is also true when you run run bq ls -j
on the command line.
But, what if you need to get all jobs that have been run? An example would be auotmated jobs run by service accounts. A quick tip is to use the --all
flag:
-a,--[no]all: Show all results. For jobs, will show jobs from all users.
For datasets, will list hidden datasets. For transfer configs and runs, this flag is redundant and not necessary.
bq ls -j --all
(this tip originated from a question on Stack Overflow: https://stackoverflow.com/questions/47583485/bigquery-history-of-jobs-submitted-through-python-api)