Skip to content

Instantly share code, notes, and snippets.

View spark2ignite's full-sized avatar

Vadim Solovey spark2ignite

View GitHub Profile
@sayle-doit
sayle-doit / bq_job_editions_cost_comparison_with_autoscaler.sql
Last active May 5, 2026 20:00
Compare BigQuery job costs when running a job on either BigQuery Editions with the autoscaler or on-demand with both new and old pricing models.
/*
* This query will look at the past 30 days of job history to analyze it for costs under
* BigQuery Editions while utilizing the new autoscaling feature that was introduced.
* It does this for those using both PAYG (Pay As You Go) and commitment models.
* It will also compare this versus running the query with the on-demand model.
*
* Note that this query utilizes some math modeling behaviors that the BigQuery
* autoscaler uses. Namely these are the up to 10 seconds "slot scale up time,"
* the minimum of 60 seconds "slot scale down time," and the behavior that the
* autoscaler scales up and down in factors of 100 slots for each job.
package main
import (
"context"
"fmt"
"os"
"cloud.google.com/go/bigquery"
)
@igrigorik
igrigorik / json-bq-schema-generator.rb
Last active November 3, 2022 17:34
BigQuery JSON schema generator
require 'open-uri'
require 'zlib'
require 'yajl'
# References
# - https://developers.google.com/bigquery/preparing-data-for-bigquery#dataformats
# - https://developers.google.com/bigquery/docs/data#nested
#
def type(t)