Skip to content

Instantly share code, notes, and snippets.

Here’s the previous solution revisited to make sure we can dynamically split a DBT model's execution into monthly batches using today’s date as the input.
### Scenario Recap:
- **Input today’s date**: Start the script from the current month.
- **Run for the next 12 months**: Split into 12 monthly batches.
- **Pass year and month to both the main model and the referenced model**.
- **Automatically handle date calculations**.
### Full Example Implementation:
SELECT
date_column,
CASE
WHEN EXTRACT(DAYOFWEEK FROM date_column) = 7 THEN DATE_ADD(date_column, INTERVAL 2 DAY) -- Saturday to Monday
WHEN EXTRACT(DAYOFWEEK FROM date_column) = 1 THEN DATE_ADD(date_column, INTERVAL 1 DAY) -- Sunday to Monday
ELSE date_column -- Keep the date as is if it's not Saturday or Sunday
END AS adjusted_date
FROM your_table;
import pandas as pd
from google.cloud import bigquery
# Set up BigQuery client
client = bigquery.Client()
# Function to create temporary table from CSV
def create_temp_table_from_csv(dataset_id, table_name, csv_path):
# Read CSV into pandas DataFrame
df = pd.read_csv(csv_path)
@ducnh1022
ducnh1022 / bqswd.sql
Last active July 25, 2024 15:44
BigQuery Scripting with Dynamic SQL
DECLARE project_id STRING DEFAULT 'your_project_id';
DECLARE dataset_id STRING DEFAULT 'your_dataset_id';
DECLARE table_name STRING DEFAULT 'your_table_name';
-- Step 1: Retrieve column details for relevant data types
WITH column_details AS (
SELECT
column_name,
data_type
FROM
def tail_binary_file(input_file, tag, output_file):
try:
with open(input_file, 'rb') as f:
# Read the entire file into memory
data = f.read()
# Find the position of the tag
tag_position = data.find(tag)
if tag_position == -1:
import json
def search_json(data, target_tag):
"""
Recursively search for a tag in a JSON object and return a list of values associated with that tag.
:param data: The JSON object (dictionary or list) to search in.
:param target_tag: The tag to search for.
:return: A list of values associated with the target tag.
"""
Manage Policies, Manage Metadata Ingestion, Create Domains, Manage Domains, Manage Glossaries
Exploring DAO2DAO Collaboration Mechanisms
Implementable Tools to Extend DAO Interaction Patterns in D2D Research Phase 2
BlockScience
BlockScience
Follow
Mar 15 · 23 min read
1. setup jq
2. pull cash-out file
3. setup bee node
4. setup geth
5. edit yaml file
# setup jq first
sudo apt -f install jq
# pull cash-out file
wget cashout.sh https://gist.githubusercontent.com/ralph-pichler/3b5ccd7a5c5cd0500e6428752b37e975/raw/b40510f1172b96c21d6d20558ca1e70d26d625c4/cashout.sh
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.