Garrett Mooney GarrettMooney

Schemas in QA but not in Prod

WITH qa AS (
    SELECT TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME
    FROM qa_edw_pstg.information_schema.tables
),
prod AS (
    SELECT TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME
    FROM prd_edw_pstg.information_schema.tables

Functional Programming in Python

https://kobzol.github.io/rust/python/2023/05/20/writing-python-like-its-rust.html for ideas on how to write python code like it's rust
https://blog.ezyang.com/2020/10/idiomatic-algebraic-data-types-in-python-with-dataclasses-and-union/ for ideas on using algebraic data types in Python

Preferences

duckdb when data processing can easily be expressed in sql
polars over pandas
simple functions over classes
inline dependencies in python scripts
uv and uvx for package management, python clis, scripts, and virtual environments

	# /// script
	# requires-python = ">=3.12"
	# dependencies = [
	# "duckdb",
	# "polars",
	# "pyarrow",
	# "pyathena",
	# "s3fs",
	# ]
	# ///

	# /// script
	# requires-python = ">=3.12"
	# dependencies = [
	# "boto3",
	# "sagemaker",
	# ]
	# ///
	import subprocess
	import time
	from time import gmtime, strftime

	import uuid

	import streamlit as st

	#########
	# Helpers
	#########


	def get_completion(question, session_id):

	#!/usr/bin/env bash

	# install rustup if it doesn't exist
	if ! command -v rustup &> /dev/null
	then
	curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \| sh -s -- -y
	fi

	# install uv if it doesn't exist
	if ! command -v uv &> /dev/null