Skip to content

Instantly share code, notes, and snippets.

View sbailliez's full-sized avatar

Stephane B. sbailliez

View GitHub Profile
@sbailliez
sbailliez / compress.sql
Created June 1, 2024 19:11
dbt macro (redshift only) to compress a specified set of columns
{#
Compresses the columns of a table to the specified encodings. Unlike the compress_table macro in dbt-redshift,
you have to manually specify the encoding of the columns you want to compress, it does not run and use
the result of `ANALYZE compression <table>` which would is extremely slow to run every time
on a large table, especially when there is nothing to do and the table is already compressed.
If the specified column does not exist or if the specified encoding is identical to the current encoding,
the column is skipped.
ALTER statements are generated for each column where encoding needs to be changed.
@sbailliez
sbailliez / table_columns.sql
Created June 17, 2024 22:06
Redshift - Returns all columns datatype, encoding, nullable and default value. Useful for maintenance
WITH table_columns AS (
SELECT
c.relowner as tableowner
,n.nspname AS schema_name
,c.relname AS table_name
,a.attnum AS ordinal
,QUOTE_IDENT(a.attname) AS column_name
,CASE WHEN STRPOS(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER VARYING') > 0
THEN REPLACE(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER VARYING', 'VARCHAR')
WHEN STRPOS(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER') > 0
@sbailliez
sbailliez / f_add_business_day.sql
Created July 3, 2024 19:16
Equivalent to WORKDAY function in Excel/Google Sheet (does not support holidays, only handles weekends as saturday/sunday)
CREATE OR REPLACE FUNCTION f_add_business_day(start_date date, num_days integer)
RETURNS date AS
$BODY$
SELECT COALESCE(
(
SELECT workdays.date
FROM (
SELECT calendar.date::date,
row_number() OVER (ORDER BY CASE WHEN num_days = abs(num_days) THEN calendar.date END, calendar.date DESC) as elapsed_days
FROM generate_series(
@sbailliez
sbailliez / install-libvips.sh
Last active January 21, 2025 02:39
Attempt at compiling libvips on amazon linux 2023
#!/bin/bash
#
# Compile libvips and some of its needed dependencies to support jpeg,avif,webp,png and gif processing
# This has not been tested extensively, so use at your own risk.
#
# This takes ~18min to compile on a t3.small, ~10min on a m6a.large
#
# TODO cleanup the flags everywhere to be consistent
#
set -ex