Skip to content

Instantly share code, notes, and snippets.

View sspaeti's full-sized avatar
🔗
maintaining awareness of new emerging open-source #dataengineering technologies

Simon Späti sspaeti

🔗
maintaining awareness of new emerging open-source #dataengineering technologies
View GitHub Profile
@sspaeti
sspaeti / query-stackoverflow-survey-motherduck.sql
Last active February 3, 2025 09:25
This queries a shared DuckDB database on MotherDuck with StackOverflow Survey
--connect to MotherDuck databases
-- configure MOTHERDUCK_TOKEN and run DuckDB that connects to all shared Databases with
-- `duckdb "md:"`
-- and then run this query:
WITH language_counts AS (
SELECT
language,
COUNT(*) AS count
FROM (
SELECT UNNEST(STRING_SPLIT(LanguageHaveWorkedWith, ';')) AS language
@michael-simons
michael-simons / copy_to_mermaid.sql
Created November 21, 2024 11:21
Quickly create Mermaid ER-Diagrams for DuckDB Tables
-- Inspired by https://gist.github.com/Bilbottom/e1d3d677d2479e0602132327703ff15d
-- Fixed datatypes that don't render in mermaid (structs, decimal etc.)
-- Uses 1:n as base, cardinalities are hard to derive
-- Looks for column comments
-- Avoids regex.
COPY (
WITH hlp AS (
SELECT referenced_table, c.table_name,
trim(string_agg(d.comment, ' ')) AS comment,
@sspaeti
sspaeti / query_read_bsky_feed.duckdb.sql
Created October 30, 2024 09:13
Reading bsky posts with DuckDB example.
-- Query the API directly and flatten the nested JSON structure
WITH raw_data AS (
SELECT * FROM read_json_auto('https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=did:plc:edglm4muiyzty2snc55ysuqx&limit=10')
),
unnested_feed AS (
SELECT unnest(feed) as post_data FROM raw_data
)
SELECT
-- Post basics
post_data.post.uri as post_uri,
@mdornseif
mdornseif / mt940toOFX.py
Last active July 14, 2020 13:42
Convert SWIFT MT940 data to OFX.
#!/usr/bin/env python
# encoding: utf-8
"""
mt940toOFX.py - Dieses Progrtamm liesst MT940 SWIFT Kontostände und konvertiert sie in OFX.
OFX wurde mit xero.com getestet.
Created by Maximillian Dornseif on 2010-06-05.
Copyright (c) 2010, 2013, 2014 HUDORA. All rights reserved.
"""