Skip to content

Instantly share code, notes, and snippets.

@cnolanminich
cnolanminich / Doc.md
Created March 12, 2026 09:15
Merge dbt cloud dagster

Step 1 — Understand the Two Starting Repos

Existing dbt Cloud Repo

dbt-cloud-repo/                        # github.com/myorg/dbt-cloud-repo
├── .github/
│   └── workflows/
│       └── dbt_cloud_deploy.yml       # dbt Cloud CI/CD
SELECT
customer_id,
COUNT(*) AS order_count,
SUM(amount) AS total_spent
FROM orders
GROUP BY customer_id
@cnolanminich
cnolanminich / component.py
Created April 6, 2026 14:45
dbt cost insights with a component
from collections.abc import Iterator
import dagster as dg
from dagster_dbt import DbtCliResource, DbtProjectComponent
class DbtProjectWithInsightsComponent(DbtProjectComponent):
"""A DbtProjectComponent subclass that enables Dagster+ Insights.
Chains `.with_insights()` onto the dbt event iterator so that warehouse
@cnolanminich
cnolanminich / Versioning_assets.md
Last active April 8, 2026 00:36
Asset versioning

Dagster Asset Versioning & Staleness for ML Pipelines

1. code_version: Manually Set, Not Auto-Hashed

code_version is a manually assigned string on the @asset decorator or AssetSpec. Dagster does not automatically hash your function source code.

import dagster as dg

@dg.asset(code_version="1.0.0")
@cnolanminich
cnolanminich / code_server_sizing.md
Created April 9, 2026 13:00
Sizing Considerations for Dagster+ ECS non-isolated runs

Dagster+ ECS Sizing Guide: Non-Isolated Runs

Overview

With non-isolated runs, each run executes as a thread inside the long-running code server process. Memory and CPU are shared across all concurrent runs. This guide covers how to profile your runs, size your ECS tasks, configure Dagster's gRPC settings, and decide when to split into multiple replicas.


The Core Sizing Equation

@cnolanminich
cnolanminich / pin_saved_selections.py
Last active June 26, 2026 16:07
pin saved selections
"""CLI for managing Dagster+ saved selections (catalog views) and homepage pins.
A *saved selection* is a ``CatalogView`` — a named, filterable view of the
asset catalog (e.g. "all assets owned by alice", "assets tagged core_kpis").
A pinned saved selection appears on a user's Dagster+ homepage and in the
left-nav saved-selections list.
Besides asset selections, the homepage can also pin **jobs** and asset
groups. A pinned job is stored as a (public) ``CatalogView`` of type ``JOB``
and surfaces as a ``JobPinnableItem``.