Debasish Bose dbose

Design Document: Lightdash-Inspired Measure and Metric System for dbt-core

A unified metrics layer implementation leveraging dbt-core's meta properties and macro system

This design document presents a comprehensive approach to implementing a measure and metric system on dbt-core, inspired by Lightdash's architecture while working within dbt-core's constraints. The system uses YAML-based configurations through dbt's meta properties, combined with a macro-based SQL generation layer to create a powerful metrics framework.

Core Architecture Overview

The proposed system follows Lightdash's unified approach to metrics, where everything is considered a "metric" but distinguished by behavior: aggregate metrics perform SQL aggregations (sum, count, average), while non-aggregate metrics perform calculations on already-aggregated values. This distinction aligns perfectly with Lightdash's model while remaining implementable in dbt-core.

// ============================================================================= // COMPLETE DBT-DOCS METRICS INTEGRATION // ============================================================================= // This file contains all the code changes needed to integrate YAML-based // metrics from model configurations into dbt-docs natively. // // Implementation Order: // 1. Service Layer (project_service.js) // 2. Routing (app.js) // 3. Metric Controller & Template

	Debasish-Boses-MacBook-Pro:blog quantum19812001$ rails g hobo:migration
	/Users/quantum19812001/Projects/ror-test/blog/vendor/plugins/hobo_fields/lib/hobo_fields/model.rb:95:in `belongs_to': undefined method `foreign_key' for #<ActiveRecord::Reflection::AssociationReflection:0x1023ea488> (NoMethodError)
	from /Users/quantum19812001/Projects/ror-test/blog/app/models/order.rb:6
	from /Users/quantum19812001/.rvm/gems/ruby-1.8.7-p352@ror-test/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:454:in `load'
	from /Users/quantum19812001/.rvm/gems/ruby-1.8.7-p352@ror-test/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:454:in `load_file'
	from /Users/quantum19812001/.rvm/gems/ruby-1.8.7-p352@ror-test/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:596:in `new_constants_in'
	from /Users/quantum19812001/.rvm/gems/ruby-1.8.7-p352@ror-test/gems/activesupport-3.0.3/lib/active_support/dependencies.rb:453:in `load_file'
	from /Users/quantum19812001/.rvm/gems/ruby-1.8.7-p352@ror-test/gems/act

	# http://www.thisisthegreenroom.com/2011/installing-python-numpy-scipy-matplotlib-and-ipython-on-lion/#homebrew

	brew install python
	brew install gfortran

	# Error: Library not loaded:
	sudo ln -s /opt/X11/lib/libpng14.14.dylib /opt/local/lib/libpng14.14.dylib
	sudo ln -s /opt/X11/lib/libfreetype.6.dylib /opt/local/lib/libfreetype.6.dylib

	git clone https://github.com/numpy/numpy.git numpy

	df$CLASSWKR_f <- factor(df$CLASSWKR,
	levels=c(0,10,13,14,20,21,22,23,24,25,26,27,28,29,99),
	labels=c("NIU",
	"Self-employed",
	"Self-employed, not incorporated",
	"Self-employed, incorporated",
	"Works for wages or salary",
	"Wage/salary, private",
	"Private, for profit",
	"Private, nonprofit",

	const BTCMarkets = require('btc-markets')
	const async = require('async')
	const _ = require('lodash')
	const schedule = require('node-schedule')
	const winston = require('winston')
	const os = require('os')

	const log = winston.createLogger({
	transports: [
	new (winston.transports.File)({

	import pandas as pd
	import numpy as np
	import matplotlib.pyplot as plt
	from pandas_datareader import data as pdr
	from datetime import datetime
	import quandl

	# Define the time range for data collection
	start = datetime(2010, 1, 1)
	end = datetime(2024, 1, 1)

	const $ = require('jquery');
	const _ = require('underscore');

	const graphlib = require('graphlib');
	const selectorGraph = require('./selector_graph');
	const colorValidation = require('./validate_node_color');

	angular
	.module('dbt')
	.factory('graph', [