Skip to content

Instantly share code, notes, and snippets.

View dataders's full-sized avatar

Anders dataders

View GitHub Profile
gource \
--s 0.1 \
--auto-skip-seconds 0.05 \
--key \
--title "mart_core: 2021" \
--highlight-users \
--max-file-lag 0.1 \
--font-scale 1.5 \
--filename-time 2 \
-1920x1080 -o - \
gource \
--s 0.1 \
--auto-skip-seconds 0.05 \
--key \
--title "dbt: the 'beta' era" \
--highlight-users \
--max-file-lag 0.1 \
--caption-file caption_versions.psv \
--caption-duration 5 \
--caption-size 80 \
@dataders
dataders / extensions.json
Last active February 24, 2022 05:47
.vscode folder for getting started with dbt in VSCode
{
"recommendations": [
"ms-python.python",
"mechatroner.rainbow-csv",
"samuelcolvin.jinjahtml",
"bastienboutonnet.vscode-dbt",
"amodio.find-related"
]
}
@dataders
dataders / azureml_OutputFileDatasetConfig.md
Created September 15, 2021 20:49
Azure ML OutputFileDatasetConfig rant

originally for this SO question before I realized they were hitting a bug

TL;DR

the .read_json_lines_files() method is part of the Dataset.Tabular class and is used for creating Datasets.

OutputFileDatasetConfig is for capturing data coming out of a PipelineStep and passing it to downstream step (or, optionally register

Deep Dive

@dataders
dataders / README.md
Created August 29, 2021 19:15
Stopping Junk Mail

swanderz's War on Junk Mail

1: Blanket Unsubscribes

Make sure everyone in your household has signed up for these two things. They'll prolly take a few months to kick in.

DMA Choice Pay $2 for 10 years of opt-outs

<#
.SYNOPSIS
.DESCRIPTION
This script will configure a DBT setup
.EXAMPLE
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@dataders
dataders / get_metrics.py
Last active February 11, 2021 16:29
fetching best model from HyperDriveStep
import argparse
import os
from pprint import pprint
import pandas as pd
from azureml.core import Run
def df2csv(df, dir, filename, **kwargs):
path = os.path.join(dir, filename)
print("saving {} to {}".format(filename, dir))
@dataders
dataders / pipeline_run_id.txt
Last active April 2, 2019 22:21
hypodrive metric output debug
http:///pipelines/subscriptions/ff2e23ae-7d7c-4cbd-99b8-116bb94dca6e/resourceGroups/RG-ITSMLTeam-Dev/providers/Microsoft.MachineLearningServices/workspaces/avadevitsmlsvc/experiment/undefined/run/115f70d3-91f7-4052-bbb1-1aef0da3c2e3