dupuy

Markdown and reStructuredText

GitHub supports several lightweight markup languages for documentation; the most popular ones (generally, not just at GitHub) are Markdown and reStructuredText. Markdown is sometimes considered easier to use, and is often preferred when the purpose is simply to generate HTML. On the other hand, reStructuredText is more extensible and powerful, with native support (not just embedded HTML) for tables, as well as things like automatic generation of tables of contents.

7rin0

# Root.
$ docker exec -u 0 i -t {container_id/image_name} bash

or

# Default container's user.
$ docker exec i -t {container_id/image_name} bash

abelsonlive

profile_dir: '~/.dbt'                    # the location of dbt configurations
target: prod                             # the dbt profile to use for connection
source_schema: fivetran_app              # the tables' source schema (where fivetran imports your data to)
materialized: view                       # how to materialize these views
target_schema: app                       # the schema to materialize the base models under (where users will access the data from)
table_prefix: app                        # a prefix to prepend to each base model/view
directory: models/base/                  # the directory under which to save the base models
empty_as_null: true                      # whether or not to apply EMPTYASNULL logic to all text fields
incl_fivetran_deleted: false             # whether or not to include records marked as "deleted" by fivetran
excl_fivetran_synced: true               # whether or not to include the time fivetran synced each record

jthandy

Running vacuum and analyze in Sinter

dbt and Sinter have the ability to run regular Redshift maintenance jobs. It's great to set these up early on in a project so that things stay clean as the project grows, and implementing these jobs in Sinter allows the same easy transparency and notifications as with your other dbt jobs.

This document will go through the specific steps necessary to configure vacuum and analyze jobs in the current version of dbt and Sinter. In the future, there will likely be a more idiomatically consistent way to express this logic using native dbt operations. Currently, this does work even if it is not elegant.

Step 1: Create the macros

macros/redshift_maintenance.sql

tconbeer

# Any lines preceded by > can be typed or pasted directly into Powershell.
# This has been tested on Windows 10 and dbt v 0.12. Last updated Jan 4, 2019

# Before trying to follow this recipe, you need to set the Execution Policy
# You probably need to do this running Powershell as an admin
> Set-ExecutionPolicy RemoteSigned

# Next install chocolatey:
> iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

mplovepop

#!/usr/bin/python

import os, sys, random, string

length = int(sys.argv[1]) if len(sys.argv) > 1 else 4

if length < 8:
    with open('/usr/share/dict/words', 'r') as f:
        choice_list = [l.strip() for l in f.readlines()]
    ch = ' '