Skip to content

Instantly share code, notes, and snippets.

@drewbanin

drewbanin/changelog-0.17.md

Last active June 8, 2020 13:44
Show Gist options
  • Save drewbanin/bb6610796515f738ef20d83d6147c771 to your computer and use it in GitHub Desktop.
Save drewbanin/bb6610796515f738ef20d83d6147c771 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
changelog-0.17.md

dbt 0.17.0 - Octavius Catto (June 8th, 2020)

Breaking changes

  • Added a new dbt_project.yml version format. This emits a deprecation warning currently, but support for the existing version will be removed in a future dbt version (#2300, #2312)
  • The graph object available in some dbt contexts now has an additional member sources (along side the existing nodes). Sources have been removed from nodes and added to sources instead (#2312)
  • The 'location' field has been removed from bigquery catalogs (#2382)
  • The list_relations_without_caching, drop_schema, and create_schema macros and methods now accept a single argument of a Relation object with no identifier field. (#2411)

Features

Core

  • Add --fail-fast argument for dbt run and dbt test to fail on first test failure or runtime error. (#1649, #2224)
  • Support for appending query comments to SQL queries. (#2138, #2199)
  • Add a get-manifest API call. (#2168, #2232)
  • Support adapter-specific aliases (like project and dataset on BigQuery) in source definitions. (#2133, #2244)
  • Users can now use jinja as arguments to tests. Test arguments are rendered in the native context and injected into the test execution context directly. (#2149, #2220)
  • Users can supply paths as arguments to --models and --select, either explicitily by prefixing with path: or implicitly with no prefix. (#454, #2258)
  • dbt now builds the relation cache for "dbt compile" and "dbt ls" as well as "dbt run" (#1705, #2319)
  • Add a 'depends_on' attribute to the log record extra field (#2316, #2341)
  • Add a '--no-browser' argument to "dbt docs serve" so you can serve docs in an environment that only has a CLI browser which would otherwise deadlock dbt (#2004, #2364)
  • Sources (and therefore freshness tests) can be enabled and disabled via dbt_project.yml (#2283, #2312, #2357)
  • schema.yml files are now fully rendered in a context that is aware of vars declared in from dbt_project.yml files (#2269, #2357)
  • Sources from dependencies can be overridden in schema.yml files (#2287, #2357)
  • Add a filter named as_text to the native environment rendering code that allows users to mark a value as always being a string (#2384, #2395)
  • Add warning to nodes selector if nothing was matched (#2115, #2343)

Snowflake

  • Snowflake now uses "show terse objects" to build the relations cache instead of selecting from the information schema (#2174, #2322)
  • Snowflake now uses "describe table" to get the columns in a relation (#2260, #2324)
  • Snowflake now uses "describe table" to get the columns in a relation (#2260, #2324)
  • Relation comments supported for Snowflake tables and views. Column comments supported for tables. (#1722, #2321)

BigQuery

  • Suport column descriptions for BigQuery models (#2335, #2402)
  • Add BigQuery option maximum_bytes_billed to set an upper limit for query costs (#2346, #2427)

Postgres & Redshift

  • Added support for db_groups and autocreate flags in Redshift configurations. (#1995, #2262)
  • Implement persist_docs for both relation and comments on postgres and redshift, and extract them when getting the catalog. (#2333, #2378)

Docs

Fixes

  • When a jinja value is undefined, give a helpful error instead of failing with cryptic "cannot pickle ParserMacroCapture" errors (#2110, #2184)
  • Added timeout to registry download call (#2195, #2228)
  • When a macro is called with invalid arguments, include the calling model in the output (#2073, #2238)
  • When a warn exception is not in a jinja do block, return an empty string instead of None (#2222, #2259)
  • Add dbt plugin versions to --version(#2272, #2279)
  • When a Redshift table is defined as "auto", don't provide diststyle (#2246, #2298)
  • Made file names lookups case-insensitve (.sql, .SQL, .yml, .YML) and if .yaml files are found, raise a warning indicating dbt will parse these files in future releases. (#1681, #2263)
  • Return error message when profile is empty in profiles.yml. (#2292, #2297)
  • Fix skipped node count in stdout at the end of a run (#2095, #2310)
  • Fix an issue where BigQuery incorrectly used a relation's quote policy as the basis for the information schema's include policy, instead of the relation's include policy. (#2188, #2325)
  • Fix "dbt deps" command so it respects the "--project-dir" arg if specified. (#2338, #2339)
  • On run_cli API calls that are passed --vars differing from the server's --vars, the RPC server rebuilds the manifest for that call. (#2265, #2363)
  • Remove the query job SQL from bigquery exceptions (#2383, #2393)
  • Fix "Object of type Decimal is not JSON serializable" error when BigQuery queries returned numeric types in nested data structures (#2336, #2348)
  • No longer query the information_schema.schemata view on bigquery (#2320, #2382)
  • Preserve original subdirectory structure in compiled files. (#2173, #2349)
  • Add support for sql_header config in incremental models (#2136, #2200)
  • The ambiguous alias check now examines the node's database value as well as the schema/identifier (#2326, #2387)
  • Postgres array types can now be returned via run_query macro calls (#2337, #2376)
  • Add missing comma to dbt compile help text (#2388 #2389)
  • Fix for non-atomic snapshot staging table creation (#1884, #2390)
  • Fix for snapshot errors when strategy changes from check to timestamp between runs (#2350, #2391)
  • On snowflake, get_columns_in_relation now returns an empty list again if the relation does not exist, instead of raising an exception. (#2504, #2509)
  • Added filename, project, and the value that failed to render to the exception raised when rendering fails. (#2499, #2501)
  • When no columns are documented and persist_docs.columns is True, skip creating comments instead of failing with errors (#2439, #2440)
  • Fixed an argument issue with the create_schema macro on bigquery (#2445, #2448)
  • dbt now logs using the adapter plugin's ideas about how relations should be displayed (dbt-spark/#74, #2450)
  • The create_adapter_plugin.py script creates a version 2 dbt_project.yml file (#2451, #2455)
  • Fixed dbt crashing with an AttributeError on duplicate sources (#2463, #2464)
  • Fixed a number of issues with globally-scoped vars (#2473, #2472, #2469, #2477)
  • Fixed DBT Docker entrypoint (#2470, #2475)
  • Fixed a performance regression that occurred even when a user was not using the relevant feature (#2474, #2478)
  • Substantial performance improvements for parsing on large projects, especially projects with many docs definition. (#2480, #2481)
  • Expose Snowflake query id in case of an exception raised by connector (#2201, #2358)
  • When tracking is disabled due to errors, do not reset the invocation ID (#2398, #2400)
  • Fix for logic error in compilation errors for duplicate data test names (#2406, #2407)
  • Fix list_schemas macro failing for BigQuery (#2412, #2413)
  • When plugins are installed in the same folder as dbt core, report their versions. (#2410, #2418)
  • Fix for making schema tests work for community plugin dbt-sqlserver #2414
  • Fix a bug where quoted uppercase schemas on snowflake were not processed properly during cache building. (#2403, #2411)
  • Fix for extra spacing and parentheses when creating views in BigQuery (#2421, #2422)

Under the hood

  • Added more tests for source inheritance (#2264, #2291)
  • Update documentation website for 0.17.0 (#2284)
  • Lock protobufs to the last version that had fully functioning releases on all supported platforms (#2490, #2491)
  • Removed pytest-logbook dependency from dbt-core (#2505)
  • Better support for optional database fields in adapters (#2487 #2489)
  • Track distinct project hashes in anonymous usage metrics for package downloads (#2351, #2429)

Contributors:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment