This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- dbt run-operation drop_pre_defer --args '{tables_to_drop: [inc_orders,dim_customers]}' | |
{% macro drop_pre_defer(models_to_drop) %} | |
{% if execute %} | |
{% for model_to_drop in models_to_drop %} | |
{%- for model in graph.nodes.values() | selectattr("name", "equalto", model_to_drop) -%} | |
{% set query %} | |
drop {{model.config.materialized}} if exists {{target.database}}.{{target.schema}}.{{model_to_drop}}; | |
{% endset %} | |
{% do run_query(query) %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- Alternative one: Limit the result regardless of the query | |
select * | |
from event_tracking.events | |
{% if target.name == 'dev' %} | |
limit 10 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{# drop_deprecated_tables | |
The purpose of this macro is to remove any stale tables/views that no longer have a corresponding model in dbt project. | |
Args: | |
- dry_run: bool -- dry run flag. When dry_run is true, the cleanup commands are printed to stdout rather than executed. This is true by default | |
Example 1 - dry run of current database | |
dbt run-operation drop_deprecated_tables |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{% macro generate_schema_name(custom_schema_name, node) -%} | |
{%- set default_schema = target.schema -%} | |
{# checks that no schema was predefined in project.yml or config | |
or we are in the development environment #} | |
{%- if custom_schema_name is none or target.name =='default' -%} | |
{{ default_schema }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo apt-get install curl ca-certificates gnupg && curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - | |
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" > /etc/apt/sources.list.d/pgdg.list' | |
sudo apt update && sudo apt-get install postgresql-11 pgadmin4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo apt-get install postgresql-11 | |
#log into postgres and create the user you'll use | |
sudo -i -u postgres | |
psql | |
CREATE USER vic WITH PASSWORD 'lala'; | |
\q | |
exit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# data structures and data analysis | |
sudo -H pip3 install numpy matplotlib scipy | |
sudo apt-get install python3-pandas | |
# Machine Learning | |
sudo -H pip3 install scikit-learn |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo apt update | |
sudo apt install python3-pip | |
#Verification | |
pip3 ----version |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo apt update | |
sudo apt install software-properties-common | |
sudo add-apt-repository ppa:deadsnakes/ppa | |
sudo apt install python3.7 | |
#Verification: should return Python 3.7.x | |
python3.7 --version |