Skip to content

Instantly share code, notes, and snippets.

@yucer
Created August 28, 2023 12:31
Show Gist options
  • Save yucer/cd30d29b05894cb2698e3a2a693a2248 to your computer and use it in GitHub Desktop.
Save yucer/cd30d29b05894cb2698e3a2a693a2248 to your computer and use it in GitHub Desktop.
poetry config after poetry init --no-interaction
[tool.poetry]
name = "airflow-dbt2"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
@yucer
Copy link
Author

yucer commented Aug 28, 2023

After:

$ poetry add apache-airflow apache-airflow-providers-postgres dbt-postgres pandas astronomer-cosmos

poetry fails to find an optimal combination of versions for the dependencies :

Using version ^2.7.0 for apache-airflow
Using version ^5.6.0 for apache-airflow-providers-postgres
Using version ^1.6.1 for dbt-postgres
Using version ^2.0.3 for pandas
Using version ^1.0.5 for astronomer-cosmos

Updating dependencies
Resolving dependencies... (9.7s)

Because no versions of dbt-postgres match >1.6.1,<1.7.0b1 || >1.7.0b1,<2.0.0
 and dbt-postgres (1.7.0b1) depends on dbt-core (1.7.0b1), dbt-postgres (>1.6.1,<2.0.0) requires dbt-core (1.7.0b1).
And because dbt-core (1.7.0b1) depends on dbt-semantic-interfaces (>=0.2.0,<0.3.0), dbt-postgres (>1.6.1,<2.0.0) requires dbt-semantic-interfaces (>=0.2.0,<0.3.0).
And because dbt-postgres (1.6.1) depends on dbt-core (1.6.1)
 and dbt-core (1.6.1) depends on dbt-semantic-interfaces (>=0.2.0,<0.3.0), dbt-postgres (>=1.6.1,<2.0.0) requires dbt-semantic-interfaces (>=0.2.0,<0.3.0).
Because dbt-semantic-interfaces (0.2.0) depends on jsonschema (>=3.0,<4.0)
 and no versions of dbt-semantic-interfaces match >0.2.0,<0.3.0, dbt-semantic-interfaces (>=0.2.0,<0.3.0) requires jsonschema (>=3.0,<4.0).
Thus, dbt-postgres (>=1.6.1,<2.0.0) requires jsonschema (>=3.0,<4.0).
And because apache-airflow (2.7.0) depends on jsonschema (>=4.18.0)
 and no versions of apache-airflow match >2.7.0,<3.0.0, dbt-postgres (>=1.6.1,<2.0.0) is incompatible with apache-airflow (>=2.7.0,<3.0.0).
So, because airflow-dbt2 depends on both apache-airflow (^2.7.0) and dbt-postgres (^1.6.1), version solving failed.

@yucer
Copy link
Author

yucer commented Aug 28, 2023

But, specifying these dependencies it doesn't fail:

[tool.poetry.dependencies]
python = "^3.10"
apache-airflow = "^2.6.2"
apache-airflow-providers-postgres = "^5.5.0"
dbt-postgres = "^1.5.1"
pandas = "^2.0.2"
astronomer-cosmos = "^0.7.3"
black = "^23.3.0"

@yucer
Copy link
Author

yucer commented Aug 28, 2023

The issue was a problem already known. When no dependencies are specified poetry uses the last version and fails. Should it also try to downgrade packages ? I guess so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment