Skip to content

Instantly share code, notes, and snippets.

View potiuk's full-sized avatar

Jarek Potiuk potiuk

View GitHub Profile
@potiuk
potiuk / gist:6bd2d855a01ed3e8877a30cefcec01b3
Last active August 6, 2023 01:05
Examples of deadlocks caused by
See https://github.com/apache/airflow/issues/32778
2023-08-05 21:33:46.760 UTC [1045] DETAIL: Process 1045 waits for ShareLock on transaction 15501; blocked by process 1124.
Process 1124 waits for ShareLock on transaction 15500; blocked by process 1045.
Process 1045: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T21:33:42.780404+00:00'::timestamptz WHERE dag_run.id = 1
Process 1124: UPDATE task_instance SET start_date='2023-08-05T21:33:45.762562+00:00'::timestamptz, end_date='2023-08-05T21:33:45.762562+00:00'::timestamptz, duration=0.0, state='skipped', updated_at='2023-08-05T21:33:45.765508+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.task_id = 'run_this_last' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND task_instance.map_index = -1
2023-08-05 21:33:46.760 UTC [1045] HINT: See server log for query details.
2023-08-05 21:33:46.760 UTC [1045] CONTEXT: while updating tuple (0,228) in relation "dag_run"
2023-
@potiuk
potiuk / keybase.md
Last active September 3, 2022 19:43

Keybase proof

I hereby claim:

  • I am potiuk on github.
  • I am jarekpotiuk (https://keybase.io/jarekpotiuk) on keybase.
  • I have a public key ASDinc_Wve13PTecFbDqHxLJOtLKiUqLAcnjDYNscLLVWAo

To claim this, I am signing this object:

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
commit e1cefc791241fd82f02ab85d801882536f77febc
Author: Jarek Potiuk <[email protected]>
Date: Tue Jun 14 11:29:53 2022 +0200
This is the result of running the scripts from
https://github.com/apache/airflow/pull/24389
Fixes: https://github.com/apache/airflow/issues/23632
@potiuk
potiuk / output.txt
Last active February 27, 2022 11:57
Airflow is installed (with warnings) but no backtracking occurs
root@f7c21e8bf1c8:/opt/airflow# pip install ".[devel_all]" --upgrade --upgrade-strategy eager "dill<0.3.3" "certifi<2021.0.0" "google-ads<14.0.1" "github3.py<3.1.2"
Processing /opt/airflow
Preparing metadata (setup.py) ... done
Requirement already satisfied: dill<0.3.3 in /usr/local/lib/python3.7/site-packages (0.3.1.1)
Collecting dill<0.3.3
Downloading dill-0.3.2.zip (177 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.6/177.6 KB 3.0 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi<2021.0.0 in /usr/local/lib/python3.7/site-packages (2020.12.5)
@potiuk
potiuk / output.txt
Created February 25, 2022 17:50
Backtracking when github3.py is not limited.
root@f7c21e8bf1c8:/opt/airflow# pip install ".[devel_all]" --upgrade --upgrade-strategy eager "dill<0.3.3" "certifi<2021.0.0" "google-ads<14.0.1"
Processing /opt/airflow
Preparing metadata (setup.py) ... done
Requirement already satisfied: dill<0.3.3 in /usr/local/lib/python3.7/site-packages (0.3.1.1)
Collecting dill<0.3.3
Downloading dill-0.3.2.zip (177 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.6/177.6 KB 2.8 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: certifi<2021.0.0 in /usr/local/lib/python3.7/site-packages (2020.12.5)
Requirement already satisfied: google-ads<14.0.1 in /usr/local/lib/python3.7/site-packages (14.0.0)
ff --git a/setup.cfg b/setup.cfg
index 65c22d2280..b83ef9be02 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -30,7 +30,6 @@ license_files =
# Start of licenses generated automatically
licenses/LICENSE-bootstrap.txt
licenses/LICENSE-bootstrap3-typeahead.txt
- licenses/LICENSE-connexion.txt
licenses/LICENSE-d3-shape.txt
https://github.com/apache/airflow/issues/18025
https://github.com/apache/airflow/issues/16316
https://github.com/apache/airflow/issues/16125
https://github.com/apache/airflow/issues/15892
https://github.com/apache/airflow/issues/15456
https://github.com/apache/airflow/issues/15326
https://github.com/apache/airflow/issues/14287
https://github.com/apache/airflow/issues/14167
https://github.com/apache/airflow/issues/13918
https://github.com/apache/airflow/issues/13070
This file has been truncated, but you can view the full file.
root@d65474eee0cb:/opt/airflow# pip install -vv apache-airflow[google]==2.0.2 --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.0.2/constraints-3.6.txt
Using pip 21.0.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-ts76mcs4
Created temporary directory: /tmp/pip-req-tracker-qdp_isuo
Initialized build tracking at /tmp/pip-req-tracker-qdp_isuo
Created build tracker: /tmp/pip-req-tracker-qdp_isuo
Entered build tracker: /tmp/pip-req-tracker-qdp_isuo
Created temporary directory: /tmp/pip-install-2u2d9gn8
Starting new HTTPS connection (1): raw.githubusercontent.com:443
This file has been truncated, but you can view the full file.
pip install -vvvv apache-airflow[google]==2.0.2 --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.0.2/constraints-3.6.txt
Using pip 21.0.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-rvwadya0
Created temporary directory: /tmp/pip-req-tracker-83tx9fgp
Initialized build tracking at /tmp/pip-req-tracker-83tx9fgp
Created build tracker: /tmp/pip-req-tracker-83tx9fgp
Entered build tracker: /tmp/pip-req-tracker-83tx9fgp
Created temporary directory: /tmp/pip-install-licfxj35
Starting new HTTPS connection (1): raw.githubusercontent.com:443