This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas | |
import yfinance as yf | |
from datetime import timedelta | |
from prefect import flow, task | |
@task(retries=3, cache_expiration=timedelta(30)) | |
def fetch_data(ticker): | |
return yf.download(ticker) | |
@task |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using work queue 'Agent queue testing' | |
Starting agent with ephemeral API... | |
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____ | |
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _| | |
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_| | |
Agent started! Looking for work from queue(s): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
if __name__=='main': | |
pipeline(msg="Super Special Message") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
schedule: | |
interval: 600 | |
timezone: America/Chicago | |
schedule: | |
cron: 0 0 * * * | |
timezone: America/Chicago | |
schedule: | |
rrule: 'FREQ=WEEKLY;BYDAY=MO,WE,FR;UNTIL=20240730T040000Z' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Created task run 'Write results to database-5512f738-0' for task 'Write results to database' | |
18:00:20.661 | INFO | Flow run 'aspiring-tuatara' - Executing 'Write results to database-5512f738-0' immediately... | |
18:00:20.690 | INFO | Task run 'Write results to database-5512f738-0' - Finished in state Completed() | |
18:00:20.704 | INFO | Flow run 'aspiring-tuatara' - Finished in state Completed('All states completed.') | |
Wrote {'data': 42, 'message': "Let's rock this!"} to database successfully! | |
18:00:20.914 | INFO | prefect.infrastructure.process - Process 'aspiring-tuatara' exited cleanly. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
``` | |
Successfully loaded 'etl' | |
Deployment 'Previously unreliable pipeline/etl' | |
successfully created with id | |
'c90f53b8-71d3-4abf-a19d-c71a0d6b3e96'. | |
To execute flow runs from this deployment, start | |
an agent that pulls work from the the 'default' | |
work queue: | |
$ prefect agent start -q 'default' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Wrote {'data': 42, 'message': 'Super Special Message'} to database successfully! | |
Or | |
--------------------------------------------------------------------------- | |
Exception Traceback (most recent call last) | |
/Users/jeffhale/Desktop/prefect/prefect-101/p101.ipynb Cell 5 in <cell line: 1>() | |
----> 1 pipeline1(msg="Super Special Message") | |
/Users/jeffhale/Desktop/prefect/prefect-101/p101.ipynb Cell 5 in pipeline1(msg) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
___ ___ ___ ___ ___ ___ _____ ___ ___ ___ ___ _ _ | |
| _ \ _ \ __| __| __/ __|_ _| / _ \| _ \_ _/ _ \| \| | | |
| _/ / _|| _|| _| (__ | | | (_) | /| | (_) | .` | | |
|_| |_|_\___|_| |___\___| |_| \___/|_|_\___\___/|_|\_| | |
Configure Prefect to communicate with the server with: | |
prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api | |
View the API reference documentation at http://127.0.0.1:4200/docs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"flow_name": "Previously unreliable pipeline", | |
"import_path": "./basic-flow.py:pipeline4", | |
"parameter_openapi_schema": { | |
"title": "Parameters", | |
"type": "object", | |
"properties": { | |
"msg": { | |
"title": "msg", | |
"type": "string" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### | |
### A complete description of a Prefect Deployment for flow 'Previously unreliable pipeline' | |
### | |
name: etl | |
description: null | |
version: 2cbeebadbddd7f0203b3a03a1d5208e8 | |
# The work queue that will handle this deployment's runs | |
work_queue_name: default | |
tags: | |
- testing |