Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def aggregation_skipped_trigger(upstream_states: Dict[Edge, State]) -> bool: | |
""" | |
Custom trigger which trigger fails only if the aggregation task | |
specifically was skipped. | |
""" | |
# upstream_states is a dictionary of _all_ upstream dependencies | |
# but this task will only care about the aggregation task specifically | |
agg_state = [ | |
state |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
The new Prefect Result interface is a mechanism for interacting with / retrieving data from possibly third party locations | |
in a secure manner. Results also provide convenient hooks for caching. | |
""" | |
class Result: | |
def __init__(self, **kwargs): | |
""" | |
Initialize the class. | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# if you wish to pass a start time other than immediately | |
# you can provide an ISO formatted timestamp for when you | |
# want this run to begin | |
from datetime import datetime, timedelta | |
# actually run the flow in two hours | |
inputs['scheduledStartTime'] = (datetime.utcnow() + timedelta(hours=2)).isoformat() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# if you wish to prevent duplicate runs from being created, you may also | |
# provide an idempotency key | |
# for example, the eventTime | |
inputs['idempotencyKey'] = event['Records'][0]['eventTime'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# if you wish to pass information about the triggering event as a parameter, | |
# simply add that to the inputs dictionary under the parameters key, | |
# whose value should be a dictionary of PARAMETER_NAME -> PARAMETER_VALUE | |
# pass the full event | |
inputs['parameters'] = dict(event=event) | |
# or just the bucket name | |
inputs['parameters'] = dict(bucket_name=event['Records'][0]['s3']['bucket']['name']) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import os | |
import urllib.parse | |
import urllib.request | |
print("Loading function") | |
def lambda_handler(event, context): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from prefect.tasks.database.sqlite import SQLiteScript | |
create_script = "CREATE TABLE IF NOT EXISTS SSHATTEMPTS (timestamp TEXT, username TEXT, port INTEGER, city TEXT, country TEXT, latitude REAL, longitude REAL)" | |
create_table = SQLiteScript( | |
db="ssh.db", script=create_script, name="Create Database and Table" | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from prefect import task, Flow | |
@task | |
def create_list(): | |
return [1, 1, 2, 3] | |
@task | |
def add_one(x): | |
return x + 1 |
NewerOlder