Skip to content

Instantly share code, notes, and snippets.

View anna-geller's full-sized avatar

Anna Geller anna-geller

View GitHub Profile
@anna-geller
anna-geller / kestra_modal_interactive_workflow.yaml
Created October 7, 2024 20:27
Example showing how to create interactive workflows that dynamically adapt to user inputs using Kestra’s open-source orchestration platform and Modal’s serverless infrastructure.
id: modal_forecast
namespace: company.team
inputs:
- id: s3_bucket
displayName: S3 bucket name
description: Name of an S3 bucket to upload final ML artifacts
type: STRING
defaults: kestra-us
{
"id": "3PXJjZGwqAgpeMTv4JRPjs",
"namespace": "dev",
"flowId": "hello-world",
"flowRevision": 5,
"state": {
"current": "CREATED",
"histories": [
{
"state": "CREATED",
{
"action": "opened",
"number": 1,
"pull_request": {
"id": 123456789,
"url": "https://api.github.com/repos/octocat/Hello-World/pulls/1",
"html_url": "https://github.com/octocat/Hello-World/pull/1",
"diff_url": "https://github.com/octocat/Hello-World/pull/1.diff",
"patch_url": "https://github.com/octocat/Hello-World/pull/1.patch",
"issue_url": "https://api.github.com/repos/octocat/Hello-World/issues/1",

Welcome to the Code Editor!

The embedded Visual Studio Code IDE lets you easily add custom scripts, queries, and configuration files along with your flow YAML configuration files.

Get started by selecting a namespace. If you type a name of a namespace that doesn't exist yet, kestra will create it for you at runtime.

Then, add a new file, e.g., a Python script. Try adding a folder named scripts and a file called hello.py with the following content:

print("Hello from the Editor!")
{
"action": "opened",
"number": 7,
"pull_request": {
"url": "https://api.github.com/repos/anna-geller/kestra-ci-cd/pulls/7",
"id": 1545417746,
"node_id": "PR_kwDOJfJXx85cHTQS",
"html_url": "https://github.com/anna-geller/kestra-ci-cd/pull/7",
"diff_url": "https://github.com/anna-geller/kestra-ci-cd/pull/7.diff",
"patch_url": "https://github.com/anna-geller/kestra-ci-cd/pull/7.patch",
{title:"a"}
{title:"b"}
{title:"c"}
{title:"d"}
[
{
"Data": "{\"Event\": \"sign-in\", \"Date\": \"2023-09-25T08:30:00\"}",
"PartitionKey": "user42"
},
{
"Data": "{\"Event\": \"log-in\", \"Date\": \"2023-09-25T08:40:00\"}",
"PartitionKey": "user42"
},
{
[
{
"Data": {
"Event": "sign-in",
"Date": "2023-09-25T08:30:00"
},
"PartitionKey": "user42"
},
{
"Data": {
id: shell
namespace: dev
description: this flow shows how to pass files between tasks.
tasks:
- id: t1
type: io.kestra.plugin.scripts.shell.Commands
commands:
- touch abc
- id: t2
type: io.kestra.plugin.scripts.shell.Commands
id: airbyteDbtCore
namespace: dev
tasks:
- id: data-ingestion
type: io.kestra.core.tasks.flows.Parallel
tasks:
- id: psyduck
type: io.kestra.plugin.airbyte.connections.Sync
connectionId: 4de8ab1e-50ef-4df0-aa01-7f21491081f1