Sean Davis seandavi

How to deploy a Prefect agent to Google Kubernetes Engine

This post contains code and commands you can use to deploy Prefect agents to Google Cloud’s Google Kubernetes Engine. The agents stand ready to execute workflows triggered by Prefect projects. One agent can run tasks from multiple projects.

The example here demonstrates how to create a single agent with minimal customization. It is configured with a Dockerfile, which installs necessary dependencies, and a k8s.cfg file, which connects the system to a Prefect account.

Agents are deployed via the gcloud command-line utility and its kubectl extension. Proper permissions within a project on Google Cloud are required.

Getting started

Security Advisories / Bulletins / vendors Responses linked to Log4Shell (CVE-2021-44228)

Errors, typos, something to say ?

If you want to add a link, comment or send it to me
Feel free to report any mistake directly below in the comment or in DM on Twitter @SwitHak

Other great resources

Royce Williams list sorted by vendors responses Royce List
Very detailed list NCSC-NL
The list maintained by U.S. Cybersecurity and Infrastructure Security Agency: CISA List

	import asyncio
	import psycopg2

	# dbname should be the same for the notifying process
	conn = psycopg2.connect(host="localhost", dbname="example", user="example", password="example")
	conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)

	cursor = conn.cursor()
	cursor.execute(f"LISTEN match_updates;")

	# Functions for downloading data on Coronavirus cases in the UK by country,
	# region, NHS region, and local authority from the government dashboard API.
	#
	# The developer guide is here:
	#
	# https://coronavirus.data.gov.uk/developers-guide
	#
	# The code below is based on the example code shown there.
	#
	# Run ...

	from starlette.applications import Starlette
	from starlette.responses import HTMLResponse
	from starlette.websockets import WebSocket
	from jinja2 import Template
	import uvicorn


	template = """\
	<!DOCTYPE HTML>
	<html>

	# Add this to your .Rprofile
	options(
	error = quote(rlang::entrace()),
	rlang__backtrace_on_error = "collapse" # or "branch" or "full"
	)

	---
	title: "Data mask stuff"
	output: github_document
	---
	```{r include=FALSE, message=FALSE, warning=FALSE}
	knitr::opts_chunk$set(
	message = FALSE, warning = FALSE, echo = TRUE, fig.retina = 2, collapse = TRUE
	)
	```

Sean Davis seandavi

How to deploy a Prefect agent to Google Kubernetes Engine

Getting started

Errors, typos, something to say ?

Other great resources

Table of Contents

set -e, -u, -x, -o pipefail

	# Can work for other delimiters as well
	# Tab delimiter
	bq load --source_format=CSV --field_delimiter=tab \
	--skip_leading_rows 1 -<destination_table> <source> \
	<schema>

	[Install]
	WantedBy=multi-user.target

	[Unit]
	Description=Google Cloud Compute Engine SQL Proxy
	Requires=networking.service
	After=networking.service

	[Service]
	Type=simple