Skip to content

Instantly share code, notes, and snippets.

View KyleBanks's full-sized avatar

Kyle Banks KyleBanks

272 followers · 0 following
View GitHub Profile
@inchoate
inchoate / including_external_package_in_dataflow.md
Last active February 2, 2024 11:40
Adding an extra package to a Python Dataflow project to run on GCP

The Problem

The documentation for how to deploy a pipeline with extra, non-PyPi, pure Python packages on GCP is missing some detail. This gist shows how to package and deploy an external pure-Python, non-PyPi dependency to a managed dataflow pipeline on GCP.

TL;DR: You external package needs to be a python (source/binary) distro properly packaged and shipped alongside your pipeline. It is not enough to only specify a tar file with a setup.py.

Preparing the External Package

Your external package must have a proper setup.py. What follow is an example setup.py for our ETL package. This is used to package version 1.1.1 of the etl library. The library requires 3 native PyPi packages to run. These are specified in the install_requires field. This package also ships with custom external JSON data, declared in the package_data section. Last, the setuptools.find_packages function searches for all available packages and returns that

@MarsVard
MarsVard / README
Last active November 3, 2022 19:19
android drawable to imitate google cards.
put card.xml in your drawables directory, put colors.xml in your values directory or add the colors to your colors.xml file.
set the background of a view to card,
as you can see in card.xml the drawable handles the card margin, so you don't have to add a margin to your view
``` xml
<View
android:layout_width="fill_parent"