Walk thru for CAB call to create a buildpack for etcd proxy and run it as a Cloud Foundry sidecar

buildpack-sidecar-walkthru.md

Distributing sidecars with buildpacks

Slides are at https://speakerdeck.com/drnic/distributing-cloud-foundry-sidecars-with-buildpacks

What are sidecars? How to use them in Cloud Foundry? - by Tim Downey

Intro to sidecars from last CAB

• until recently a running app container only ran and monitored a single process
• You could manually fork off child processs but they weren’t monitored (static file buildpack starts nginx and also starts cat-ting logs to stdout
• Secondary processes inside containers are now first class citizens
• Show manifest.yml and CD v3-push example
• Where does the sidecar software come from?

Intro to buildpacks

• separation of concerns across the org and vendors
• Supply buildpack vs normal one

A sidecar is probably not a primary concern of an app dev so shouldn’t be pushed with app So it goes in a buildpack

config-server src => binary => s3 => supply buildpack src => buildpack zip => cf update-buildpack

Walk thru

etcd proxy - to clustered etcd backend https://etcd.io/docs/v3.3.12/op-guide/grpc_proxy/

https://github.com/etcd-io/etcd/releases

sidecar command: $ etcd grpc-proxy start --endpoints=host.cfdev.sh --listen-addr=0.0.0.0:2379

Download etcd-v3.3.13-linux-amd64.tar.gz and get its sha256:

2c2e2a9867c1c61697ea0d8c0f74c7e9f1b1cf53b75dff95ca3bc03feb19ea7e  /Users/drnic/Downloads/etcd-v3.3.13-linux-amd64.tar.gz

copy manifest.yml from simple-pancake-buildpack, including "yt"

mkdir bin
touch bin/supply README.md
chmod +x bin/supply
echo "3.3.13" > VERSION

Create bin/supply

#!/bin/bash

echo "-----> Etcd Proxy Buildpack version 3.3.13"
exit 0

Build a cached buildpack

buildpack-packager build -stack cflinuxfs3 -cached

See contents of cached buildpack

$ unzip -l etcdproxy_buildpack-cached-cflinuxfs3-v3.3.13.zip
Archive:  etcdproxy_buildpack-cached-cflinuxfs3-v3.3.13.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  06-17-2019 17:11   README.md
        6  06-17-2019 17:11   VERSION
      504  06-17-2019 17:11   bin/supply
      726  06-17-2019 17:11   manifest.yml
 10423953  06-17-2019 16:48   dependencies/02866fcde1388c50c101a96ed2d1cfdb/etcd-v3.3.13-linux-amd64.tar.gz
---------                     -------

Upload to CF Dev

cf create-buildpack etcdproxy_buildpack etcdproxy_buildpack-cached-cflinuxfs3-v3.3.13.zip 100 --enable
cf buildpacks

Initial app

mkdir -p fixtures/simple
echo "hello world" > fixtures/simple/index.html

Initially let's use normal "cf push"; but to run sidecars later we need "cf v3-push" and friends

cf push simple -p fixtures/simple -b etcdproxy_buildpack -b staticfile_buildpack

Ok, that did nothing; update bin/supply to discover contents of our buildpack during staging:

export BUILDPACK_DIR=`dirname $(readlink -f ${BASH_SOURCE%/*})`
ls -alR $BUILDPACK_DIR
...

Recreate buildpack and upload

buildpack-packager build -stack cflinuxfs3 -cached && 
  cf delete-buildpack etcdproxy_buildpack -f && 
  cf create-buildpack etcdproxy_buildpack etcdproxy_buildpack-cached-cflinuxfs3-v3.3.13.zip 100 --enable &&
  cf push simple -p fixtures/simple -b etcdproxy_buildpack -b staticfile_buildpack

We see our etcd-v3.3.13-linux-amd64.tar.gz file under $BUILDPACK_DIR/dependencies/*/etcd-v3.3.13-linux-amd64.tar.gz

We also see VERSION under $BUILDPACK_DIR

Let's unpack etcd.tar.gz

#!/bin/bash

set -euo pipefail

export BUILD_DIR=$1
export CACHE_DIR=$2
export DEPS_DIR=$3
export DEPS_IDX=$4

export BUILDPACK_DIR=`dirname $(readlink -f ${BASH_SOURCE%/*})`

version=$(cat $BUILDPACK_DIR/VERSION)
echo "-----> Etcd Proxy Buildpack version $version"

archive=$(ls $BUILDPACK_DIR/dependencies/*/etcd-v$version-linux-amd64.tar.gz)

cd $DEPS_DIR/$DEPS_IDX
tar xfz $archive

ls -alR .

We see etcd binary in ./etcd-v3.3.13-linux-amd64/etcd, plus etcdctl for good measure

If we ssh into app we can see that our etcd binary etc are now available:

$ cf ssh simple
$ ls -al /home/vcap/deps/0/etcd-v3.3.13-linux-amd64/
-rwxr-xr-x  1 vcap vcap 16927136 May  2 17:55 etcd
-rwxr-xr-x  1 vcap vcap 13498880 May  2 17:55 etcdctl

If we can get "etcd" into bin/etcd then it will automatically be added to $PATH

archive=$(ls $BUILDPACK_DIR/dependencies/*/etcd-v$version-linux-amd64.tar.gz)
(
  mkdir -p /tmp/unpack
  cd /tmp/unpack
  tar xfz $archive
)
mkdir $DEPS_DIR/$DEPS_IDX/bin
cp /tmp/unpack/etcd-v$version-linux-amd64/etcd* $DEPS_DIR/$DEPS_IDX/bin

After rebuild, etcd is now in $PATH

$ cf ssh simple
$ /tmp/lifecycle/shell
$ echo $PATH
/home/vcap/deps/1/bin:/home/vcap/deps/0/bin:/bin:/usr/bin
$ etcd -version
etcd Version: 3.3.13
Git SHA: 98d3084
Go Version: go1.10.8
Go OS/Arch: linux/amd64

Now we can upgrade our application to use sidecars

Create fixtures/simple/manifest.yml

applications:
- name: simple
  instances: 1
  stack: cflinuxfs3
  buildpacks:
  - etcdproxy_buildpack
  - staticfile_buildpack
  sidecars:
  - name: etcd
    process_types: [web]
    command: etcd grpc-proxy start --endpoints=host.cfdev.sh:2379 --listen-addr=127.0.0.1:2379

Recreate app

cf delete simple -f
cf v3-create-app simple
cf v3-apply-manifest -f fixtures/simple/manifest.yml
cf v3-push simple

Look at logs to see etcd proxy running:

$ cf logs simple --recent
   2019-06-16T10:40:41.02-0700 [APP/PROC/WEB/SIDECAR/ETCD/0] ERR 2019-06-16 17:40:41.020276 I | etcdmain: listening for grpc-proxy client requests on 127.0.0.1:2379

Let's play with etcd via our proxy

$ cf ssh simple
$ /tmp/lifecycle/shell
$ ETCDCTL_API=3 etcdctl put from cab-call
$ ETCDCTL_API=3 etcdctl get from
$ exit

To confirm that this is all proxying to etcd server on my laptop

$ ETCDCTL_API=3 etcdctl get from
from
cab-call