A developer’s experience writing a Spring Boot function for a "Modern App Platform"

This document is a Work in Progress.

Content:: Introduction
Getting started developing a function
Configure development tools
Set up a CI/CD pipeline
Local feature development

Introduction

This document describes the developer experience for creating a Spring Boot Java function that processes Cloud Events and can be run using Knative Serving.

Prerequisits:

Git CLI and a GitHub account
kubectl, riff CLI and the riff knative-runtime installed on a GKE Kubernetes cluster
Skaffold
Docker and pack CLI

Overview:

We will:

initialize a function app from scratch using Spring Initializr
add a Skaffold configuration to speed up local development
push app to GitHub and configure CI
create a Dev/Ops configuration using Argo CD and Kustomize

Getting started developing a function

Initialize a project

Initialize a Spring Boot function application from start.spring.io:

APPNAME=my-func
curl https://start.spring.io/starter.tgz \
 -d dependencies=webflux,actuator,cloud-function \
 -d language=java \
 -d type=maven-project \
 -d name=${APPNAME} \
 -d packageName=com.example.${APPNAME} \
 -d baseDir=${APPNAME} | tar -xzvf -
cd ${APPNAME}

Add the function code

A basic implementation could look like this:

package com.example.myfunc;

import java.util.Map;
import java.util.TreeMap;
import java.util.function.Function;

import com.fasterxml.jackson.databind.JsonNode;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;

@SpringBootApplication
public class MyFuncApplication {

	Logger log = LoggerFactory.getLogger(MyFuncApplication.class);

	@Bean
	public Function<Message<JsonNode>, String> hello() {
		return m -> {
			Map<String, String> headers = extractCloudEventsHeaders(m.getHeaders());
			log.info("HEADERS: " + headers);
			log.info("PAYLOAD: " + m.getPayload());
			return m.getPayload().toString();
		};
	}

	private Map<String, String> extractCloudEventsHeaders(Map<String, Object> messageHeaders) {
		Map<String, String> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
		for (String key : messageHeaders.keySet()) {
			if (key.toLowerCase().startsWith("ce-")) {
				headers.put(key, messageHeaders.get(key).toString());
			}
		}
		return headers;
	}

	public static void main(String[] args) {
		SpringApplication.run(MyFuncApplication.class, args);
	}

}

Build and test locally

Build and run:

./mvnw spring-boot:run

In a separate terminal:

curl -w'\n' localhost:8080 \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H "Content-Type: application/json" \
 -d '{"msg":"Hello World"}'

Build and test on k8s

Build with riff:

there is currently an issue with the function invoker when running spring-cloud-function that accepts a Message as input, so we build this as an app

riff application create my-func --local-path .

Deploy with riff:

riff knative deployer create my-func --application-ref my-func --ingress-policy External --tail

Look up ingress:

INGRESS=$(kubectl get svc envoy-external --namespace projectcontour --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')

Send a message:

curl -w'\n' $INGRESS \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H 'Host: my-func.default.example.com' \
 -H "Content-Type: application/json" \
 -d '{"msg":"Hello World"}'

Check the logs:

kubectl logs -c user-container -l knative.projectriff.io/deployer=my-func

Clean up:

riff knative deployer delete my-func
riff application delete my-func

Configure development tools

The local experience described above only gets you so far. We need better tools to make the developer experience more productive. Here we will use Skaffold and Cloud Native Buildpacks.

Create deployment manifest:

riff container create my-func --image my-func --dry-run > riff-deployer.yaml
riff knative deployer create my-func --container-ref my-func --ingress-policy External \
 --dry-run >> riff-deployer.yaml

Initialize skaffold:

skaffold init --skip-build

Modify skaffold.yaml and add the build section

apiVersion: skaffold/v2alpha4
kind: Config
metadata:
  name: my-func
build:
  artifacts:
    - image: my-func
      buildpack:
        builder: "cloudfoundry/cnb:cflinuxfs3"
  tagPolicy:
    sha256: {}
deploy:
  kubectl:
    manifests:
    - riff-deployer.yaml

Set your own prefix for the repository name, should be your Docker ID:

skaffold config set default-repo $USER

Deploy with skaffold:

skaffold run

Look up ingress:

INGRESS=$(kubectl get svc envoy-external --namespace projectcontour --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')

Send a message:

curl -w'\n' $INGRESS \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H 'Host: my-func.default.example.com' \
 -H "Content-Type: application/json" \
 -d '{"msg":"Hello World"}'

Check the logs:

kubectl logs -c user-container -l knative.projectriff.io/deployer=my-func

Clean up:

skaffold delete

Set up a CI/CD pipeline

Next step is to make this function project part of a CI/CD pipeline so new features and bug fixes can be deployed quickly. For this sample app we will use GitHub for the source control and ArgoCD for continous delivery.

Add your function code to a Git repository on GitHub

Here I’m pushing the code to an empty GitHub repo named trisberg/my-func:

Initialize the repo:

echo "# my-func" >> README.md
git init
git add README.md
git commit -m "first commit"

Add source:

git add .
git commit -m "initial function implementation"

Add remote origin and push:

git remote add origin [email protected]:trisberg/my-func.git
git push --set-upstream origin master

Configure Continous Integration (CI)

You can use GitHub Actions.

For simplicity’s sake we’ll just build the my-func app using:

skaffold build --default-repo springdeveloper --tag 0.1.0

Create a dev-ops repository

cd ..
APPNAME=my-func
mkdir ${APPNAME}-ops
cd ${APPNAME}-ops

Add the following content to this project:

mkdir -p base/kustomizeconfig
touch base/kustomization.yaml
touch base/kustomizeconfig/riff-application.yaml
touch base/riff-deployer.yaml
mkdir -p overlays/dev
touch overlays/dev/kustomization.yaml

You should now have the following file structure:

├── base
│   ├── kustomization.yaml
│   ├── kustomizeconfig
│   │   └── riff-application.yaml
│   └── riff-deployer.yaml
└── overlays
    └── dev
        └── kustomization.yaml

Add the following file content:

This is the base kustomization file:

base/kustomization.yaml

commonLabels:
  app.kubernetes.io/name: my-func
resources:
  - riff-deployer.yaml
configurations:
  - kustomizeconfig/riff-application.yaml

We need to add a configuration file to teach kustomize where the image si specified in riff’s Application resource.

base/kustomizeconfig/riff-application.yaml

images:
- path: spec/image
  create: true
  kind: Application

We also need the Deployer configuration with the Application. This is based on the file we generated above but we remove empty parts, replace Container resource with an Application pointing to our GitHub repo. We also remove the namespace since this will deployed to namespaces other than default.

base/riff-deployer.yaml

---
apiVersion: build.projectriff.io/v1alpha1
kind: Application
metadata:
  name: my-func
spec:
  image: my-func
  source:
    git:
      revision: master
      url: https://github.com/trisberg/my-func.git
---
apiVersion: knative.projectriff.io/v1alpha1
kind: Deployer
metadata:
  name: my-func
spec:
  build:
    applicationRef: my-func
  ingressPolicy: External

Next we add the kustomization for the dev deployment.

overlays/dev/kustomization.yaml

images:
  - name: my-func
    newName: springdeveloper/my-func
    newTag: 0.1.0-snapshot
resources:
  - ../../base

Initialize the repo and push to GitHub

I have created an empty trisberg/my-func-ops repository in GitHub.

Initialize the repo:

echo "# my-func-ops" >> README.md
git init
git add README.md
git commit -m "first commit"

Add source:

git add .
git commit -m "initial kustomize implementation"

Add remote origin and push:

git remote add origin [email protected]:trisberg/my-func-ops.git
git push --set-upstream origin master

Add your function ops repo to Argo CD

Install Argo CD

For detailed instructions to install CLI and server see: https://argoproj.github.io/argo-cd/getting_started/

Quick notes:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'
kubectl get service argocd-server -n argocd --watch

ARGOCD_SERVER=$(kubectl get service argocd-server -n argocd -o json | jq --raw-output .status.loadBalancer.ingress[0].ip)
# get initial password
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
argocd login $ARGOCD_SERVER --insecure --username admin

Create the Argo CD application for my-func-ops

Create the dev namespace and initialize it for building the application:

kubectl create namespace dev
riff credentials apply docker-creds --namespace dev --docker-hub $USER

Create a YAML file for the Argo CD app for dev:

argocd-dev.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-func-dev
spec:
  destination:
    namespace: dev
    server: 'https://kubernetes.default.svc'
  source:
    path: overlays/dev
    repoURL: 'https://github.com/trisberg/my-func-ops.git'
    targetRevision: HEAD
  project: default
  syncPolicy:
    automated:
      prune: false
      selfHeal: false

Apply this Argo CD app using:

kubectl apply -n argocd -f argocd-dev.yaml

If we need to, we can sync this Argo CD app using:

argocd app sync my-func-dev

Time for production

Create a stable branch and push it to origin.

First, switch to the app directory:

cd ../${APPNAME}

Create and push the new stable branch.

git checkout -b stable
git push --set-upstream origin stable

Check out the master branch for further app development.

git checkout master

Switch back to the ops project to add the production artifacts.

cd ../${APPNAME}-ops

Add the following content to this project:

mkdir -p overlays/prod
touch overlays/prod/kustomization.yaml
touch overlays/prod/revision.yaml

You should now have the following file structure:

├── base
│   ├── kustomization.yaml
│   ├── kustomizeconfig
│   │   └── riff-application.yaml
│   └── riff-deployer.yaml
└── overlays
    ├── dev
    │   └── kustomization.yaml
    └── prod
        ├── kustomization.yaml
        └── revision.yaml

Add the kustomization for the prod deployment. This includes a revision.yaml overlay that will use the stable branch for building the application.

overlays/prod/kustomization.yaml

images:
  - name: my-func
    newName: springdeveloper/my-func
    newTag: 0.1.0
resources:
  - ../../base
patchesStrategicMerge:
  - revision.yaml

overlays/prod/revision.yaml

apiVersion: build.projectriff.io/v1alpha1
kind: Application
metadata:
  name: my-func
spec:
  source:
    git:
      revision: stable

Commit the changes and push:

git add overlays/prod
git commit -m "added production artifacts"
git push origin

Create the prod namespace and initialize it for building the application:

kubectl create namespace prod
riff credentials apply docker-creds --namespace prod --docker-hub $USER

Create a YAML file for the Argo CD app for prod:

argocd-prod.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-func-prod
spec:
  destination:
    namespace: prod
    server: 'https://kubernetes.default.svc'
  source:
    path: overlays/prod
    repoURL: 'https://github.com/trisberg/my-func-ops.git'
    targetRevision: HEAD
  project: default
  syncPolicy:
    automated:
      prune: false
      selfHeal: false

Apply this Argo CD app using:

kubectl apply -n argocd -f argocd-prod.yaml

If we need to, we can sync this Argo CD app using:

argocd app sync my-func-prod

Local feature development

We’ve done a lot of setup so far. Let’s switch back to developing some features for this function.

Add marshalling of the payload to domain object

Typically apps and functions written with Spring Boot rely on converters that can marshall JSON representation of domain objects to the POJOs that we use in the application code. Since this example uses Cloud Events and we don’t yet have these converters available we do need to do the marshalling work ourselves.

Let’s introduce the domain object POJO for a HelloWorld.

First, switch to the app directory:

cd ../${APPNAME}

Create a new Java class file:

mkdir -p src/main/java/com/example
touch src/main/java/com/example/HelloWorld.java

Add this code to the HelloWold.java class file:

src/main/java/com/example/HelloWorld.java

package com.example;

public class HelloWorld {

    private String msg;

    public String getMsg() {
        return msg;
    }

    public void setMsg(String msg) {
        this.msg = msg;
    }

    @Override
    public String toString() {
        return "HelloWorld [msg=" + msg + "]";
    }
}

Next update the Function code to convert the payload to this new POJO. We’ll add an OjectMapper and a convertPayload code function.

package com.example.myfunc;

import java.util.Map;
import java.util.TreeMap;
import java.util.function.Function;

import com.example.HelloWorld;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;

@SpringBootApplication
public class MyFuncApplication {

	Logger log = LoggerFactory.getLogger(MyFuncApplication.class);

	ObjectMapper objectMapper = new ObjectMapper();

	@Bean
	public Function<Message<JsonNode>, String> hello() {
		return m -> {
			Map<String, String> headers = extractCloudEventsHeaders(m.getHeaders());
			log.info("HEADERS: " + headers);
			log.info("PAYLOAD: " + m.getPayload());
			HelloWorld pojo = convertPayload(m.getPayload());
			if (pojo != null) {
				return pojo.toString();
			} else {
				return "Unable to parse: " + m.getPayload();
			}
		};
	}

	private Map<String, String> extractCloudEventsHeaders(Map<String, Object> messageHeaders) {
		Map<String, String> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
		for (String key : messageHeaders.keySet()) {
			if (key.toLowerCase().startsWith("ce-")) {
				headers.put(key, messageHeaders.get(key).toString());
			}
		}
		return headers;
	}

	private HelloWorld convertPayload(JsonNode payload) {
		try {
			return objectMapper.treeToValue(payload, HelloWorld.class);
		} catch (JsonProcessingException e) {
			log.error(e.getMessage());
			return null;
		}
	}

	public static void main(String[] args) {
		SpringApplication.run(MyFuncApplication.class, args);
	}

}

While we are making these changes to the source code we can run skaffold in development mode. Skaffold will watch the source and rebuild and deploy any changes we make.

Start development mode with:

skaffold dev

To test the newly deployed code we’ll use curl to target the function. Open up a new terminal window and enter:

INGRESS=$(kubectl get svc envoy-external --namespace projectcontour --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')
curl -w'\n' $INGRESS \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H 'Host: my-func.default.example.com' \
 -H "Content-Type: application/json" \
 -d '{"msg":"Testing with POJO"}'

You should see:

HelloWorld [msg=Testing with POJO]

Let’s try to pass in some JSON that doesn’t match the POJO:

INGRESS=$(kubectl get svc envoy-external --namespace projectcontour --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')
curl -w'\n' $INGRESS \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H 'Host: my-func.default.example.com' \
 -H "Content-Type: application/json" \
 -d '{"hello":"world"}'

You should see:

Unable to parse: {"hello":"world"}

Once we are happy with the changes, stop the Skaffold development mode by hitting ^C in that terminal window. You’ll notice that Skaffold now deletes the Container and Deployer resources from the default namespace.

Now we need to commit these changes to our repo and push them to GitHub.

git add src/main/java/com/example
git commit -m "added POJO and conversion from JSON"
git push origin

This change in the GitHub repo should now get picked up by the Application running in the dev namespace and you should see a new build starting. Once that completes we can test that the changes made their way across using:

INGRESS=$(kubectl get svc envoy-external --namespace projectcontour --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')
curl -w'\n' $INGRESS \
 -H "Ce-Id: a8e071e1-4337-4131-98f2-c728982a92ac" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.helloworld" \
 -H "Ce-Source: start.spring.io/my-func" \
 -H 'Host: my-func.dev.example.com' \
 -H "Content-Type: application/json" \
 -d '{"msg":"Testing in dev"}'

You should see:

HelloWorld [msg=Testing in dev]

trisberg/spring-boot-func-devx.adoc

A developer’s experience writing a Spring Boot function for a "Modern App Platform"

Introduction

Getting started developing a function

Initialize a project

Add the function code

Build and test locally

Build and test on k8s

Configure development tools

Set up a CI/CD pipeline

Add your function code to a Git repository on GitHub

Configure Continous Integration (CI)

Create a dev-ops repository

Initialize the repo and push to GitHub

Add your function ops repo to Argo CD

Install Argo CD

Create the Argo CD application for my-func-ops

Time for production

Local feature development

Add marshalling of the payload to domain object