How to deploy Lithops Multicloud with Apache Openwhisk Python 3 to Kubernetes

lithops-openwhisk.md

How to deploy Lithops Multicloud with Apache Openwhisk Python 3 to Kubernetes

This explains how to set up your serverless (FaaS) environment to run Lithops multicloud library. This allows you to avoid vendor lock-in and offboard your serverless code from cloud services.

Reset kubernetes

foo@bar$ export KUBECONFIG=
foo@bar$ minikube stop
foo@bar$ minikube delete --all
foo@bar$ rm -rf ~/.kube
foo@bar$ eval $(docker-machine env -u)
foo@bar$ clear; reset
# Turn on kubernetes
foo@bar$ minikube start

Add openwhisk repository to helm

foo@bar$ helm repo add openwhisk https://openwhisk.apache.org/charts
foo@bar$ helm repo update

Create OpenWhisk cluster config file

foo@bar$ vim mycluster.yaml

and the contents:

controller:
  replicaCount: 2
whisk:
  ingress:
    type: NodePort
    apiHostName: <CLUSTER_IP>
    apiHostPort: 31001
k8s:
  persistence:
    enabled: false
nginx:
  httpsNodePort: 31001
invoker:
  containerFactory:
    impl: "kubernetes"
metrics:
  prometheusEnabled: true
metrics:
  userMetricsEnabled: true

Install openwhisk

foo@bar$ helm install demo openwhisk/openwhisk -f mycluster.yaml --set whisk.ingress.apiHostName=$(minikube ip)

# Configure deployment and open networking
# add invoker role for spinning up new resources
foo@bar$ kubectl label nodes --all openwhisk-role=invoker 
# Kafka needs hairpin mode, communicates with localhost
foo@bar$ minikube ssh 'sudo ip link set docker0 promisc on' 
foo@bar$ kubectl get all
foo@bar$ brew install watch # Yes I am on MacOS
foo@bar$ watch -n 1 kubectl get deploy,replicaset,po

Build OpenWhisk Python 3 docker environment for serverless actions

foo@bar$ eval $(minikube docker-env)
foo@bar$ git clone https://github.com/apache/openwhisk-runtime-python
foo@bar$ docker build -t actionloop-python-v3.7:1.0-SNAPSHOT $(pwd)/openwhisk-runtime-python/core/python3ActionLoop

Add custom dependencies to new OpenWhisk image

foo@bar$ vim ./Dockerfile

and the contents:

FROM actionloop-python-v3.7:1.0-SNAPSHOT
RUN pip3 install 'lithops==2.3.3'

foo@bar$ docker build . --tag multicloud_py3:1.0

Create hello world application FaaS

foo@bar:~$ vim main.py

and the contents:

from lithops import Storage

config = {'lithops' : {'storage_bucket' : 'my_bucket_name',
                       'storage' : 'redis'},
          'redis':  {'host' : 'demo-redis',
                     'port' : '6379',
                     'password' : ''}}

def main(payload):
    storage = Storage()
    storage.put_object(bucket='my_bucket_name',
                       key='my_key_123',
                       body='Hello {}!')
    result = storage.get_object(bucket='my_bucket_name',
                                key='my_key_123')
    name = payload['name']
    greeting = str(result, 'utf-8').format(name)
    return {"body" : {"greeting" : greeting}}

Configure wsk command line tool

foo@bar$ brew install wsk
# You will need another terminal for this
foo@bar$ minikube service demo-nginx --url
    http://127.0.0.1:51080 <= http
    https://127.0.0.1:51443 <= https, remember this!
foo@bar$ helm status demo
# configure wsk cli using minikube output
foo@bar$ wsk property set --apihost 127.0.0.1:51541
# default credential for guest account
foo@bar$ wsk property set --auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP 

Deploy and run the cloud function

Note: --web parameter publishes the public api to /api/v1/web

call the development endpoint

foo@bar$ wsk -i action create helloWorld --docker multicloud_py3:1.0 main.py --web true --main main
foo@bar$ wsk -i action list
foo@bar$ wsk -i -v action invoke helloWorld --result --param name World

output:

{
    "body": {
        "greeting": "Hello World!"
    }
}

Call the published endpoint

foo@bar$ wsk -i activation logs [X-Openwhisk-Activation-Id] # optional debugging step
foo@bar$ wsk -i action get helloWorld --url
foo@bar$ curl -k 'https://127.0.0.1:56504/api/v1/web/guest/default/helloWorld' -d '{"name":"World"}' -H 'content-type: application/json'

output:

{
  "greeting": "Hello World!"
}

Cleanup

foo@bar$ wsk -i action delete helloWorld
foo@bar$ helm delete demo