Experience Report: Building a kubectl plugin for Service Catalog

svcat-plugin-experience-report.md

Experience Report: Building a kubectl plugin for Service Catalog

Why a kubectl plugin?

The goal of the service catalog cli (svcat) is to reduce the learning curve for developers and follow precedent set by kubectl when reasonable so that "finger memory" built-up from using kubectl translates to being able to use svcat without reading the help text much.

It isn't intended to be a replacement for kubectl, and instead users will jump back and forth between kubectl and svcat. As a plugin, it reinforces the mental model that svcat conforms to the way kubectl works, and makes the switch between the two less noticeable.

Here's an example flow:

$ kubectl apply -f minibroker.yaml
clusterservicebroker "minibroker" updated

$ kubectl plugin svcat sync broker minibroker
Synchronization requested for broker: minibroker

$ kubectl plugin svcat describe binding mysqldb --show-secrets
  Name:        mysqldb
  Namespace:   minibroker
  Status:      Ready - Injected bind result @ 2018-04-25 14:56:02 +0000 UTC
  Secret:      mysqldb
  Instance:    mysqldb

Secret Data:
  database              mydb
  host                  loopy-serval-mysql.minibroker.svc.cluster.local
  mysql-password        EUfghi4G3K
  mysql-root-password   q9J2SUhBQe
  password              EUfghi4G3K
  port                  3306
  uri                   mysql://admin:[email protected]:3306/mydb
  username              admin

Plugin Implementation

• The CLI is intended to be used either as a plugin or standalone binary.
• It is a go binary.
• Cobra handles our flag parsing.
• Viper is used in plugin-mode to map the kubectl plugin environment variables back onto our flags.
• k8s.io/kubectl/pkg/pluginutils handles creating a client and config from the kubectl plugin environment variables.
• We support a limited subset of the global kubectl configuration flags (like --context), and the full set (like --server) is supported in plugin mode.

Reflections on Current Plugin Architecture

The current kubectl plugin architecture works for us... to a point. For the most part people avoid using it in plugin mode because the UX isn't as good as standalone mode due to some contraints placed by the current plugin design:

• The plugin prefix, e.g. kubectl plugin svcat, is the #1 reason people don't want to use plugin-mode.
• Lack of support for boolean flags is the #2 reason, forcing weird hacks like --traverse=true instead of the more natural --traverse.

Those are the most obvious UX problems that we have. Otherwise we spend a bit of effort trying to replicate behaviors of kubectl that aren't easily reusable, leading to minor differences in output/behavior:

• We fully support flags and use cobra but because kubectl doesn't pass the flags to us, we have ugly hacks using spf13/viper to bind those environment variables back to our flags.
• We hoped that as part of being a plugin, there would be existing libraries to assist with common tasks, like creating a k8s client from the config flags. That didn't exist so we (jberkhahn) wrote k8s.io/kubectl/pkg/pluginutils.
• We try to format our get and describe output like kubectl, but since it's not exposed, we reproduce it using other libraries and have been slowly duplicating print logic similar to kubectl.
• We have reimplemented functionality in kubectl, like -o json|yaml but even so it's not a 100% match because kubectl has extra logic (unexported) that rehydrates the TypeMeta appropriately. Though to be fair, that really should be handled by k8s.io/client-go.

What would svcat would look like as a "git style" plugin?

There has been talk of moving towards a "git style" plugin model, so let's think about what that would look like for svcat. Our commands would mostly look the same, though thankfully the "plugin" subcommand would go away.

$ kubectl svcat get brokers

Unfortunately, it appears that we would lose all the extra help we get right now from kubectl figuring out the current namespace based on the context and flags, or help parsing global flags like --server that our standalone binary doesn't support.

Ideal svcat plugin

As long as I've tricked people into reading this, here is an outline of our ideal kubectl plugin experience.

Register custom nouns for existing verbs

Ideal UX

# List brokers in the current namespace
$ kubectl get brokers

# List all brokers across all namespaces and at the cluster level
$ kubectl get brokers --all-namespaces --cluster

Current UX

# List brokers in the current namespace
$ kubectl get servicebrokers
$ kubectl svcat get brokers

# List all brokers across all namespaces and at the cluster level
$ kubectl get clusterservicebrokers,servicebrokers
$ kubectl svcat get brokers --all-namespaces --cluster

The brokers resource is an alias, service catalog has clusterservicebrokers and servicebrokers. Those are a) too long and b) represent essentially the same thing "a broker", but one is scoped to the cluster, and the other to a namespace. Forcing user to interact with them separately is a non-starter.

Note: I am showing the "pure" kubectl experience to highlight why we are replicating existing kubectl functionality, such as get, in svcat.

Register custom verbs

Ideal UX

# Deprovision a database
$ kubectl deprovision mysqldb --unbind

Current UX

# Deprovision a database
$ kubectl plugin svcat deprovision mysqldb --unbind==true

In this scenario, I have a verb that is unique to a noun (instance), and so the noun is omitted.

Take over an existing verb+noun

Ideal UX

# Show a binding, and its associated decoded secret
$ kubectl describe binding mysqldb --show-secrets

Current UX

# Show a binding, and the user looks-up the assocated secret name
$ kubectl describe servicebinding mysqldb
# Decode the associated secret, for each key in the secret
$ kubectl get secret mysqldb-root -o=jsonpath='{.data.uri}' | base64 --decode

# Show a binding, and its associated decoded secret
$ kubectl plugin svcat describe binding mysqldb --show-secrets=true

Not only would we like to format the binding differently than kubectl, but we perform extra api calls to retrieve associated resources and display them as well. This also involves a custom flag, --show-secrets on an existing kubectl command.

Plugin Library

As a plugin developer, there are things that I am interseted in solving, like my domain-specific problems, and then there are things that I would prefer to consume in a library:

• Parsing global kubectl configuration flags, like --context, and --server.
• Turning those flags into a config and client.
• Print functions so that my output can look like kubectl's.
• Output functions so that I can support -o json|yaml on my commands.
• Plugin functions that assist with generating the plugin manifest, understanding when I am running as a kubectl plugin, etc.
• Glog helper functions so that I don't need to do ugly hacks to get verbose output written to stdout.