Michael Venezia venezia

Summary

It would be great to indicate whether or not a custom resource is intended for general usage as opposed to being a resource used internally to a concern.

Generally speaking, this can help encourage increased custom resource usage while at the same time providing a reasonable user experience.

Background

Kubernetes API as messaging between two concerns

Kubernetes API can be looked as a message bus between two parties:

One party creates an object (custom resource) that describes what they want done, typically in a spec field

Separate etcd CA
Separate etcd peer CA
network isolation of etcd to only talk to api servers
enable secret encryption (experimental feature)
secure Kubernetes cluster via pod tolerations / node selectors / pod security policies as appropriate
don't get tiller more access than it needs

Definitions

customer = a user other than a cluster operations agent
cluster operations = collection of agents that are used to provide a viable cluster

Current Vulnerabilities

Hostpath volume mounts

Hostpath volume mounts are rarely needed by a customer pod. At the same time, kubelet needs read a certificate to authenticate against the cluster. We do not want someone to create a creative pod request that will map in /etc/kubernetes and thus expose to the individual kubelet's certificate. If such occurred, the individual would be able to act as if it was kubelet, which includes reading of any secret stored in the cluster

It is not possible right now to pass in certificates as environmental variables to kubelet, so more could be done later to better secure kubelet's credentials

Introduction

Did you know that you can have multiple helm tillers running in a cluster?
This may actually be useful to us for both security as well as isolating what our cluster ops team does from what our customers can do.

Background

Current Helm Access Control

Helm security is such that "If you can talk to tiller, you can use that tiller" To be able to contact tiller, one needs to be able to create a pod in the namespace tiller resides in. Otherwise you will get an error message like:

	node {
	echo 'Results included as an inline comment exactly how they are returned as of Jenkins 2.121, with $BUILD_NUMBER = 1'
	echo 'No quotes, pipeline command in single quotes'
	sh 'echo $BUILD_NUMBER' // 1
	echo 'Double quotes are silently dropped'
	sh 'echo "$BUILD_NUMBER"' // 1
	echo 'Even escaped with a single backslash they are dropped'
	sh 'echo \"$BUILD_NUMBER\"' // 1
	echo 'Using two backslashes, the quotes are preserved'
	sh 'echo \\"$BUILD_NUMBER\\"' // "1"

	variant: fcos
	version: 1.4.0
	passwd:
	users:
	- name: core
	ssh_authorized_keys:
	- ssh-ed25519 AAAA... somebody@something
	storage:
	files:
	- path: /etc/yum.repos.d/kubernetes.repo

	{
	"configs": {
	"routes": {
	"@type": "type.googleapis.com/envoy.admin.v2alpha.RoutesConfigDump",
	"static_route_configs": [
	{
	"route_config": {
	"name": "inbound\|80\|\|louie.default.svc.cluster.local",
	"virtual_hosts": [
	{