Generic conditions, patch, simple reconciler packages in Controller-Runtime

README.md

Generic conditions, patch, simple reconciler packages

The branch feature/generic-controller leverages generics from Golang 1.18 to support the following enhancements to Controller-Runtime:

• Port the conditions and patch utilities from Cluster API into Controller-Runtime in such a way they are reusable with any API's types
• Introduce a new, simple reconciler type to make it even easier for people who want to write a controller using Controller-Runtime

New packages

There are four, new packages:

Package	Description
./pkg/conditions/	For getting, setting, and patching status conditions
./pkg/patch/	For patching resources both with and without status conditions
./pkg/reconcile/simple/	For building a simple reconciler that automatically gets the resource from the API server and patches the resource and its optional status subresource prior to returning from the Reconcile function
./examples/simple/	A working example of all of the above that creates two controllers using simple reconcilers for a new Dog API and the built-in Node API

Build a simple controller

Let's first look at the new, simple reconciler from ./pkg/reconcile/simple/. Imagine for a moment there is a variable named mgr that references a Controller-Runtime Manager. Creating a new controller for the Dog API would now be as easy as the following:

ctrl.NewControllerManagedBy(mgr).
	For(&Dog{}).
	Complete(&simple.Reconciler[*Dog]{
		Client: mgr.GetClient(),
		OnNormal: func(
			ctx context.Context, 
			obj *Dog) (ctrl.Result, error) {

			// Update the object's spec, status,
			// and even status conditions

			return ctrl.Result{}, nil
		},
	},
})

Then above controller will:

1. Watch Dog resources
2. Get a Dog resource
3. Patch the Dog resource and its status subresource, including conditions

The notation Reconciler[*Dog] is how explicit type information is indicated to Go types that use generics. For example, the new simple.Reconciler type is defined as:

type Reconciler[TObject client.object] struct

This means any object that can satisfy the constraint client.Object may be used to instantiate the simple reconciler. Once compiled, the binary artifact would actually include the following type:

Reconciler[*Dog]

Build a more intelligent, simple controller

The ./pkg/reconcile/simple package also provides a second type of reconciler:

type ReconcilerWithConditions[
    TObject conditions.Setter[
        TConditionType,
        TConditionSeverity,
        TCondition,
    ],
    TConditionType,
    TConditionSeverity,
    TCondition[
        TConditionType,
        TConditionSeverity,
    ],
] struct

The ReconcilerWithConditions type makes it possible to create an even more intelligent controller while still retaining the simplicity of the previous example. Imagine the following:

• the Dog API implements the Getter and Setter interfaces from the new ./pkg/conditions/ package
• the Dog API includes the following types:

  Type	Description
  Condition	surfaced as part of the Dog API's status.conditions field and satisfies the Condition constraint from ./pkg/conditions/
  ConditionType	expresses the type for a Dog API's Condition and satisifies the Type constraint from ./pkg/conditions/
  ConditionSeverity	expresses the severity for a Dog API's Condition and satisifies the Severity constraint from ./pkg/conditions/

With this knowledge it is possible to instantiate a new ReconcilerWithConditions like so:

ctrl.NewControllerManagedBy(mgr).
	For(&Dog{}).
	Complete(&simple.ReconcilerWithConditions[
		*Dog,
		ConditionType,
		ConditionSeverity,
		*Condition,
	]{
		Client: mgr.GetClient(),
		OnNormal: func(
			ctx context.Context, 
			obj *Dog) (ctrl.Result, error) {

			// Update the object's spec, status,
			// and even status conditions

			return ctrl.Result{}, nil
		},
	},
})

Much like the previous example, the above controller will:

1. Watch Dog resources
2. Get a Dog resource
3. Patch the Dog resource and its status subresource, including conditions

However, the difference this time how the conditions are patched. This patchset ports the conditions and patch helper logic from Cluster API into Controller-Runtime. This was not possible in the past due to the Cluster API conditions and patch logic relying on a known Condition type. With Go generics it is possible to implement the same logic using constraints for expressing what is and is not a condition along with its type and severity.

Conditions and patch helpers

With the new ./pkg/conditions/ and ./pkg/patch/ packages, anything that satifies the constraint conditions.Setter will benefit from enhanced patch logic around a resource's status conditions, including:

• A three-way merge of a resource's status conditions
• Attempts to resolve conflicts automatically using merge priority based on severity
• The ability to prefer conditions owned/set by a controller
• And more!

Like the simple reconcilers, the patch helper comes in two variants:

Type	Description
patch.Helper	Patches a resource and its status subresource, including conditions
patch.HelperWithConditions	Patches a resource and its status subresource, including conditions using the aforementioned patch logic

The first helper may be used with any Kubernetes API, even the built-in types such as Node. The patch.Helper may also be used with APIs that are eligible for patch.HelperWithConditions, but when doing so, the status conditions will be treated as any other field from the status subresource.

There is also ./examples/simple -- a new example for how to use all of the above lessons to produce controllers for the built-in Node resource and a new Dog resource that leverages the enhanced conditions logic.

Testing

Finally, this patch also includes 124 tests to validate that everything works as it should:

$ git show | \
  grep -c \
  '\(func Test\)\|\(\(It\|Specify\)(\)\|\(name: \{1,\}"\)' 
124

👏👏👏

nit: Typo Wtch

Can you explain the rational for OnNormal()? Never seen this wording before.

Why do you need the call to .For() when the type is part of the reconciler's setup?

Why do you need the call to .For() when the type is part of the reconciler's setup?

I suppose technically you do not, but I did not care to take this patch that far. I could be wrong, but I think you would have to introduce generics into the type registry to obviate the need for the For call.

Really good point though...I bet I could do it, but it would require a separate builder or entrance point into the controller standup logic. One thing I think is apparent in the above write-up is that you have to go all-in on generics, or you end up duplicating existing types to maintain compile-time, backwards compatibility with older code patterns.

Hi @stevekuznetsov,

Just looking at the CR code, I think you could do this:

// ControllerManagedByT returns a new controller builder that watches
// objects of type T and will be started by the provided Manager.
func ControllerManagedByT[T client.Object](m manager.Manager) *Builder {
	b := &Builder{mgr: m}
	t := reflect.New(reflect.TypeOf(*(new(T))).Elem()).Interface().(client.Object)
	return b.For(t)
}

Hi @stevekuznetsov,

I traced the code, and this is an example of when generics would not help so much. The CR builder:

• has the For call, which
• sets up a forInput
• which maintains a reference to the client.Object
• which is used by bldr.doWatch
• which is used to initialize a handler.EnqueueRequestForOwner as the OwnerType field
• which is used by the Scheme to get the GVK from the typeToGVK map

All this could be replaced by a pure type, but for the Scheme's support for unstructured objects. For that you need the actual object, not just the type.

Hi @embano1,

Can you explain the rational for `OnNormal()? Never seen this wording before.

The standard pattern for a CR-based reconciler is:

• Enter Reconcile call
  • Get resource
  • Create a patch helper
  • Set up a defer to patch the resource upon exiting the Reconcile call
  • Check the resource's deletion timestamp:
    • Has a non-zero deletion timestamp:
      • Reconcile a resource being deleted
    • Does not have a non-zero deletion timestamp:
      • Reconcile a resource normally
  • Patch the resource and its status subresource
• Exit the Reconcile call

The two bits from the above that are relevant are:

• Reconcile a resource being deleted
• Reconcile a resource normally

These functions are normally defined on a Reconciler as ReconcileDelete and ReconcileNormal. To allow people that use simple.Reconciler to provide their own implementations of this logic, I defined callback functions called OnDelete and OnNormal.

I'm confused by the use of generics for conditions - so I feel I'm missing some context

Complete(&simple.ReconcilerWithConditions[
	*Dog,
	ConditionType,
	ConditionSeverity,
	*Condition,
]{

Hi @dprotaso,

The Cluster API conditions package/util/logic is based on the following type:

// Condition defines an observation of a Cluster API resource operational state.
type Condition struct {
	// Type of condition in CamelCase or in foo.example.com/CamelCase.
	// Many .condition.type values are consistent across resources like Available, but because arbitrary conditions
	// can be useful (see .node.status.conditions), the ability to deconflict is important.
	Type ConditionType `json:"type"`

	// Status of the condition, one of True, False, Unknown.
	Status corev1.ConditionStatus `json:"status"`

	// Severity provides an explicit classification of Reason code, so the users or machines can immediately
	// understand the current situation and act accordingly.
	// The Severity field MUST be set only when Status=False.
	// +optional
	Severity ConditionSeverity `json:"severity,omitempty"`

	// Last time the condition transitioned from one status to another.
	// This should be when the underlying condition changed. If that is not known, then using the time when
	// the API field changed is acceptable.
	LastTransitionTime metav1.Time `json:"lastTransitionTime"`

	// The reason for the condition's last transition in CamelCase.
	// The specific API may choose whether or not this field is considered a guaranteed API.
	// This field may not be empty.
	// +optional
	Reason string `json:"reason,omitempty"`

	// A human readable message indicating details about the transition.
	// This field may be empty.
	// +optional
	Message string `json:"message,omitempty"`
}

Please note the Type and Severity fields are not built-in types but rather custom ones per Kubernetes API recommendations. Many other implementations of a Conditions-like type behave the same way.

Anyway, because the ported patch logic makes intelligent decisions about status conditions, it is necessary to use generics to allow callers to specify the:

• Go type of Condition.Type
• Go type of Condition.Severity
• Go type of Condition

Please note a lot of this would be simpler if there was a standard Go types for Condition, ConditionType, and ConditionSeverity, but metav1.Condition uses a string for its Type field and does not include severity at all. Many implementations of conditions do include severity.

As I am 100% against an API package linking to Controller-Runtime for a type as part of a custom API, I did not want to define a standard types here either. That is why I used generics to express them.

These functions are normally defined on a Reconciler as ReconcileDelete and ReconcileNormal. To allow people that use simple.Reconciler to provide their own implementations of this logic, I defined callback functions called OnDelete and OnNormal.

Thx, I always used one implementation/callback for both. But makes sense.