CompositeResourceDefinition.yaml

# A composite resource definition aggregates one or more child
# resources into a single higher level abstraction. The abstraction
# is defined by a CRD and can be consumed as a standard resource. Child
# resources can themselves by composite enabling a hierarchy.
apiVersion: core.crossplane.io/v1alpha1
kind: CompositeResourceDefinition
metadata:
  name: private-mysql-server
spec:
  # Each CompositeResourceDefinition references exactly one CRD that 
  # represents the abstract resource definition. When an CR of this abstract
  # CRD is created, the child resources in this definition are created.
  # The abstractResourceDefinition can be cluster scoped or namespace scoped,
  # see rules below.
  abstractResourceDefinition:
    apiVersion: database.example.org/v1alpha1
    kind: MySQLInstance

  # The scope of *all* child resources in this composite resource definition.
  # Child resources can not be of different scopes, either all namespace
  # scoped or all cluster-scoped.
  scope: Namespaced

  # Supported scopes combinations
  # 
  # Child Resources Scope = Namespaced, Abstract CRD Scope = Namespaced
  #     The abstract CR will have owner references to all its composed child resources.
  # Child Resources Scope = Cluster, Abstract CRD Scope = Cluster
  #     The abstract CR will have owner references to all its composed child resources.
  # Child Resources Scope = Cluster, Abstract CRD Scope = Namespaced
  #     The namespaced scoped abstract CR will have binding relationships to all
  #     its cluster-scoped resources.

  # The array of composed child resources that will be dynamically
  # provisioned to satisfy MySQLInstance claims that use this class. The base must
  # be a valid resource. The patch specifies how the abstract resource's fields may be
  # used to extend or override the base's fields.
  resources:
  - base:
      apiVersion: azure.crossplane.io/v1alpha3
      kind: ResourceGroup
      metadata:
      spec:
        location: West US
        providerRef:
          name: example
        reclaimPolicy: Delete
    patch:
    - fromFieldPath: "spec.region"
      toFiledPath: "spec.forProvider.location"
      transforms:
      - type: map
        map:
          us-west: "West US"
          us-east: "East US"
  - base:
      apiVersion: database.azure.crossplane.io/v1beta1
      kind: MySQLServer
      spec:
        forProvider:
          administratorLogin: myadmin
          # A resource selector allows a resource dynamically provisioned using
          # a particular composite resource class to reference other resources
          # provisioned by the same class. In this case MySQLServer resources
          # provisioned using this class will select the ResourceGroup
          # provisioned by the same class and claim.
          resourceGroupNameSelector:
            matchComposite: true
          location: West US
          sslEnforcement: Disabled
          version: "5.6"
          sku:
            tier: Basic
            capacity: 1
            family: Gen5
          storageProfile:
            storageMB: 20480
        writeConnectionSecretToRef:
          namespace: crossplane-system
        providerRef:
          name: example
        reclaimPolicy: Delete
    patch:
    - fromFieldPath: ".metadata.uid"
      toFiledPath: "spec.writeConnectionSecretToRef.name"
    - fromFieldPath: "spec.engineVersion"
      toFiledPath: "spec.forProvider.version"
    - fromFieldPath: "spec.storageGB"
      toFiledPath: "spec.forProvider.storageMB"
      transforms:
      - type: math
        math:
          multiply: 1024
    - fromFieldPath: "spec.region"
      toFiledPath: "spec.forProvider.location"
      transforms:
      - type: map
        map:
          us-west: "West US"
          us-east: "East US"
  - base:
      apiVersion: database.azure.crossplane.io/v1alpha3
      kind: MySQLServerVirtualNetworkRule
      spec:
        name: my-cool-vnet-rule
        serverNameSelector:
          matchComposite: true
        resourceGroupNameSelector:
          matchComposite: true
        properties:
          virtualNetworkSubnetIdRef:
            name: sample-subnet
        reclaimPolicy: Delete
        providerRef:
          name: azure-provider
 

Owner references are insufficient for this purpose because they exist only from child to parent, making it very difficult to discover the children of a particular resource.

A little more color on this. The two main detractors behind owner references are:

1. Owner references are unidirectional from child to parent. It's very hard, given a parent resource, to discover its children if you do not know their kind in advance. Tools like https://github.com/ahmetb/kubectl-tree solve this by discovering and listing every single resource in a cluster in order to inspect all children and build a tree of references. This may be acceptable for a tool but does not seem like a scalable behaviour for a controller.
2. Owner references leave the decision of whether to garbage collect children up to the entity that deletes the parent. In some cases this entity may be an application operator, and the infrastructure operator may wish to determine what happens to child resources when the parent is deleted.

As I mentioned in the meeting, I am not sure about the value of ResourceDefinition (compared to writing CRD for claim and using a generic CompositionResource as managed resource) because its upside about making it easier to write CRDs is not that big; in the end features array can be predicted by controller to some extent(claim vs managed), user still has to write the rest CRD stuff except for subresources array. From the implementation side, we still work with bare unstructured.Unstructured though it might make it easier to construct a new controller specific to that generated CRD.

In addition to that, I think one of the problems we'll face is making sure that different instances of CompositionDefinition resources correctly specify the fields to be taken from the claim/managed resource in fromFieldPaths and also the connection secret details stay same, i.e. the case where same resource is provisioned but connection secret looks different. In that case, the app author cannot be sure that the secret keys will stay same even though they used that specific resource, SecureDatabase or MySQLInstance.

I was thinking maybe we can expand ResourceDefinition to include those responsibilities so that it acts as source of truth for both app devs and infra ops. For example, we can move all patch and connectionDetails blocks to ResourceDefinition and CompositionDefinition would only include resource parameters, just like how it is today with resource classes. In YAML language:

# Maybe ClaimDefinition(namespaced claims) and ResourceDefinition(cluster-scoped resource) would be separate so that we don't need features array.
apiVersion: crossplane.io/v1alpha1
kind: ResourceDefinition
metadata:
  name: mysqlinstances.database.example.org
spec:
  resources:
  - identifier: resourcegroup
    apiVersion: azure.crossplane.io/v1alpha3
    kind: ResourceGroup
    patch:
    - fromFieldPath: "spec.providerRef.name"
      toFieldPath: "spec.providerRef.name"
    - fromFieldPath: "spec.location"
      toFieldPath: "spec.forProvider.location"
  - identifier: database
    apiVersion: database.azure.crossplane.io/v1beta1
    kind: MySQLServer
    patch:
    - fromFieldPath: "metadata.uid"
      toFieldPath: "spec.writeConnectionSecretToRef.name"
    - fromFieldPath: "spec.engineVersion"
      toFieldPath: "spec.forProvider.version"
    - fromFieldPath: "spec.storageGB"
      toFieldPath: "spec.forProvider.storageMB"
      transforms:
      - type: math
        math:
          multiply: 1024
    - fromFieldPath: "spec.region"
      toFieldPath: "spec.forProvider.location"
      transforms:
      - type: map
        map:
          us-west: "West US"
          us-east: "East US"
    connectionDetails:
    - name: username
      fromConnectionSecretKey: username
    - name: password
      fromConnectionSecretKey: password
    - name: endpoint
      fromConnectionSecretKey: endpoint
  - identifier: mysqlvnetrule
    apiVersion: database.azure.crossplane.io/v1alpha3
    kind: MySQLServerVirtualNetworkRule
  features:
  - composite.crossplane.io
  - connectable.crossplane.io
  - workload.crossplane.io
  crdRef:
    name: mysqlinstances.database.example.org
  # This template is used to create a CRD (if crdRef is unset). Only a single
  # version is supported.
  crdSpecTemplate:
    group: database.example.org
    version: v1alpha1
    names:
      kind: MySQLInstance
      listKind: MySQLInstanceList
      plural: mysqlinstances
      singular: mysqlinstance
    scope: Namespaced
    validation:
      openAPIV3Schema:
        properties:
          apiVersion:
            type: string
          kind:
            type: string
          metadata:
            type: object
          spec:
            properties:
              engineVersion:
                enum:
                - "5.6"
                - "5.7"
                type: string
              storageGb:
                type: integer
              region:
                type: string
            type: object
        type: object

Then we can actually generate 2 CRDs from this ResourceDefinition: MySQLInstance(namespaced claim or cluster-scoped managed resource) and MySQLInstanceClass(cluster-scoped resource class) which has the identifiers above as field names.

# A MySQLInstanceClass CRD is generated with field names given in `ResourceDefinition`.spec.resources[*].identifier
apiVersion: crossplane.io/v1alpha1
kind: MySQLInstanceClass
metadata:
  name: private-mysql-server
  labels:
    example: "true"
spec:
  resouregroup:
    apiVersion: azure.crossplane.io/v1alpha3
    kind: ResourceGroup
    metadata:
    spec:
      location: West US
      providerRef:
        name: example
      reclaimPolicy: Delete
  database:
    apiVersion: database.azure.crossplane.io/v1beta1
    kind: MySQLServer
    spec:
      forProvider:
        administratorLogin: myadmin
        resourceGroupNameSelector:
          matchCompositeIdentifier: resourcegroup # direct identifier
        location: West US
        sslEnforcement: Disabled
        version: "5.6"
        sku:
          tier: Basic
          capacity: 1
          family: Gen5
        storageProfile:
          storageMB: 20480
      writeConnectionSecretToRef:
        namespace: crossplane-system
      providerRef:
        name: example
      reclaimPolicy: Delete
  mysqlvnetrule:
    apiVersion: database.azure.crossplane.io/v1alpha3
    kind: MySQLServerVirtualNetworkRule
    spec:
      name: my-cool-vnet-rule
      serverNameSelector:
        matchCompositeIdentifier: database # direct identifier
      resourceGroupNameSelector:
        matchCompositeIdentifier: resourcegroup # direct identifier
      properties:
        virtualNetworkSubnetIdRef:
          name: sample-subnet
      reclaimPolicy: Delete
      providerRef:
        name: azure-provider

End-user will create the following:

apiVersion: database.example.org/v1alpha1
kind: MySQLInstance
metadata:
  namespace: default
  name: sql
spec:
  # These fields were specified by the crdTemplateSpec.
  engineVersion: "5.7"
  storageGB: 10
  region: us-west
  # This field was injected by the connectable feature. It allows this resource
  # to publish its connection details via a secret.
  writeConnectionSecretToRef:
    name: sql
  # Class selector like today.
  classSelector:
    matchLabels:
      example: "true"
  # Class ref like today.
  classRef:
    name: private-mysql-server
  composed:
  - ref:
      apiVersion: azure.crossplane.io/v1alpha3
      kind: ResourceGroup
      name: default-sql-34jd2
  - ref:
      apiVersion: database.azure.crossplane.io/v1beta1
      kind: MySQLServer
      name: default-sql-3i3d1
  - ref:
      apiVersion: database.azure.crossplane.io/v1alpha3
      kind: MySQLServerVirtualNetworkRule
      name: default-sql-2mdus

In this scenario, when users write a resource class, they can be limited by the validation of the top-level fields under MySQLInstanceClass.spec, i.e. you can only specify 1 resourcegroup, 1 vnetrule and 1 mysqlserver with the given identifiers. If we'd like to, we can even go ahead and fetch the CRD's validation objects with given apiVersion and kind from the api-server(from actual CRDs: MySQLServer, ResourceGroup and MySQLServerVirtualNetworkRule) and embed them into the MySQLInstanceClass.spec fields during CRD generation, which could result in fully validated composition class CRD.

The burden of writing bindings, transformations, fields and their types would be on the one who writes that one ResourceDefinition. Resource class authorship doesn't get harder since it's just multiple known managed resources.
This will solve the problem where different composition classes point to fields that do not exist in the actual resource as well as the case where MySQLInstance type resulting in completely different things that app dev didn't expect because resource classes in this scenario do not decide which kinds, bindings and everything independent from each other.