Description of Pkger Stacks

Pkger: the What and How

Responsibilities

Translating a declarative package file (either JSON | Yaml | Jsonnet) into resources in the platform
Exporting existing resources in the form of a pkg (either JSON | Yaml)
Managing the state of a pkg's side effects via a stack

Anatomy of a package

A package is a collection of resource configurations. These resource configurations can be seen in full in the pkger/testdata directory. The package itself does have any state. Packages may contain resources that are not uniquely identifiable within the platform and some that are not. If it is desired to use packages in a gitops scenario or in a manner that requires all resources are not duplicated, you will want to explore using a stack.

Stacks

A stack is a stateful entity for which packages can be applied and managed. A stack uses a combination of a resource's kind and metadata.name fields to uniquely identify a resource inside a package and map that to state in the platform. Via this state, a stack provides the ability to apply a package idempotently. Stack's manage the full lifecyle of a package's resources, including creating, updating, and deleting resources.

Packages may contain resources that are not uniquely identifiable within the platform. For instance, a dashboard resource, does not have any unique identifier within the platform beyond its UUID. A stack uses the metadata.name field to uniquely identify a resource inside a package and map that to state in the platform.

Stacks will manage the following use cases:

We create a stack without any URLs to packages, henceforth identified as S1:

# S1 package - initial
kind: Label
metadata:
	name: lucid_einstein
spec:
	name: label_1
---
kind: Bucket
metadta: 
	name: pristine_noir
spec: 
	name: bucket_1
	association:
	 	- kind: Label
		   name: lucid_einstein
---
kind: Dashboard
metadata:
	name: charmed_saratoba
spec:
	name: dash_1
	association:
	 	- kind: Label
		   name: lucid_einstein

The S1 package (seen above) with all new resources is applied

Side effects: all resources are created and a record of all resources (id, res type, etc) is added to the S1 stack record

Stack Record

   {
     "stack_id": S1_UUID,
     "createdAt": CreatedAtTimestamp,
     "updatedAt": CreatedAtTimestamp,
     "config": {},
     "resources": [
       {
         "kind": "Label",
         "id": LABEL_UUID,
         "pkgName": "lucid_einstein"
       },
       {
         "kind": "Bucket",
         "id": BUCKET_UUID,
         "pkgName": "pristine_noir",
         "associations": [
           {
             "kind": "Label",
             "pkgName": "lucid_einstein"
           }
         ]
       },
       {
         "kind": "Dashboard",
         "id": DASHBOARD_UUID,
         "pkgName": "charmed_saratoba",
         "associations": [
           {
             "kind": "Label",
             "pkgName": "lucid_einstein"
           }
         ]
       }
     ]
   }

Same S1 package (seen above) is reapplied with no changes from step 1

Side effects: nothing, no changes

Stack Record

  {
    "stack_id": S1_UUID,
    "createdAt": CreatedAtTimestamp,
    "updatedAt": CreatedAtTimestamp,
    "config": {},
    "resources": [
      {
        "kind": "Label",
        "id": LABEL_UUID,
        "pkgName": "lucid_einstein"
      },
      {
        "kind": "Bucket",
        "id": BUCKET_UUID,
        "pkgName": "pristine_noir",
        "associations": [
          {
            "kind": "Label",
            "pkgName": "lucid_einstein"
          }
        ]
      },
      {
        "kind": "Dashboard",
        "id": DASHBOARD_UUID,
        "pkgName": "charmed_saratoba",
        "associations": [
          {
            "kind": "Label",
            "pkgName": "lucid_einstein"
          }
        ]
      }
    ]
  }

# S1 package - updated label name
kind: Label
metadata:
	name: lucid_einstein
spec:
	name: cool label name  #<<<<<< THIS NAME CHANGES
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes

The S1 package is applied with an update to the label resource

Side effects: platform label (LABEL_UUID) is renamed and updatedAt field in S1 record is updated

Stack Record

  {
       "stack_id": S1_UUID,
       "createdAt": CreatedAtTimestamp,
       "updatedAt": LABEL_UPDATE_TIMESTAMP,
       "config": {},
       "resources": [
         ... snip, all resoruces are same
       ]
  }

# S1 package - new reosource added
kind: Label
metadata:
	name: lucid_einstein
# snip - no change
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes
---
kind: Task  #<<<<<< THIS RESOURCE IS ADDED
metadata:
	name: alcord_mumphries
spec:
	name: task_1
	association:
	 	- kind: Label
		   name: lucid_einstein

The S1 package is applied with a new resource added

Side effects: new task is created and S1 record is updated

Stack Record

  {
       "stack_id": S1_UUID,
       "createdAt": CreatedAtTimestamp,
       "updatedAt": TASK_ADD_TIMESTAMP,
       "config": {},
       "resources": [
         ... snip, all resoruces from before,
         {
         	"kind": "Task",
         	"id": TASK_UUID,
   				"pkgName": "alcord_mumphries",
           "associations": [
             {
               "kind": "Label",
               "pkgName": "lucid_einstein"
             }
           ]
         }
       ]
  }

# S1 package - task resource is removed
kind: Label
metadata:
	name: lucid_einstein
# snip - no change
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes

The S1 package is applied with changes that removes an existing resource

Side effects: task is deleted from platform and S1 record is updated

Stack Record

  {
      "stack_id": S1_UUID,
      "createdAt": CreatedAtTimestamp,
      "updatedAt": TASK_DELETE_TIMESTAMP,
      "config": {},
      "resources": [
        {
         "kind": "Label",
         "id": LABEL_UUID,
         "pkgName": "lucid_einstein"
       },
       {
         "kind": "Bucket",
         "id": BUCKET_UUID,
         "pkgName": "pristine_noir",
         "associations": [
           {
             "kind": "Label",
             "pkgName": "lucid_einstein"
           }
         ]
       },
       {
         "kind": "Dashboard",
         "id": DASHBOARD_UUID,
         "pkgName": "charmed_saratoba",
         "associations": [
           {
             "kind": "Label",
             "pkgName": "lucid_einstein"
           }
         ]
       }
      ]
  }

# S1 package - label and associations to it are removed
kind: Bucket
metadta: 
	name: pristine_noir
spec: 
	name: bucket_1
---
kind: Dashboard
metadata:
	name: charmed_saratoba
spec:
	name: dash_1

The S1 package is apllied with label and associations to that label removed

Side effects: label and all label assocations for that label are removed from the platform and S1 record is updated

Stack Record

 {
     "stack_id": S1_UUID,
     "createdAt": CreatedAtTimestamp,
     "updatedAt": Label_DELETE_TIMESTAMP,
     "config": {},
     "resources": [
      {
        "kind": "Bucket",
        "id": BUCKET_UUID,
        "pkgName": "pristine_noir",
        "associations": []
      },
      {
        "kind": "Dashboard",
        "id": DASHBOARD_UUID,
        "pkgName": "charmed_saratoba",
        "associations": []
      }
     ]
 }

jsteenb2/pkger_stack_use_cases.md

Anatomy of a package

Stacks

Stacks will manage the following use cases: