After working on the test suite for a good amount of time and getting help from fellow folks. These are some issues I'm thinking of proposing to make the suite better. During our last retro, we have already talked about running smoke tests.
Some Proposals about proceeding further with current CI/CD infrastructure.
The control Plane:
- Controller, Builder, Registry, Database, Minio Any changes made to these components or Deis Cli or controller-sdk-go repository will affect workflow functionality and should run the full test suite.
The Logging and Monitoring stack:
- Fluentd, Logger, Grafana, NSQ, InfluxDB.
A Pr change to this stack will not affect any CLI functionality or workflow features that are related to control plane except
Deis logs
sub-components:
- Slugbuilder, slugrunner, dockerfilebuilder A pr change to these will only affect a few tests in the workflow e2e and instead of running entire test suite and git push with different app types test should be sufficinet for testing.
- Each component in the architecture should have it's own dedicated test suite and clusters.
- For example :
- control plane should have a helm chart or manifests, which will install the control plane components on to Deis namespace. The cluster should already be running Logging and monitoring stack.
- Logging and monitoring stack should have it's own dedicated cluster where control plane is already present and helm chart or manifests are needed to install this stack on to existing control plane in Deis namespace.
- subcomponents like slugbuilder, slugrunner and dockerfile builder etc., should run specific tests in e2e or should have a dedicated test suite that would run specific tests to check if these components are working.
- We should also look at installing Deis in different namespaces withing the same cluster. This might require some code change in each component but would save cluster costs. Not sure how to proceed here. comments and discussions are encouraged.
I think this is great. As a general, potentially wide-reaching proposal/discussion topic, this presents a ton of good ideas. My only reservation is if we are bringing up too many new ideas/discussion points and whether or not we limit the scope. That being said, maybe we put this up as a proposal and see what sticks/what falls out and actual work tickets can be generated from that.