Created
March 27, 2024 15:11
-
-
Save deads2k/b868a1178581c84bdbd0914a2ca9d6a2 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apiVersion | |
kind: NeverCreated | |
metadata | |
annotations: | |
"include.release.openshift.io/ibm-cloud-managed": false | |
spec: | |
featureGateTests: | |
- featureGateName: Example | |
tests: | |
- "[sig-arch][OCPFeatureGate:Example] should only run FeatureGated test when enabled" | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For convenience, you could extract OCPFeatureGate and auto-create capabilities based on it. Code mostly exists for this already | |
Problem: there is no historical basis for new tests. We need to show good or bad anyway. | |
Problem: need to say, "show me the CR table where the testName contains `[OCPFeatureGate:Example]`" | |
These two things be a CR view for an OCPFeatureGate. This allows rapid inspection of "how good is the new feature we're adding". | |
*** This solves the human problem: is this ready to promote and make a clear threshold for managers when they're saying, "David is slowing me down" | |
From this point you could create links in release status, but you could not automatically fail a payload | |
Your URL would probably not age gracefully. Perhaps embedding times in the link? | |
What's still missing | |
This is still missing automated enforcement and how would we do that. | |
Problem: Want to ask component readiness: show me the number of tests written for `[OCPFeatureGate:Example]` and show me the pass rates for them. | |
what do I want to inspect | |
1. do I have at least three tests | |
2. do those tests run on AWS, GCP, Azure, Metal, Vsphere (whatever else we decide) | |
3. do those tests run on the clusterprofile that is being promoted (hypershift or selfmanagedHA) | |
4. don't know what else we'll want later | |
and if the pass for any of those combination is less than 95% over the last two weeks, fail, so the PR won't merge. | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment