Skip to content

Instantly share code, notes, and snippets.

@david-martin

david-martin/_files_review.txt

Last active November 27, 2019 14:54
Show Gist options
  • Save david-martin/71d8dcc1cf8405d0e9ef5a5a47012af2 to your computer and use it in GitHub Desktop.
Save david-martin/71d8dcc1cf8405d0e9ef5a5a47012af2 to your computer and use it in GitHub Desktop.
Download ZIP
`git diff v1.5 master` `git diff v1.5 master --name-status --oneline`
Raw
_files_review.txt
These are all non code changes and can be dismissed.
M .github/PULL_REQUEST_TEMPLATE.md
A CHANGELOG.md
M README.adoc
M docs/customisation.md
A docs/monitoring/customisation.md
A docs/monitoring/grafana-endpoints-detailed.png
A docs/monitoring/prometheus-alert-working.png
A docs/monitoring_docs.asciidoc
Review for any recent changes to the release process.
M docs/release.md
M scripts/release.sh
Review these changes to ensure there are no suprises with default values changing or unexpected version chanages
M inventories/group_vars/all/manifest.yaml
M inventories/group_vars/poc/poc.yml
Review all install related playbooks to ensure there are no unexpected things being installed in new clusterd, and if they are, there are upgrades in place.
Review all ad-hoc playbooks (e.g. cve_rollout) to ensure they are not referenced by install or upgrade. In that case they can be dismissed.
Review all upgrade playbook changes to ensure they are only relevant for 1.5.2 to 1.6.0, and there are no remnants from a previous release.
A playbooks/cve_rollout.yml
M playbooks/generate-customisation-inventory.yml
A playbooks/group_vars/all/cve.yml
A playbooks/group_vars/all/upgrade.yml
M playbooks/install.yml
M playbooks/install_backups.yml
M playbooks/install_middleware_monitoring_config.yml
D playbooks/install_mobile_middleware_monitoring_config.yml
M playbooks/install_mobile_services.yml
M playbooks/install_services.yml
M playbooks/mobile_generate_manifest.yml
M playbooks/uninstall.yml
M playbooks/uninstall_mobile.yml
M playbooks/update_resources.yml
A playbooks/upgrade.yml
M playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml
D playbooks/upgrades/install_user_rhsso.yml
M playbooks/upgrades/upgrade.yaml
Should be covered by testing of new Integration Product versions
M roles/3scale/defaults/main.yml
D roles/3scale/tasks/_wt2_route.yml
M roles/3scale/tasks/install.yml
M roles/3scale/tasks/new_limits.yml
D roles/3scale/tasks/routes.yml
D roles/3scale/tasks/upgrade.yml
A roles/3scale/tasks/upgrade_images.yml
M roles/enmasse/defaults/main.yml
M roles/enmasse/tasks/new_limits.yml
R059 roles/enmasse/tasks/upgrade.yml roles/enmasse/tasks/upgrade_images.yml
M roles/fuse/defaults/main.yml
M roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml
M roles/fuse/tasks/main.yml
M roles/fuse/tasks/uninstall.yml
M roles/fuse/tasks/upgrade.yml
A roles/fuse/tasks/upgrade_images.yml
M roles/fuse_managed/defaults/main.yml
M roles/fuse_managed/tasks/main.yml
A roles/fuse_managed/tasks/new_limits.yml
M roles/fuse_managed/tasks/uninstall.yml
M roles/fuse_managed/tasks/upgrade.yml
A roles/fuse_managed/tasks/upgrade_images.yml
M roles/fuse_managed/templates/syndesis-customresource.yml.j2
Looks like a dir rename for the most part, but please review for any actual code changes
D roles/code-ready/tasks/upgrade.yml
R074 roles/code-ready/defaults/main.yml roles/codeready/defaults/main.yml
R100 roles/code-ready/tasks/backup.yml roles/codeready/tasks/backup.yml
R077 roles/code-ready/tasks/download_installer.yml roles/codeready/tasks/download_installer.yml
R100 roles/code-ready/tasks/install.yml roles/codeready/tasks/install.yml
R100 roles/code-ready/tasks/keycloak-client.yml roles/codeready/tasks/keycloak-client.yml
R100 roles/code-ready/tasks/main.yaml roles/codeready/tasks/main.yaml
A roles/codeready/tasks/new_limits.yml
R100 roles/code-ready/tasks/uninstall.yml roles/codeready/tasks/uninstall.yml
R100 roles/code-ready/tasks/upgrade_1.0_to_1.2.yml roles/codeready/tasks/upgrade_1.0_to_1.2.yml
A roles/codeready/tasks/upgrade_images.yml
R100 roles/code-ready/templates/config.yaml roles/codeready/templates/config.yaml
R100 roles/code-ready/templates/keycloak/client.json roles/codeready/templates/keycloak/client.json
Review all mdc & mss references to ensure all are removed (and upgrade playbook removes resources from existing clusters)
M roles/mdc/defaults/main.yml
M roles/mdc/tasks/install-operator.yml
M roles/mdc/tasks/monitoring.yml
A roles/mdc/tasks/new_limits.yml
M roles/mdc/tasks/uninstall.yml
A roles/mdc/tasks/upgrade_images.yml
M roles/mdc/templates/mdc_prometheus_rules.yaml.j2
M roles/mdc/templates/operator_prometheus_rules.yaml.j2
D roles/mobile_security_service/OWNERS
M roles/mobile_security_service/defaults/main.yml
D roles/mobile_security_service/tasks/backup.yml
D roles/mobile_security_service/tasks/main.yml
D roles/mobile_security_service/tasks/monitoring.yml
D roles/mobile_security_service/tasks/upgrade.yml
D roles/mobile_security_service/templates/backup_cr.yml.j2
D roles/mobile_security_service/templates/cluster_role_binding.yml.j2
D roles/mobile_security_service/templates/mss_grafana_dashboard.yml.j2
D roles/mobile_security_service/templates/mss_operator_grafana_dashboard.yml.j2
D roles/mobile_security_service/templates/mss_operator_prometheus_rule.yml.j2
D roles/mobile_security_service/templates/mss_prometheus_rule.yml.j2
D roles/mobile_security_service/templates/operator.yml.j2
Check if the version of SSO or operator has changed, or if this is *only* related to the upgrade_resources playbook and setting limits in an existing cluster ad-hoc.
M roles/rhsso-user/defaults/main.yml
A roles/rhsso-user/tasks/new_limits.yml
D roles/rhsso-user/tasks/upgrade.yaml
M roles/rhsso/defaults/main.yml
A roles/rhsso/tasks/new_limits.yml
M roles/rhsso/tasks/upgrade.yaml
A roles/rhsso/tasks/upgrade_images.yml
Check if the version of Apicurito or operator has changed, or if this is *only* related to the upgrade_resources playbook and setting limits in an existing cluster ad-hoc.
M roles/apicurito/defaults/main.yml
A roles/apicurito/tasks/new_limits.yml
A roles/apicurito/tasks/upgrade_images.yml
Review to ensure upgrade playbook applies all monitoring resources during upgrade.
Alert testing & Dashboard check in test plan should cover any issues introduced by this.
Good to check that any new dashboards or alerts show up OK.
M roles/middleware_monitoring/defaults/main.yml
M roles/middleware_monitoring/tasks/main.yml
A roles/middleware_monitoring/tasks/new_limits.yml
M roles/middleware_monitoring/tasks/uninstall.yml
M roles/middleware_monitoring/tasks/upgrade/grafana.yml
M roles/middleware_monitoring/tasks/upgrade/prometheus.yml
M roles/middleware_monitoring/tasks/upgrade/trigger.yml
A roles/middleware_monitoring/tasks/upgrade_images.yml
M roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2
M roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2
M roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2
M roles/middleware_monitoring_config/defaults/main.yml
A roles/middleware_monitoring_config/tasks/create_alertmanager.yml
R100 roles/middleware_monitoring_config/tasks/kube_state_metrics_alerts.yml roles/middleware_monitoring_config/tasks/create_alerts.yml
M roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml
M roles/middleware_monitoring_config/tasks/main.yml
M roles/middleware_monitoring_config/tasks/upgrade.yml
A roles/middleware_monitoring_config/templates/alertmanager.yml.j2
M roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2
A roles/middleware_monitoring_config/templates/kube_state_metrics_3scale_alerts.yml.j2
M roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2
A roles/middleware_monitoring_config/templates/kube_state_metrics_fuse_online_alerts.yml.j2
M roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2
M roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2
Review to ensure these changes are only related to Integration product
M roles/msbroker/defaults/main.yml
M roles/msbroker/tasks/apply_msbroker_template.yml
A roles/msbroker/tasks/upgrade_images.yml
Check if UPS version being updated from 1.5.2 to 1.6.0?
M roles/ups/defaults/main.yml
M roles/ups/tasks/install-operator.yml
M roles/ups/tasks/monitoring.yml
A roles/ups/tasks/new_limits.yml
A roles/ups/tasks/upgrade_images.yml
M roles/ups/templates/operator.yml.j2
M roles/ups/templates/operator_prometheus_rule.yml.j2
M roles/ups/templates/prometheus_rule.yml.j2
Check if webapp & walkthroughs are being updated from 1.5.2 to 1.6.0
M roles/walkthroughs/templates/crud_spboot_example.yml
M roles/webapp/defaults/main.yml
A roles/webapp/tasks/new_limits.yml
M roles/webapp/tasks/upgrade.yaml
A roles/webapp/tasks/upgrade_images.yml
Check if launcher version changed in 1.6.0, and if there are upgrade changes needed
M roles/launcher/defaults/main.yml
A roles/launcher/tasks/new_limits.yml
A roles/launcher/tasks/upgrade_images.yml
D roles/launcher/tasks/upgrade_sso_7.2_to_7.3.yml
D roles/launcher/templates/sso_7.3_deploymentconfig.json
These should only be referenced by the upgrade_resources playbook. If so, they can be dismissed in the context of 1.6.0
M roles/resource_limits/defaults/main.yml
M roles/resource_limits/tasks/main.yml
M roles/resource_limits/tasks/patch_resource.yml
Review if not familiar with how upgrade playbook work since 1.5.2. (I'm not completely familiar, but believe there's a generation element to it)
A scripts/upgrade.template.yml
A scripts/upgrade_vars.template.yml
Check if there are changes needed to patch any resources during upgrade for this
M roles/backup/tasks/monitoring.yml
M roles/backup/tasks/upgrade.yaml
A roles/backup/tasks/upgrade_images.yml
M roles/backup/templates/backup-monitoring-alerts.yml.j2
Unsure about remaining files, best to review them to understand the changes and the impact.
A roles/amq_streams/tasks/upgrade_images.yml
M roles/customisation/tasks/main.yaml
M roles/datasync/tasks/main.yml
A roles/gitea/tasks/upgrade_images.yml
M roles/namespace/defaults/main.yml
A roles/namespace/tasks/upgrade.yml
M roles/namespace/templates/namespace.yml.j2
Raw
files.out
M .github/PULL_REQUEST_TEMPLATE.md
A CHANGELOG.md
M README.adoc
M docs/customisation.md
A docs/monitoring/customisation.md
A docs/monitoring/grafana-endpoints-detailed.png
A docs/monitoring/prometheus-alert-working.png
A docs/monitoring_docs.asciidoc
M docs/release.md
M inventories/group_vars/all/manifest.yaml
M inventories/group_vars/poc/poc.yml
A playbooks/cve_rollout.yml
M playbooks/generate-customisation-inventory.yml
A playbooks/group_vars/all/cve.yml
A playbooks/group_vars/all/upgrade.yml
M playbooks/install.yml
M playbooks/install_backups.yml
M playbooks/install_middleware_monitoring_config.yml
D playbooks/install_mobile_middleware_monitoring_config.yml
M playbooks/install_mobile_services.yml
M playbooks/install_services.yml
M playbooks/mobile_generate_manifest.yml
M playbooks/uninstall.yml
M playbooks/uninstall_mobile.yml
M playbooks/update_resources.yml
A playbooks/upgrade.yml
M playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml
D playbooks/upgrades/install_user_rhsso.yml
M playbooks/upgrades/upgrade.yaml
M roles/3scale/defaults/main.yml
D roles/3scale/tasks/_wt2_route.yml
M roles/3scale/tasks/install.yml
M roles/3scale/tasks/new_limits.yml
D roles/3scale/tasks/routes.yml
D roles/3scale/tasks/upgrade.yml
A roles/3scale/tasks/upgrade_images.yml
A roles/amq_streams/tasks/upgrade_images.yml
M roles/apicurito/defaults/main.yml
A roles/apicurito/tasks/new_limits.yml
A roles/apicurito/tasks/upgrade_images.yml
M roles/backup/tasks/monitoring.yml
M roles/backup/tasks/upgrade.yaml
A roles/backup/tasks/upgrade_images.yml
M roles/backup/templates/backup-monitoring-alerts.yml.j2
D roles/code-ready/tasks/upgrade.yml
R074 roles/code-ready/defaults/main.yml roles/codeready/defaults/main.yml
R100 roles/code-ready/tasks/backup.yml roles/codeready/tasks/backup.yml
R077 roles/code-ready/tasks/download_installer.yml roles/codeready/tasks/download_installer.yml
R100 roles/code-ready/tasks/install.yml roles/codeready/tasks/install.yml
R100 roles/code-ready/tasks/keycloak-client.yml roles/codeready/tasks/keycloak-client.yml
R100 roles/code-ready/tasks/main.yaml roles/codeready/tasks/main.yaml
A roles/codeready/tasks/new_limits.yml
R100 roles/code-ready/tasks/uninstall.yml roles/codeready/tasks/uninstall.yml
R100 roles/code-ready/tasks/upgrade_1.0_to_1.2.yml roles/codeready/tasks/upgrade_1.0_to_1.2.yml
A roles/codeready/tasks/upgrade_images.yml
R100 roles/code-ready/templates/config.yaml roles/codeready/templates/config.yaml
R100 roles/code-ready/templates/keycloak/client.json roles/codeready/templates/keycloak/client.json
M roles/customisation/tasks/main.yaml
M roles/datasync/tasks/main.yml
M roles/enmasse/defaults/main.yml
M roles/enmasse/tasks/new_limits.yml
R059 roles/enmasse/tasks/upgrade.yml roles/enmasse/tasks/upgrade_images.yml
M roles/fuse/defaults/main.yml
M roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml
M roles/fuse/tasks/main.yml
M roles/fuse/tasks/uninstall.yml
M roles/fuse/tasks/upgrade.yml
A roles/fuse/tasks/upgrade_images.yml
M roles/fuse_managed/defaults/main.yml
M roles/fuse_managed/tasks/main.yml
A roles/fuse_managed/tasks/new_limits.yml
M roles/fuse_managed/tasks/uninstall.yml
M roles/fuse_managed/tasks/upgrade.yml
A roles/fuse_managed/tasks/upgrade_images.yml
M roles/fuse_managed/templates/syndesis-customresource.yml.j2
A roles/gitea/tasks/upgrade_images.yml
M roles/launcher/defaults/main.yml
A roles/launcher/tasks/new_limits.yml
A roles/launcher/tasks/upgrade_images.yml
D roles/launcher/tasks/upgrade_sso_7.2_to_7.3.yml
D roles/launcher/templates/sso_7.3_deploymentconfig.json
M roles/mdc/defaults/main.yml
M roles/mdc/tasks/install-operator.yml
M roles/mdc/tasks/monitoring.yml
A roles/mdc/tasks/new_limits.yml
M roles/mdc/tasks/uninstall.yml
A roles/mdc/tasks/upgrade_images.yml
M roles/mdc/templates/mdc_prometheus_rules.yaml.j2
M roles/mdc/templates/operator_prometheus_rules.yaml.j2
M roles/middleware_monitoring/defaults/main.yml
M roles/middleware_monitoring/tasks/main.yml
A roles/middleware_monitoring/tasks/new_limits.yml
M roles/middleware_monitoring/tasks/uninstall.yml
M roles/middleware_monitoring/tasks/upgrade/grafana.yml
M roles/middleware_monitoring/tasks/upgrade/prometheus.yml
M roles/middleware_monitoring/tasks/upgrade/trigger.yml
A roles/middleware_monitoring/tasks/upgrade_images.yml
M roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2
M roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2
M roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2
M roles/middleware_monitoring_config/defaults/main.yml
A roles/middleware_monitoring_config/tasks/create_alertmanager.yml
R100 roles/middleware_monitoring_config/tasks/kube_state_metrics_alerts.yml roles/middleware_monitoring_config/tasks/create_alerts.yml
M roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml
M roles/middleware_monitoring_config/tasks/main.yml
M roles/middleware_monitoring_config/tasks/upgrade.yml
A roles/middleware_monitoring_config/templates/alertmanager.yml.j2
M roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2
A roles/middleware_monitoring_config/templates/kube_state_metrics_3scale_alerts.yml.j2
M roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2
A roles/middleware_monitoring_config/templates/kube_state_metrics_fuse_online_alerts.yml.j2
M roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2
M roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2
D roles/mobile_security_service/OWNERS
M roles/mobile_security_service/defaults/main.yml
D roles/mobile_security_service/tasks/backup.yml
D roles/mobile_security_service/tasks/main.yml
D roles/mobile_security_service/tasks/monitoring.yml
D roles/mobile_security_service/tasks/upgrade.yml
D roles/mobile_security_service/templates/backup_cr.yml.j2
D roles/mobile_security_service/templates/cluster_role_binding.yml.j2
D roles/mobile_security_service/templates/mss_grafana_dashboard.yml.j2
D roles/mobile_security_service/templates/mss_operator_grafana_dashboard.yml.j2
D roles/mobile_security_service/templates/mss_operator_prometheus_rule.yml.j2
D roles/mobile_security_service/templates/mss_prometheus_rule.yml.j2
D roles/mobile_security_service/templates/operator.yml.j2
M roles/msbroker/defaults/main.yml
M roles/msbroker/tasks/apply_msbroker_template.yml
A roles/msbroker/tasks/upgrade_images.yml
M roles/namespace/defaults/main.yml
A roles/namespace/tasks/upgrade.yml
M roles/namespace/templates/namespace.yml.j2
M roles/resource_limits/defaults/main.yml
M roles/resource_limits/tasks/main.yml
M roles/resource_limits/tasks/patch_resource.yml
M roles/rhsso-user/defaults/main.yml
A roles/rhsso-user/tasks/new_limits.yml
D roles/rhsso-user/tasks/upgrade.yaml
M roles/rhsso/defaults/main.yml
A roles/rhsso/tasks/new_limits.yml
M roles/rhsso/tasks/upgrade.yaml
A roles/rhsso/tasks/upgrade_images.yml
M roles/ups/defaults/main.yml
M roles/ups/tasks/install-operator.yml
M roles/ups/tasks/monitoring.yml
A roles/ups/tasks/new_limits.yml
A roles/ups/tasks/upgrade_images.yml
M roles/ups/templates/operator.yml.j2
M roles/ups/templates/operator_prometheus_rule.yml.j2
M roles/ups/templates/prometheus_rule.yml.j2
M roles/walkthroughs/templates/crud_spboot_example.yml
M roles/webapp/defaults/main.yml
A roles/webapp/tasks/new_limits.yml
M roles/webapp/tasks/upgrade.yaml
A roles/webapp/tasks/upgrade_images.yml
M scripts/release.sh
A scripts/upgrade.template.yml
A scripts/upgrade_vars.template.yml
Raw
full.diff
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
index 1147e3ed..75136a75 100644
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -2,6 +2,17 @@
<!-- Add any additional information needed. Such as the Jira or GH issue this PR relates to or any other context you feel is necessary.) -->
## Verification Steps
+As the verifier of the PR the following process should be done:
+
+### Installation Verification
+- Ensure the author of the PR has attached a log of the installation run from his branch to the jira or pr and check that it exited as expected.
+- Verify the fresh installation is correct on cluster provided by PR author
+### Upgrade Verification
+- After installation verification, notify the PR author to begin an upgrade on their cluster
+- Ensure the developer of the PR has attached a log of the upgrade run from his branch to the jira or pr and check that it exited as expected.
+- If possible, look at the tasks that ran and see they match the PR
+- Verify the upgrade is correct on cluster provided by PR author
+
<!--
Add the steps required to check this change. Following an example.
@@ -14,7 +25,8 @@ Add the steps required to check this change. Following an example.
## Is an upgrade task required and are there additional steps needed to test this?
<!-- If there is an upgrade required, either outline the steps to test it or link to the issue for the upgrade -->
-
+- [ ] Yes
+- [ ] No
diff --git a/CHANGELOG.md b/CHANGELOG.md
new file mode 100644
index 00000000..ea853560
--- /dev/null
+++ b/CHANGELOG.md
@@ -0,0 +1,2 @@
+## Unreleased
+* [INTLY-3623] - Refactor of inventories and associated group_vars to support POC, OSD and PDS environments
\ No newline at end of file
diff --git a/README.adoc b/README.adoc
index 246cf54d..3f24c50d 100644
--- a/README.adoc
+++ b/README.adoc
@@ -16,7 +16,7 @@ toc::[]
= Overview
-The purpose of this repository is to provide a set of Ansible playbooks that can be used to install a range of Red Hat middleware products on Openshift.
+The purpose of this repository is to provide a set of Ansible playbooks that can be used to install a range of Red Hat middleware products & other projects via Ansible Tower on openshift.
These products include:
@@ -26,6 +26,16 @@ These products include:
* Eclipse Che
* Launcher
* 3Scale
+* Fuse
+* Gitea
+* Apicurito
+* Nexus
+* Datasync
+* Unified push
+* Mobile developer console
+* AMQ streams
+
+
== Prerequisites
@@ -38,10 +48,6 @@ These products include:
|>= v3.10
|Openshift CLI (OC)
|>= v3.10
-|Template Service Broker
-|n/a
-|Ansible Service Broker
-|n/a
|===
@@ -54,75 +60,122 @@ These products include:
== Installation Steps
-The following section demonstrates how to install each of the products listed above on an existing Openshift cluster.
+The following will provide information on how to install Integreatly on a openshift cluster via an bastion host.
-:numbered:
-== Clone installation GIT repository locally
+
+#### 1. Sign into your bastion host and become root of the host
[source,shell]
-----
+
+```
+ssh -i /path/ocpkey.pem [email protected]
+
+sudo -i
+```
+
+
+#### 2. Clone installation GIT repository on the bastion host
+
+[source,shell]
+```
git clone https://github.com/integr8ly/installation.git
-----
+```
+
+#### 3. Obtain your openshift master hostname using the following command
+
+
+__NOTE__: This hostname will be needed in a following step to allow connection to your cluster
+
+[source,shell]
+
+```
+oc get nodes
+```
-== Create the inventory hosts file
+#### 4. Create the inventory hosts file
-. Create the host file based on the template(`../inventories/hosts.default`). Following the an example.
+. Create the host file based on the template(`../inventories/hosts.default`) located in the inventories directory.
+
[source,shell]
----
-$ cp inventories/hosts.template inventories/hosts
+cp inventories/hosts.template inventories/hosts
----
-. Update the host file to connect in your OpenShift cluster
-+
-Prior to running the playbooks the master hostname and associated SSH username *must* be set in the inventory host file to match the target cluster configuration. The following example sets the SSH username to *evals* and the master hostname to *master.evals.example.com*:
+. Update the hosts file master field with the master hostname from the previous step.
+
+
[source]
----
~/installation/inventories/hosts
+[local:vars]
+ansible_connection=local
+
+run_master_tasks=true
+
+[local]
+127.0.0.1
+
[OSEv3:children]
master
[OSEv3:vars]
-ansible_user=evals
+ansible_user=ec2-user
[master]
-master.evals.example.com
+127.0.0.1
----
-+
-NOTE: It is possible to add the variable `ansible_ssh_private_key_file` for the master host when the ssh connection requires a public key.(E.g `ansible_ssh_private_key_file=~/.ssh/ocp-workshop.pem`)
-== Check the connection with the OpenShift cluster
-Run the following command in order to check the connection with the OpenShift cluster from the root of the repository.
+#### 5. Check the connection with the OpenShift cluster
+Run the following command to verify the connection to the OpenShift cluster.
[source,shell]
-----
-$ ansible -m ping all
-----
+```
+ansible -m ping all
+```
-Following an example of the expected output.
+Output:
[source,shell]
-----
-$ ansible -m ping all
+
+```
+ansible -m ping all
master.example.openshiftworkshop.com | SUCCESS => {
"changed": false,
"ping": "pong"
}
-----
+```
-== Log into OpenShift
+#### 6. Create GitHub OAuth to enable GitHub authorization
-Before run the scripts it is required login via oc client tool to the master/OCP. Following an example.
+. Login into GitHub
+. Go to `Settings >> Developer Settings >> New OAuth App`.
++
+image::https://user-images.githubusercontent.com/7708031/48856646-dea13780-edae-11e8-9999-16b61dcc05ca.png[GitHub OAuth App]
+
+. Add the following fields values
++
+.Fields values descriptions
+|===
+|Field |Value
+|Application Name
+|Any value
+|Home Page URL
+|http://localhost
+|Authorization callback URL
+|http://localhost
+|===
++
+IMPORTANT: The callback URL is a placeholder for now and will be changed after the installation playbook is finished.
+
+. Click on `Register Application`
+. The values found in GitHub OAuth App, `Client ID` and `Client Secret`, will be needed in the next step to install Integreatly.
+
+image::https://user-images.githubusercontent.com/7708031/48856868-7141d680-edaf-11e8-836f-4d533f8ed402.png[GitHub OAuth App Fields]
-[source,shell]
-----
-oc login master.example.openshiftworkshop.com -u <user> -p <password>
-----
[[install-all]]
-== Install all products from a single playbook
+#### 7. Install all products from a single playbook
All products can be installed using the *install.yml* playbook located in the *playbooks/* directory.
@@ -144,83 +197,47 @@ By default Mobile Developer Services will not be installed. In order to install
.Install Mobile Developer Services
|===
-| Variable | Description
-| mobile_security_service | Whether Mobile Security Service will be installed or not. Defaults to `false`
+| Variable | Description | Default Value
+| mdc | Whether Mobile Developer Console will be installed or not | `true`
+| ups | Whether Mobile Unified Push Server will be installed or not | `true`
+| datasync | Whether DataSync components will be installed or not | `true`
|===
-=== Create GitHub OAuth to enable GitHub authorization for Launcher
-. Login into GitHub
-. Go to `Settings >> Developer Settings >> New OAuth App`. Following an image as example to ilustrate this area.
-+
-image::https://user-images.githubusercontent.com/7708031/48856646-dea13780-edae-11e8-9999-16b61dcc05ca.png[GitHub OAuth App]
-
-. Add the following fields values
-+
-.Fields values descriptions
-|===
-|Field |Value
-|Application Name
-|Any value
-|Home Page URL
-|http://localhost
-|Authorization callback URL
-|http://localhost
-|===
-+
-IMPORTANT: The callback URL is a placeholder for now and will be changed after the installation playbook is finished.
-
-. Click on `Register Application`
-. The values found in GitHub OAuth App, `Client ID` and `Client Secret`, will be required in the next step to install Integreatly enabling GitHub authorization for Launcher. Following an example of this screen.
-+
-image::https://user-images.githubusercontent.com/7708031/48856868-7141d680-edaf-11e8-836f-4d533f8ed402.png[GitHub OAuth App Fields]
-
-=== Run the playbook
+#### 8. Run the playbook
[source,shell]
----
-$ oc login https://<openshift-master-url> -u <user> -p <password>
-$ ansible-playbook -i inventories/hosts playbooks/install.yml -e github_client_id=<your_client-id> -e github_client_secret=<your_client_secret>
+ ansible-playbook -i inventories/hosts playbooks/install.yml -e github_client_id=<your_client-id> -e github_client_secret=<your_client_secret>
----
-[TIP]
+
====
-The following command installs Integreatly without GitHub authorization for Launcher.
+The following flag can be used if self signed certs are used.
+
[source,shell]
----
-$ ansible-playbook -i inventories/hosts playbooks/install.yml
+-e eval_self_signed_certs=true
----
====
[TIP]
====
-The following command installs Integreatly disabling some components and enabling others.
+The following command installs Integreatly without GitHub authorization for Launcher.
+
[source,shell]
----
-$ ansible-playbook -i inventories/hosts playbooks/install.yml \
- -e rhsso_hide_default_identity_providers=False \
- -e create_cluster_admin=False \
- -e eval_seed_users_count=5 \
- -e ns_prefix="integr8tly-" \
- -e gitea=False \
- -e che=False \
- -e datasync=False \
- -e mobile_security_service=False \
- -e ups=False \
- -e mdc=False \
- -e application_metrics=True \
- -e eval_self_signed_certs=True \
- -e eval_app_host=apps.$GUID.open.redhat.com
+$ ansible-playbook -i inventories/hosts playbooks/install.yml
----
====
-For more detailed description of each variable used in the above example see the link:inventories/group_vars/all/manifest.yaml[manifest.yaml] file.
-=== Add the generated Authorization callback URL to GitHub OAuth
-Following and example of the output made at the end of the playbook with this URL.
+#### 9. Add the generated Authorization callback URL to GitHub OAuth
+
+Replace the authorization callback URL previously containing a temporary localhost value with the output of the installation seen below.
[source,shell]
----
@@ -230,11 +247,10 @@ ok: [127.0.0.1] => {
}
----
-The `http://localhost` placeholder added in the GitHub OAuth App should be replaced with this value. Following an example.
image::https://user-images.githubusercontent.com/7708031/48856981-c1209d80-edaf-11e8-9d23-f550c7ec31be.png[GitHub OAuth auhotization callback URL, 640]
-=== Add backup jobs
+#### 10.Add backup jobs (Optional not needed for dev)
__NOTE__: Needs to be used in an existing integreatly cluster.
@@ -261,28 +277,22 @@ Parameters:
| backup_namespace | backup namespace name to add all cronjobs | `openshift-integreatly-backups`
|===
-== Check the installation
+#### 11. Check the installation
IMPORTANT: Once the installation has finished you will no longer be able to login via the Openshift console or oc cli as the admin if there is an sso redirect in place. The new admin user is `[email protected]` password is `Password1`
-The URL for the Integraly view is `https://tutorial-web-app-webapp.apps.<domain>/`
+The URL for the solution explorer is `https://tutorial-web-app-webapp.apps.<domain>/`
For example, if the master url is `https://master.example.openshiftworkshop.com/`, the web app is available at `https://tutorial-web-app-webapp.apps.example.openshiftworkshop.com/`.
-image::https://user-images.githubusercontent.com/7708031/48856455-528f1000-edae-11e8-8c1a-f0b37a1049ce.png[integr8ly WebApp]
+image::https://user-images.githubusercontent.com/53817495/64680924-a3bfdb80-d476-11e9-801e-08f8a28c47a8.png[integr8ly WebApp]
-TIP: The project https://github.com/integr8ly/tutorial-web-app[Webapp] is responsible for the Integraly interface. You can find the URL looking for the router created for this project. As the following example.
-image::https://user-images.githubusercontent.com/7708031/48856461-5884f100-edae-11e8-92ca-ef4c93f8961f.png[integr8ly WebApp Router]
+TIP: The project https://github.com/integr8ly/tutorial-web-app[Webapp] is responsible for the solution explorer. You can find the URL looking for the router created for this project.
-Also, with the *evals* users created by the installer is possible to check the services in the OpenShift catalog.
IMPORTANT: The default login credentials are `[email protected]` / `Password1`
-Following an image of this console as example.
-
-image::https://user-images.githubusercontent.com/7708031/48856465-5ae74b00-edae-11e8-954d-2267a5d5d5d2.png[OCP Console with integr8ly]
-:numbered!:
== Uninstalling Integreatly
Run the uninstall.yml playbook from the root of the repository:
@@ -344,4 +354,4 @@ Some error happened with terminal WebSocket connection
Failed to import project
----
-- In order to solve these issues, you will need to accept the certs for all the routes that was created for that workspace. These routes are listed in the workspace deployment within the Che namespace.
\ No newline at end of file
+- In order to solve these issues, you will need to accept the certs for all the routes that was created for that workspace. These routes are listed in the workspace deployment within the Che namespace.
diff --git a/docs/customisation.md b/docs/customisation.md
index 9225bb84..0bc91bee 100644
--- a/docs/customisation.md
+++ b/docs/customisation.md
@@ -16,16 +16,16 @@ Currently Che is linked to launcher's sso instance. If you choose not to install
Installing backing RH-SSO is not optional
-## Adding new components and customising existing components
+### Adding new components and customising existing components
-# Prerequisites
+#### Prerequisites
- Familiar with Ansible
- Have credentials to login as an admin in the Integreatly cluster
- Ensure logged into your cluster via oc
-## Pull down your cluster's inventory file
+#### Pull down your cluster's inventory file
Each Integreatly cluster has a secret in the webapp namespace with an inventory file that
can be used to help you add customisations using ansible playbooks.
@@ -37,15 +37,17 @@ oc get secret inventory -n webapp --template '{{index .data "generated_inventory
Each of the components has a set of variables exposed via this inventory file. Each variable has a comment explaining what the value is. If there are things that you need which are missing, please create a issue on the installer repo.
-## Limitations
+#### Limitations
You do not have ssh access to the cluster so all customisations are limited to what can be done by the user you are logged in as via the OpenShift API and also via the various product APIs.
-## Example Customisation
+### Example Customisation
See the following repo for some examples of how to do customisations using the in cluster inventory file https://github.com/integr8ly/example-customisations
+## Customising monitoring
+Creating new monitoring rules and alerts is part of [separate document](monitoring/customisation.md).
diff --git a/docs/monitoring/customisation.md b/docs/monitoring/customisation.md
new file mode 100644
index 00000000..c2452dab
--- /dev/null
+++ b/docs/monitoring/customisation.md
@@ -0,0 +1,213 @@
+# Monitoring customisation
+
+It's possible to add your own monitoring and alerting to the cluster after it's installed.
+
+## Prerequisties
+
+* we'll need Integre8ly 1.5.x installed on the cluster
+* `oc` commands require user to be logged in already as a cluster admin
+
+If there is any service that needs to be added for the monitoring, this service usually runs in its own namespace. Any such namespace needs to have `monitoring-key=middleware` label set to allow monitoring of the service. There is the namespace `my-customservice` in the following examples.
+
+To add such label, you can use `oc` command:
+
+```bash
+oc label namespace my-customservice "monitoring-key=middleware"
+```
+
+There are 3 different examples for adding monitoring on the middleware service.
+
+* HTTP(s) endpoint monitoring - simply alert if service is not returning proper HTTP status code
+
+* Kubernetes monitoring - use various stats provided by [https://github.com/kubernetes/kube-state-metrics](kube-state-metrics), such as count of pods in Ready state
+
+* CPU/Memory per pod limits monitoring - check if CPU utilisation or used memory doesn't go above certain threshold
+
+
+### HTTP(s) endpoint monitoring
+
+Your service usually has a HTTP/HTTPs endpoint and we want to monitor if it returns HTTP 2xx status code. The easiest way to monitor that is by addding your rule using *BlackboxTarget*. The Blackbox Target CR accepts the following properties in the spec:
+
+* *blackboxTargets*: A list of targets for the blackbox exporter to probe.
+
+The `blackboxTargets` should be provided as an array in the form of:
+
+```yaml
+ blackboxTargets:
+ - service: example
+ url: https://example.com
+ module: http_extern_2xx
+```
+
+where `service` will be added as a label to the metric, `url` is the URL of the route to probe and `module` can be one of:
+
+* *http_2xx*: Probe http or https targets via GET using the cluster certificates
+* *http_post_2xx*: Probe http or https targets via POST using the cluster certificates
+* *http_extern_2xx*: Probe http or https targets via GET relying on a valid external certificate
+
+Follow up on the example here - https://github.com/integr8ly/application-monitoring-operator/blob/master/deploy/examples/BlackboxTarget.yaml
+
+The process of adding your own alert is this:
+
+1) Create yaml file locally with the `BlackboxTarget` CR (by modifying `BlackboxTarget.yaml` example above):
+
+```yaml
+apiVersion: applicationmonitoring.integreatly.org/v1alpha1
+kind: BlackboxTarget
+metadata:
+ name: custom-mycustomservice-blackboxtarget
+spec:
+ blackboxTargets:
+ - service: mycustomservice
+ url: http://mycustomservice-my-nodejsproject.apps.vsazel-a4c3.open.redhat.com/ #this is an URL of the service you want to monitor, replace it with yours
+ module: http_extern_2xx
+```
+
+and import CR to your cluster:
+
+```bash
+$ oc create -f BlackboxTarget.yaml -n middleware-monitoring
+blackboxtarget.applicationmonitoring.integreatly.org/example-blackboxtarget created
+```
+
+2) Create yaml file `CustomMonitoringRule.yaml` locally with the alerting CR `PrometheusRule`.
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ monitoring-key: custom-monitoring
+ prometheus: application-monitoring
+ role: alert-rules
+ name: custom-alerts
+spec:
+ groups:
+ - name: mycustomservice.rules
+ rules:
+ - alert: ExampleCustomServiceUnavailableAlert
+ annotations:
+ message: >-
+ Custom Service unavailable: If this console is
+ unavailable, the clients won't be able to do something.
+ expr: >
+ probe_success{job="blackbox",service="mycustomservice"} < 1 or
+ absent(probe_success{job="blackbox",service="mycustomservice"})
+ for: 5m
+ labels:
+ severity: critical
+```
+and import it same as in previous case
+
+```bash
+$ oc create -f CustomMonitoringRule.yaml -n middleware-monitoring
+prometheusrule.monitoring.coreos.com/custom-alerts created
+```
+
+3) In *Prometheus* UI you should see the new alert. Check if it's working by killing the monitored service (decreasing pod count to 0).
+
+![Prometheus alert](prometheus-alert-working.png).
+
+
+### Kubernetes monitoring
+
+Other way to monitor your custom services is to use Kubernetes monitoring itself. Integr8ly itself contains *kube-state-metrics* statistics.
+
+Check out the documentation on that there - https://github.com/kubernetes/kube-state-metrics/tree/master/docs
+
+1) Create yaml file `CustomMonitoringKubernetesRules.yaml` and in there custom `PrometheusRule` CR:
+
+_Note: (Example Node.js project has `mycustomservice` container and is running in namespace `my-nodejsproject`) _
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ monitoring-key: middleware
+ prometheus: application-monitoring
+ role: alert-rules
+ name: custom-kubernetes-alerts
+spec:
+ groups:
+ - name: mycustomservice.rules
+ rules:
+ - alert: ExampleCustomServicePodsAlert
+ annotations:
+ message: >-
+ Custom Service pod: No pods ready.
+ expr: > #replace in the following expression namespace, and label_deploymentconfig to fit your service
+ (1-absent(kube_pod_labels{namespace="my-nodejsproject",label_deploymentconfig="mycustomservice"} *
+ on(pod,pod) kube_pod_status_ready{namespace="my-nodejsproject", condition="true"}))
+ for: 5m
+ labels:
+ severity: critical
+```
+
+2) Save it and import
+
+```bash
+$ oc create -f CustomMonitoringKubernetesRules.yaml -n middleware-monitoring
+prometheusrule.monitoring.coreos.com/custom-kubernetes-alerts created
+```
+
+3) Check it in *Prometheus* UI
+
+### CPU and memory monitoring
+
+Another useful metric is usually to check CPU and memory utilisation and how it goes with set up pod limits on Kubernetes cluster. This requires CPU and memory limits for the pod to be set in the deployment config.
+
+_Note: Some service is running in `my-customservice` container_
+
+1) Create yaml file `CustomMonitoringLimitsRules.yaml` and in there custom `PrometheusRule` CR:
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ monitoring-key: middleware
+ prometheus: application-monitoring
+ role: alert-rules
+ name: custom-limits-alerts
+spec:
+ groups:
+ - name: mycustomservice.rules
+ rules:
+ - alert: MyServicePodCPUHigh
+ expr: > #replace in the following expression namespace to fit your service
+ "(sum(label_replace(sum by(namespace, pod_name, container_name) (rate(container_cpu_usage_seconds_total{namespace='my-customservice'}[5m])), 'container', '$1', 'container_name', '(.*)')) by (container) / sum(kube_pod_container_resource_limits_cpu_cores{namespace='my-customservice'}) by (container) * 100) > 90"
+ for: 5m
+ labels:
+ severity: warning
+ annotations:
+ description: "The MyService pod has been at 90% CPU usage for more than 5 minutes."
+ summary: "The MyService is reporting high cpu usage for more that 5 minutes."
+ - alert: MyServicePodMemoryHigh
+ expr: > #replace in the following expression namespace to fit your service
+ "(sum by(container) (label_replace(container_memory_usage_bytes{container_name!='',namespace='my-customservice'}, 'container', '$1', 'container_name', '(.*)')) / sum by(container) (kube_pod_container_resource_limits_memory_bytes{namespace='my-customservice'}) * 100) > 90"
+ for: 5m
+ labels:
+ severity: warning
+ annotations:
+ description: "The MyService pod has been at 90% memory usage for more than 5 minutes."
+ summary: "The MyService is reporting high memory usage for more that 5 minutes."
+```
+
+2) Save it and import
+
+```bash
+$ oc create -f CustomMonitoringLimitsRules.yaml -n middleware-monitoring
+prometheusrule.monitoring.coreos.com/custom-limits-alerts created
+```
+
+3) Check it in *Prometheus* UI
+
+
+## Items created in the built-in Grafana dashboards
+
+If you add `BlackboxTarget` it makes service automatically visible on the *Endpoints Summary*, *Endpoints Report* and *Endpoints Detailed* dashboards.
+
+![Endpoints Detailed Graphana Dashboard](grafana-endpoints-detailed.png)
+
+
diff --git a/docs/monitoring/grafana-endpoints-detailed.png b/docs/monitoring/grafana-endpoints-detailed.png
new file mode 100644
index 00000000..a53a89d1
Binary files /dev/null and b/docs/monitoring/grafana-endpoints-detailed.png differ
diff --git a/docs/monitoring/prometheus-alert-working.png b/docs/monitoring/prometheus-alert-working.png
new file mode 100644
index 00000000..99457320
Binary files /dev/null and b/docs/monitoring/prometheus-alert-working.png differ
diff --git a/docs/monitoring_docs.asciidoc b/docs/monitoring_docs.asciidoc
new file mode 100644
index 00000000..9c0b469b
--- /dev/null
+++ b/docs/monitoring_docs.asciidoc
@@ -0,0 +1,200 @@
+= Integreatly Monitoring
+The following document will cover different aspects of monitoring an Integreatly cluster.
+
+:toc:
+== Intended audience and prerequisites
+The document is intended for the following users:
+
+* SRE managing an Integreatly/RHMI cluster
+* Integreatly users:
+** Workshop user
+** RHMI customer
+* Developers or users of individual Integreatly components e.g Fuse/Syndesis
+* Developers of integreatly:
+** RH engineering
+** Upstream contributor
+
+It is assumed that the reader knows what Integreatly is, and the features of Integreatly.
+
+NOTE: Certain dashboards and alerts may not be visible if users don't have appropriate permissions.
+
+== Middleware Monitoring Architecture
+In the middleware monitoring architecture there are two monitoring stacks running.
+
+=== OpenShift Monitoring Stack
+This stack gathers metrics from kube-state-metrics & node_exporter. These metrics give state information about kubernetes resources, and container metrics like cpu, memory & volumes across all nodes in the kubernetes cluster.
+
+=== Middleware Monitoring Stack
+The middleware monitoring stack comprises of Prometheus, Alertmanager & Grafana, all managed by the application-monitoring-operator.
+
+==== Prometheus
+Prometheus is configured to scrape metrics from various services across the managed component namespaces.
+Components in managed namespaces need to satisfy some criteria for metrics to be scraped:
+
+* The namespace has a specific label with a specific value i.e. `monitoring-key=middleware`. This is to ensure we only monitor the namespaces we care about.
+* The namespace has a ServiceMonitor resource that defines the Services or Pods to scrape metrics from. This ServiceMonitor must also have a label of `monitoring-key=middleware` on it.
+
+Prometheus is also configured with Alerts via PrometheusRules resources. These alerts can make use of custom scraped metrics (as defined in a ServiceMonitor) or already scraped metrics like kube-state-metrics.
+
+==== AlertManager
+AlertManager receives any active alerts from Prometheus, and sends them to configured receivers based on the severity. Only currently active alerts will appear in AlertManger's web console.
+
+==== Grafana
+Grafana is configured via the Grafana Operator with dashboards from managed component namespaces. It leverages the GrafanaDashboard custom resource definition.
+
+== Navigating to the monitoring stack resources through the UI
+1. Login in to the OpenShift web console.
+2. Navigate to Managed service monitoring
+3. Navigate to Applications > Routes on the left-nav
+4. Open the relevant route to access that service e.g. grafana, alertmanager, prometheus
+5. Authenticate with your OpenShift credentials and 'Allow' your account to be linked (one time process per service)
+
+=== Permissions:
+Permissions are checked via `SubjectAccessReview` (SAR) to see if the logged in user has access to the various monitoring services. In the case of the monitoring services, if a user can `get namespaces`, then they pass the review and have permission. To confirm if you have the appropriate permission, try the following command `oc get namespaces`
+
+*NOTE* Some urls may skip authentication, e.g. /metrics
+
+== Navigating to the monitoring stack through the CLI
+
+=== Prometheus
+To get the Prometheus web console url:
+```
+oc get route prometheus-route -n openshift-middleware-monitoring -o template --template "https://{{.spec.host}}"
+
+```
+=== Grafana
+To get the Grafana web console url:
+```
+oc get route grafana-route -n openshift-middleware-monitoring -o template --template "https://{{.spec.host}}"
+```
+
+=== AlertManager
+To get the AlertManager web console url:
+```
+oc get route alertmanager-route -n openshift-middleware-monitoring -o template --template "https://{{.spec.host}}"
+```
+
+
+== What metrics are available in the Integreatly monitoring stack?
+
+The Prometheus instance in the `openshift-middleware-monitoring` namespace federates the below metrics from the prometheus instance in the `openshift-monitoring` namespace. To get the current federation configuration use the following command: `oc get secret additional-scrape-configs -n middleware-monitoring --template '{{index .data "integreatly.yaml"}}' | base64 --decode | grep -A 10 "params:"`
+
+For example:
+```
+ params:
+ match[]:
+ - '{ endpoint="https-metrics" }'
+ - '{ service="kube-state-metrics" }'
+ - '{ service="node-exporter" }'
+ - '{ __name__=~"namespace_pod_name_container_name:.*" }'
+ - '{ __name__=~"node:.*" }'
+ - '{ __name__=~"container_memory_.*" }'
+ - '{ __name__=~":node_memory_.*" }'
+ scheme: https
+ tls_config:
+```
+
+=== Kube state metrics
+Kube-state-metrics is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. It is not focused on the health of the individual Kubernetes components, but rather on the health of the various objects inside, such as deployments, nodes and pods.
+
+Kube-state-metrics is about generating metrics from Kubernetes API objects without modification. This ensures that features provided by kube-state-metrics have the same grade of stability as the Kubernetes API objects themselves. In turn, this means that kube-state-metrics in certain situations may not show the exact same values as kubectl, as kubectl applies certain heuristics to display comprehensible messages. Kube-state-metrics exposes raw data unmodified from the Kubernetes API, this way users have all the data they require and perform heuristics as they see fit.
+
+The metrics are exported on the HTTP endpoint /metrics on the listening port (default 80). They are served as plaintext. They are designed to be consumed either by Prometheus itself or by a scraper that is compatible with scraping a Prometheus client endpoint. You can also open /metrics in a browser to see the raw metrics.
+
+Exposed metrics:
+Per group of metrics there is one file for each metrics. See each file for specific documentation about the exposed metrics:
+https://github.com/kubernetes/kube-state-metrics/tree/master/docs
+
+=== Node-exporter metrics
+The node exporter runs on every node in the openshift cluster gathering metrics about everything on that node and then sending the information back to prometheus.The metrics have a node="whatever-ip" label on them so you know which node the information came from. The node exporter for hardware and OS metrics exposed by *NIX kernels, written in Go with pluggable metric collectors.
+
+Enabled and disabled by default:
+To see the list of what is exposed or not exposed by default follow the following link:
+https://github.com/prometheus/node_exporter#collectors
+
+
+== What Alerts are in the Integreatly monitoring stack?
+
+=== What alerting is available?
+The monitoring stack has many different alerts depending on the metrics being monitored these alerts include:
+
+* 3Scale
+* Apicurito
+* Backups
+* CodeReady
+* ElasticSearch
+* Enmasse
+* Fuse Online
+* Keycloak/SSO
+* Kube State across RHMI namespaces
+* Launcher
+* Middleware Monitoring stack
+* Managed Service Broker
+* Nexus
+* Solution Explorer
+
+== How is alerting setup?
+
+Alerting can be setup in a few ways. Fro example, email as a default receiver, Pager duty with email and DeadMansSwitch for absence of alerts.
+
+1. Email as a default receiver
+Email server settings are defined at the global config level in the various smtp_ keys. This global config sets defaults for any receivers. The default receiver is configured to send alert & resolve emails to the configure recipients (comma separated).
+
+2. Pager duty with email for critical
+Any Prometheus Alerts with a label of `severity=critical` will be routed to the critical receiver. This receiver has the pagerduty_configs & email_configs sections defined. This will cause an alert email to be send to the configured recipients (comma separated) and a Pager Duty incident to be triggered.
+
+3. If an alert has a label of `alertname=DeadMansSwitch` it will be routed to the deadmansswitch alert. In this case, it will result in a mail being sent to the configured recipient. This is useful if you want to use the Dead Man's Snitch Integration with Pager Duty. For example, Prometheus will periodically send out a mail to alert that the monitoring stack is running. If the mail is not sent within a time period, a Pager Duty Incident will be triggered.
+
+== Configuring alerts
+To see the current alerts config use the following command `oc get secret alertmanager-application-monitoring -n openshift-middleware-monitoring --template='{{index .data "alertmanager.yaml"}}' | base64 --decode`. The configuration file is written in YAML format and usually follows the following:
+```
+global:
+ resolve_timeout: 5m
+ smtp_smarthost: smtp.sendgrid.net:587
+ smtp_from: noreply@<alertmanager_route>
+ smtp_auth_username: apikey
+ smtp_auth_password: <apikey_secret>
+route:
+ group_wait: 30s
+ group_interval: 5m
+ repeat_interval: 12h
+ receiver: default
+ routes:
+ - match:
+ severity: critical
+ receiver: critical
+ - match:
+ alertname: DeadMansSwitch
+ repeat_interval: 5m
+ receiver: deadmansswitch
+receivers:
+- name: default
+ email_configs:
+ - send_resolved: true
+ to: [email protected]
+- name: critical
+ pagerduty_configs:
+ - service_key: <pagerduty_service_integration_key>
+ email_configs:
+ - send_resolved: true
+ to: [email protected]
+- name: deadmansswitch
+inhibit_rules:
+- source_match:
+ alertname: 'JobRunningTimeExceeded'
+ severity: 'critical'
+ target_match:
+ alertname: 'JobRunningTimeExceeded'
+ severity: 'warning'
+ equal: ['alertname', 'job', 'label_cronjob_name']
+```
+
+
+toc::[]
+
+
+
+
+
+
+
diff --git a/docs/release.md b/docs/release.md
index aba1c3a2..02b75f32 100644
--- a/docs/release.md
+++ b/docs/release.md
@@ -1,25 +1,102 @@
# Releases
-## New Releases
+## Jira & other non-code process
-1) Check with the team to see do any of our components need a new release (gitea operator, webapp operator, keycloak operator etc)
-2) To cut a brand new release checkout master and create a new branch e.g. v2.7 and push this branch to integr8ly upstream
-3) Run the release script:
-
- ``` ./scripts/release.sh -b v2.7 -r release-2.7.0-rc1```
+* Ensure the scope of the release is agreed on with relevant stakeholders.
+* All issues should have a fixVersion of the planned patch release e.g. 1.5.0.
+* A test plan to cover the installation and upgrade paths for what's changing in the release should be agreed with QE.
+* Setup the release dashboard in Jira (See Jira Release Dashboard section below)
+* Setup a recurring checkpoint call for the release (once per day) as soon as an ER1 or RC1 is cut
-## New RCs and Patch releases
-If you are cutting a new rc or a patch release for an existing release then do the following
+## Jira Release Dashboard
-1) Check with the team if there is anything remaining to be cherry picked to the release branch
-2) Run the release script:
+There is a release dashboard that shows all relevant issues for a release and their status (https://issues.jboss.org/secure/Dashboard.jspa?selectPageId=12329297).
+This dashboard and all it's sub-filters are driven by 2 main filters:
-For example to cut rc2 for release 2.7.0
-
- ``` ./scripts/release.sh -b v2.7 -r release-2.7.0-rc2```
-
-To create rc1 of a patch release
+* RHMI Release - `_fixVersion` (https://issues.jboss.org/issues/?filter=12341116)
+* RHMI Release - `_affectedVersion` (https://issues.jboss.org/issues/?filter=12341117)
+
+When a release has started, this dashboard can be reused by updating the fixVersion & affectsVersion accordingly in these 2 filters.
+You may need to request permissions to modify these.
+
+## Installation Repo
+
+* Check with the team if there is anything remaining to be merged/cherry-picked to the appropriate branch.
- ``` ./scripts/release.sh -b v2.7 -r release-2.7.1-rc1```
+### Minor Release
+
+To cut the first RC for a new minor release (e.g. 1.5.0):
+
+* Checkout master and create a new branch e.g. v1.5 and push this branch to integr8ly upstream
+* Run the release script e.g. `./scripts/release.sh -b v1.5 -r release-1.5.0-rc1`
+
+For subsequent RCs, do the following:
-
\ No newline at end of file
+`./scripts/release.sh -b v1.5 -r release-1.5.0-rc2`
+
+### Patch Release
+
+To cut the first RC for a new patch release (e.g. v1.5.1):
+
+`./scripts/release.sh -b v1.5 -r release-1.5.1-rc1`
+
+To cut subsequent RCs for a patch release:
+
+`./scripts/release.sh -b v1.5 -r release-1.5.1-rc2`
+
+
+## Resetting the upgrade playbook
+
+
+### Minor Release
+
+There may be logic in the upgrade playbook that is targetted at a specific release only.
+After the minor release branch is created, the upgrade playbook in `playbooks/upgrades/upgrade.yml` should be reviewed and reset on `master` to remove any version specific blocks, tasks or roles being included.
+
+As this is a manual task, here are some guidelines for doing the review.
+
+* Any blocks that include a version specific upgrade task such as `upgrade_sso_72_to_73` can be removed
+* Any blocks that are doing an install of a new product can be removed.
+* Any blocks that are calling out to a generic `upgrade` task in a product role can usually be kept. These are likely to be doing an `oc apply` to resources that have been modified between releases and are safe to apply. Alternatively, they may be changing the version of a product operator, and the operator could be upgrading the product.
+
+All changes should be PR'd against `master`.
+Any release specific upgrade changes that need to be merged while a release is in progress should probably only land on the release branch. Discretion is advised based on the upgrade change being proposed and the above guidelines.
+
+### Patch Release
+
+The upgrade playbook in `playbooks/upgrades/upgrade.yml` should be emptied of all tasks except for the version prerequisite check and manifest update task.
+A patch release relies on the previous patch version having being installed/upgraded to already.
+For example
+
+
+## SOPs/help repo
+
+
+### Minor Release
+
+1) Checkout and pull down the latest `master` of https://github.com/fheng/integreatly-help (private repo)
+2) Create a new branch for the release. e.g.:
+
+ ```git checkout -b v1.5```
+3) Review any `Known Issues` in the [Installation SOP](https://github.com/fheng/integreatly-help/blob/master/sops/OSD_SRE_integreatly_install.asciidoc), and add/remove as appropriate for this release.
+4) Commit and push back the new branch to the upstream
+
+### Patch Release
+
+1) Checkout and pull down the release branch (e.g. v1.5) of https://github.com/fheng/integreatly-help (private repo)
+
+ ```git checkout v1.5```
+2) Review any `Known Issues` in the [Installation SOP](https://github.com/fheng/integreatly-help/blob/master/sops/OSD_SRE_integreatly_install.asciidoc), and add/remove as appropriate for this patch release.
+3) Commit and push back any changes to the upstream
+
+### Tagging the help repo
+
+Once a release has been signed off, the [help repo](https://github.com/fheng/integreatly-help) will need to be tagged on the release branch:
+
+```
+git checkout v1.5
+git fetch origin
+git reset --hard HEAD
+git tag release-1.5.0
+git push origin release-1.5.0
+```
diff --git a/inventories/group_vars/all/manifest.yaml b/inventories/group_vars/all/manifest.yaml
index 42f1e414..63194349 100644
--- a/inventories/group_vars/all/manifest.yaml
+++ b/inventories/group_vars/all/manifest.yaml
@@ -1,5 +1,5 @@
---
-integreatly_version: release-1.5.2
+integreatly_version: master
# Possible values are: rhpds, poc, osd, dev
cluster_type: rhpds
@@ -25,21 +25,22 @@ che_git_url: https://github.com/redhat-developer/codeready-workspaces-deprecated
#controls whether enmasse is installed or not
enmasse: True
-enmasse_version: '1.2.2.GA'
+enmasse_version: '1.3.1.GA'
enmasse_git_url: https://github.com/jboss-container-images/amq-online-images.git
#controls whether fuse is installed or not
fuse: True
#This is the release tag for fuse on openshift in github currently used to pull in the templates and image streams
-fuse_release_tag: 'application-templates-2.1.fuse-740025-redhat-00004'
+fuse_release_tag: 'application-templates-2.1.fuse-750056-redhat-00006'
#not currently used as the operator decides what to install
-fuse_version: '7.4'
+fuse_version: '7.4.1'
#controls whether Fuse Online is installed or not
fuse_online: True
# Below Fuse vars are not currently used but will be used to source the resources needed to do the install.
-fuse_online_release_tag: '1.7.25'
-fuse_online_resources_base: 'https://raw.githubusercontent.com/syndesisio/fuse-online-install/{{fuse_online_release_tag}}/resources'
+fuse_online_release_tag: '1.8.13'
+fuse_online_resources_base: 'https://raw.githubusercontent.com/integr8ly/fuse-online-install/{{fuse_online_release_tag}}/resources'
+fuse_online_binary_resources_base: https://github.com/jboss-fuse/fuse-clients/releases/download/{{fuse_online_release_tag}}/syndesis-{{fuse_online_release_tag}}-linux-64bit.tar.gz
fuse_online_operator_resources: '{{fuse_online_resources_base}}/fuse-online-operator.yml'
fuse_online_imagestream_resources: '{{fuse_online_resources_base}}/fuse-online-image-streams.yml'
fuse_online_crd_resources: '{{fuse_online_resources_base}}/syndesis-crd.yml'
@@ -93,13 +94,6 @@ nexus_version: '2.14.11-01'
datasync: True
datasync_template_tag: '0.8.1'
-#controls whether the mobile security service is installed or not
-mobile_security_service: True
-mobile_security_service_operator_release_tag: '0.4.0'
-mobile_security_service_operator_resources: 'https://raw.githubusercontent.com/aerogear/mobile-security-service-operator/{{mobile_security_service_operator_release_tag}}/deploy'
-mobile_security_service_operator_image: 'quay.io/aerogear/mobile-security-service-operator:{{ mobile_security_service_operator_release_tag }}'
-mss_version: '0.2.2'
-
rhsso_user_operator_resources: "{{ rhsso_operator_resources }}"
#unifiedpush
@@ -131,10 +125,10 @@ mobile_walkthrough_location: 'https://github.com/aerogear/mobile-walkthrough#{{
monitoring_label_name: 'monitoring-key'
monitoring_label_value: 'middleware'
-middleware_monitoring_operator_release_tag: '0.0.27'
+middleware_monitoring_operator_release_tag: '0.0.29'
middleware_monitoring_operator_resources: 'https://raw.githubusercontent.com/integr8ly/application-monitoring-operator/{{ middleware_monitoring_operator_release_tag }}/deploy'
-msbroker_release_tag: 'v0.0.10'
+msbroker_release_tag: 'v0.0.11'
msbroker_template: 'https://raw.githubusercontent.com/integr8ly/managed-service-broker/{{ msbroker_release_tag }}/templates/broker.template.yaml'
#AMQ Streams
diff --git a/inventories/group_vars/poc/poc.yml b/inventories/group_vars/poc/poc.yml
index f421ed9f..cd10c5cc 100644
--- a/inventories/group_vars/poc/poc.yml
+++ b/inventories/group_vars/poc/poc.yml
@@ -31,7 +31,7 @@ openshift_login: True
prerequisites_install: False
# Run tasks on master nodes
-# See group_vars/all/common.yml & relevant SOP for more info when
+# See group_vars/all/common.yml & relevant SOP for more info when
# setting this to False.
run_master_tasks: True
@@ -44,9 +44,8 @@ eval_threescale_storage_s3_aws_region: "{{ openshift_aws_region }}"
eval_threescale_storage_s3_aws_bucket: "{{ openshift_aws_clusterid }}-3scale"
# Mobile Developer Services configuration
-mobile_security_service: True
ups: True
mdc: True
# Use Integreatly Operator for Installation
-integreatly_operator: False
\ No newline at end of file
+integreatly_operator: False
diff --git a/playbooks/cve_rollout.yml b/playbooks/cve_rollout.yml
new file mode 100644
index 00000000..c72a7ce5
--- /dev/null
+++ b/playbooks/cve_rollout.yml
@@ -0,0 +1,21 @@
+# Generic playbook for CVE rollouts
+---
+- hosts: localhost
+ gather_facts: yes
+ # Ensure task is always run regardless of whether tags are specified
+ tags: ['always']
+ tasks:
+ # Required for Ansible Tower installs that need to login via oc as a prerequisite
+ - name: Openshift Login
+ include_role:
+ name: openshift
+ tasks_from: login.yml
+
+ - name: Update product images
+ include_role:
+ name: "{{ item }}"
+ tasks_from: "upgrade_images"
+ with_items: "{{ upgrade_product_images }}"
+ # ansible_run_tags var defaults to 'all' when no tags are specified
+ when: item in ansible_run_tags or 'all' in ansible_run_tags
+
diff --git a/playbooks/generate-customisation-inventory.yml b/playbooks/generate-customisation-inventory.yml
index 039d4efd..be555014 100644
--- a/playbooks/generate-customisation-inventory.yml
+++ b/playbooks/generate-customisation-inventory.yml
@@ -3,7 +3,7 @@
gather_facts: no
tasks:
- include_vars: ../roles/3scale/defaults/main.yml
- - include_vars: ../roles/code-ready/defaults/main.yml
+ - include_vars: ../roles/codeready/defaults/main.yml
- include_vars: ../roles/enmasse/defaults/main.yml
- include_vars: ../roles/fuse_managed/defaults/main.yml
- include_vars: ../roles/gitea/defaults/main.yml
@@ -12,7 +12,6 @@
- include_vars: ../roles/rhsso/defaults/main.yml
- include_vars: ../roles/webapp/defaults/main.yml
- include_vars: ../roles/ups/defaults/main.yml
- - include_vars: ../roles/mobile_security_service/defaults/main.yml
- include_vars: ../roles/mdc/defaults/main.yml
- name: Set eval_app_host var
diff --git a/playbooks/group_vars/all/cve.yml b/playbooks/group_vars/all/cve.yml
new file mode 100644
index 00000000..e1b77ec4
--- /dev/null
+++ b/playbooks/group_vars/all/cve.yml
@@ -0,0 +1,8 @@
+---
+upgrade_product_images:
+ - 3scale
+ - apicurito
+ - codeready
+ - enmasse
+ - fuse_managed
+ - rhsso
\ No newline at end of file
diff --git a/playbooks/group_vars/all/upgrade.yml b/playbooks/group_vars/all/upgrade.yml
new file mode 100644
index 00000000..48004ece
--- /dev/null
+++ b/playbooks/group_vars/all/upgrade.yml
@@ -0,0 +1,3 @@
+# DO NOT EDIT This file is auto generated!!
+upgrade_from_version: release-1.5.1
+upgrade_product_roles: []
diff --git a/playbooks/install.yml b/playbooks/install.yml
index ac61f1cd..a3a472e0 100644
--- a/playbooks/install.yml
+++ b/playbooks/install.yml
@@ -13,7 +13,6 @@
# the mobile services need to be installed before the monitoring config playbook is run
- import_playbook: "./install_mobile_services.yml"
- import_playbook: "./install_middleware_monitoring_config.yml"
-- import_playbook: "./install_mobile_middleware_monitoring_config.yml"
- import_playbook: "./install_application_metrics.yml"
- import_playbook: "./generate-customisation-inventory.yml"
- import_playbook: "./customise_web_console_install.yml"
diff --git a/playbooks/install_backups.yml b/playbooks/install_backups.yml
index 587a211b..6fbcac99 100644
--- a/playbooks/install_backups.yml
+++ b/playbooks/install_backups.yml
@@ -7,20 +7,15 @@
- include_vars: ../roles/3scale/defaults/main.yml
- include_vars: ../roles/resources_backup/defaults/main.yml
- include_vars: ../roles/enmasse/defaults/main.yml
- - include_vars: ../roles/code-ready/defaults/main.yml
+ - include_vars: ../roles/codeready/defaults/main.yml
- include_vars: ../roles/fuse_managed/defaults/main.yml
- include_vars: ../roles/launcher/defaults/main.yml
- import_tasks: ../roles/3scale/tasks/backup.yml
- import_tasks: ../roles/resources_backup/tasks/main.yml
- import_tasks: ../roles/enmasse/tasks/backup.yml
- - import_tasks: ../roles/code-ready/tasks/backup.yml
+ - import_tasks: ../roles/codeready/tasks/backup.yml
- import_tasks: ../roles/fuse_managed/tasks/backup.yml
- import_tasks: ../roles/launcher/tasks/backup.yml
- -
- include_role:
- name: mobile_security_service
- tasks_from: backup.yml
- when: mobile_security_service | default(false) | bool
-
include_role:
name: rhsso
diff --git a/playbooks/install_middleware_monitoring_config.yml b/playbooks/install_middleware_monitoring_config.yml
index db38a4f4..5a9f0c6d 100644
--- a/playbooks/install_middleware_monitoring_config.yml
+++ b/playbooks/install_middleware_monitoring_config.yml
@@ -18,4 +18,16 @@
tasks_from: monitoring
when:
- middleware_monitoring | default(true) | bool
- - webapp | default(true) | bool
\ No newline at end of file
+ - webapp | default(true) | bool
+ - include_role:
+ name: ups
+ tasks_from: monitoring
+ when:
+ - middleware_monitoring | default(true) | bool
+ - ups | default(true) | bool
+ - include_role:
+ name: mdc
+ tasks_from: monitoring
+ when:
+ - middleware_monitoring | default(true) | bool
+ - mdc | default(true) | bool
diff --git a/playbooks/install_mobile_middleware_monitoring_config.yml b/playbooks/install_mobile_middleware_monitoring_config.yml
deleted file mode 100644
index 07d10f19..00000000
--- a/playbooks/install_mobile_middleware_monitoring_config.yml
+++ /dev/null
@@ -1,25 +0,0 @@
----
-- hosts: localhost
- gather_facts: no
- roles:
- - role: middleware_monitoring_config
- when: middleware_monitoring | default(true) | bool
- tasks:
- - include_role:
- name: mobile_security_service
- tasks_from: monitoring
- when:
- - middleware_monitoring | default(true) | bool
- - mobile_security_service | default(true) | bool
- - include_role:
- name: ups
- tasks_from: monitoring
- when:
- - middleware_monitoring | default(true) | bool
- - ups | default(true) | bool
- - include_role:
- name: mdc
- tasks_from: monitoring
- when:
- - middleware_monitoring | default(true) | bool
- - mdc | default(true) | bool
\ No newline at end of file
diff --git a/playbooks/install_mobile_services.yml b/playbooks/install_mobile_services.yml
index f6f2dade..d949b789 100644
--- a/playbooks/install_mobile_services.yml
+++ b/playbooks/install_mobile_services.yml
@@ -4,7 +4,6 @@
- hosts: localhost
tasks:
- - include_vars: ../roles/mobile_security_service/defaults/main.yml
- include_vars: ../roles/ups/defaults/main.yml
- include_vars: ../roles/mdc/defaults/main.yml
- include_vars: ../roles/datasync/defaults/main.yml
@@ -14,38 +13,13 @@
name: datasync
tags: ['datasync']
when: datasync | default(true) | bool
-
+
- name: Install unified push server (ups)
include_role:
name: ups
tags: ['ups']
when: ups | default(true) | bool
- - name: Install Mobile Security Service
- include_role:
- name: mobile_security_service
- tags: ['mobile_security_service']
- when: mobile_security_service | default(true) | bool
-
- - name: Enable Mobile Security Service Backups
- include_role:
- name: mobile_security_service
- tasks_from: backup.yml
- tags: ['mobile_security_service']
- when:
- - mobile_security_service | default(true) | bool
- - backup_restore_install | default(false) | bool
-
- - name: Patch webapp
- include_role:
- name: webapp
- tasks_from: provision-webapp.yml
- vars:
- openshift_master_url: "{{ hostvars['EVAL_VARS']['openshift_master_url'] }}"
- openshift_asset_url: "{{ hostvars['EVAL_VARS']['openshift_asset_url'] }}"
- tags: ['mdc']
- when: mdc | default(true) | bool
-
- name: Install mobile developer console (mdc)
include_role:
name: mdc
@@ -54,14 +28,13 @@
tags: ['mdc']
when: mdc | default(true) | bool
-
- name: User rhsso configuration
include_role:
name: rhsso-user
when:
- user_rhsso | default(true) | bool
- mdc | default(true) | bool
-
+
- name: Install managed services broker
include_role:
name: msbroker
diff --git a/playbooks/install_services.yml b/playbooks/install_services.yml
index 693e084f..3c4f4dd3 100644
--- a/playbooks/install_services.yml
+++ b/playbooks/install_services.yml
@@ -70,11 +70,11 @@
when: launcher
- name: Expose vars
- include_vars: "../roles/code-ready/defaults/main.yml"
+ include_vars: "../roles/codeready/defaults/main.yml"
-
name: Install che
include_role:
- name: code-ready
+ name: codeready
vars:
che_keycloak_user: "{{ eval_launcher_sso_admin_username }}"
che_keycloak_password: "{{ eval_launcher_sso_admin_password }}"
@@ -86,7 +86,7 @@
cluster_url: "{{ hostvars['master']['master_url'] }}"
che_keycloak_realm: "{{ eval_launcher_sso_realm }}"
tags: ['che']
- when: eval_action == 'install' and (che | bool) and (launcher | bool)
+ when: eval_action == 'install' and che and launcher
- name: Expose vars
include_vars: "../roles/3scale/defaults/main.yml"
diff --git a/playbooks/mobile_generate_manifest.yml b/playbooks/mobile_generate_manifest.yml
index ce966f09..3e695cde 100644
--- a/playbooks/mobile_generate_manifest.yml
+++ b/playbooks/mobile_generate_manifest.yml
@@ -3,7 +3,6 @@
- hosts: localhost
tasks:
- - include_vars: ../roles/mobile_security_service/defaults/main.yml
- include_vars: ../roles/ups/defaults/main.yml
- include_vars: ../roles/mdc/defaults/main.yml
@@ -18,30 +17,6 @@
- set_fact:
mobile_components: []
- #mobile security service(mss)
- - name: Get Mobile Security Service rest route
- shell: oc get route/route -o template --template \{\{.spec.host\}\} -n {{ eval_mobile_security_service_namespace | default('mobile-security-service') }}
- register: mss_route_cmd
- when: mobile_security_service | default(true) | bool
-
- - set_fact:
- mss_route: "https://{{mss_route_cmd.stdout}}"
- when: mobile_security_service
-
- - name: Set Mobile Security Service component
- set_fact:
- mss_manifest:
- - name: Mobile Security Service
- version: "{{ mss_version }}"
- host: "{{ mss_route }}"
- mobile: true
- type: "security"
- when: mobile_security_service | default(true) | bool
-
- - set_fact:
- mobile_components: "{{mobile_components}} + {{ mss_manifest }}"
- when: mobile_security_service | default(true) | bool
-
#unified push server(ups)
- name: Unified Push Server
block:
@@ -119,7 +94,7 @@
- name: Create manifest secret
shell: oc create secret generic manifest --from-file=generated_manifest=/tmp/mobile-manifest.json --type=application/json -n {{ mdc_namespace }}
when: mdc | default(true) | bool
-
+
- include_role:
name: mdc
tasks_from: patch-manifest.yml
@@ -127,4 +102,4 @@
mdc_manifest_secret_name: manifest
tags: ['mdc']
when:
- - mdc | default(true) | bool
\ No newline at end of file
+ - mdc | default(true) | bool
diff --git a/playbooks/uninstall.yml b/playbooks/uninstall.yml
index a179dccb..65802263 100644
--- a/playbooks/uninstall.yml
+++ b/playbooks/uninstall.yml
@@ -124,7 +124,7 @@
-
name: Uninstall che
include_role:
- name: code-ready
+ name: codeready
tasks_from: uninstall
tags: ['che']
when: che | default(true) | bool
diff --git a/playbooks/uninstall_mobile.yml b/playbooks/uninstall_mobile.yml
index cf60adf2..84400d2d 100644
--- a/playbooks/uninstall_mobile.yml
+++ b/playbooks/uninstall_mobile.yml
@@ -1,6 +1,6 @@
- hosts: localhost
tasks:
- -
+ -
# MDC namespace needs to be deleted first to make sure all the CRs can be removed by their operators
name: Uninstall Mobile Developer Console (mdc)
include_role:
@@ -10,16 +10,7 @@
mdc_namespace: "{{ eval_mdc_namespace }}"
tags: ['mdc']
when: mdc | default(true) | bool
- -
- name: Uninstall Mobile Security Service
- include_role:
- name: mobile_security_service
- tasks_from: uninstall_mss.yml
- vars:
- mss_namespace: "{{ eval_mobile_security_service_namespace }}"
- tags: ['mss']
- when: mobile_security_service | default(true) | bool
- -
+ -
name: Uninstall Unified Push Server (ups)
include_role:
name: ups
@@ -28,7 +19,6 @@
ups_namespace: "{{ eval_ups_namespace }}"
tags: ['ups']
when: ups | default(true) | bool
-
-
name: Uninstall Data Sync Templates
include_role:
diff --git a/playbooks/update_resources.yml b/playbooks/update_resources.yml
index cc503a5c..a8ad18d5 100644
--- a/playbooks/update_resources.yml
+++ b/playbooks/update_resources.yml
@@ -5,8 +5,36 @@
- include_role:
name: 3scale
tasks_from: new_limits.yml
- tags: ['3scale']
+ - include_role:
+ name: apicurito
+ tasks_from: new_limits.yml
+ - include_role:
+ name: codeready
+ tasks_from: new_limits.yml
- include_role:
name: enmasse
tasks_from: new_limits.yml
- tags: ['enmasse']
+ - include_role:
+ name: fuse_managed
+ tasks_from: new_limits.yml
+ - include_role:
+ name: launcher
+ tasks_from: new_limits.yml
+ - include_role:
+ name: mdc
+ tasks_from: new_limits.yml
+ - include_role:
+ name: middleware_monitoring
+ tasks_from: new_limits.yml
+ - include_role:
+ name: rhsso
+ tasks_from: new_limits.yml
+ - include_role:
+ name: rhsso-user
+ tasks_from: new_limits.yml
+ - include_role:
+ name: ups
+ tasks_from: new_limits.yml
+ - include_role:
+ name: webapp
+ tasks_from: new_limits.yml
diff --git a/playbooks/upgrade.yml b/playbooks/upgrade.yml
new file mode 100644
index 00000000..eaef67e2
--- /dev/null
+++ b/playbooks/upgrade.yml
@@ -0,0 +1,45 @@
+# This file is re-generated after each release!
+# If you need to add tasks that should run on every upgrade, update the template here ../../scripts/upgrade.template.yml
+---
+- hosts: master
+ gather_facts: no
+ tasks:
+ - include_role:
+ name: openshift
+ tasks_from: set_master_vars
+ when: run_master_tasks | default(true) | bool
+
+# Required for Ansible Tower installs that need to login via oc as a prerequisite
+- import_playbook: "./openshift.yml"
+
+- hosts: localhost
+ gather_facts: yes
+ tasks:
+ - include_role:
+ name: prerequisites
+ tasks_from: upgrade
+ vars:
+ from_versions:
+ - "{{ upgrade_from_version }}"
+
+ - name: Set Upgrade Facts
+ set_fact: upgrade_{{ item }}=true
+ with_items: "{{ upgrade_product_roles }}"
+
+ - name: Upgrade product images
+ include_role:
+ name: "{{ item }}"
+ tasks_from: "upgrade_images"
+ with_items: "{{ upgrade_product_roles }}"
+
+# Add product specific upgrade tasks here, make sure to use the "when: upgrade_<product>|bool" condition on any new tasks added!!
+#
+# - name: Some Special webapp only upgrade thing
+# include_role:
+# name: webapp
+# tasks_from: upgrade_patch
+# when: upgrade_webapp|bool
+
+#Update product version (should always be last)
+- import_playbook: "./generate-customisation-inventory.yml"
+- import_playbook: "./mobile_generate_manifest.yml"
diff --git a/playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml b/playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml
index e7336937..8e906dc2 100644
--- a/playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml
+++ b/playbooks/upgrades/3scale_upgrade_2.5_to_2.6.yml
@@ -9,7 +9,6 @@
threescale_target_deployment_configs: ['apicast-production', 'apicast-staging', 'backend-cron', 'backend-listener', 'backend-redis', 'backend-worker', 'system-memcache', 'system-mysql', 'system-redis', 'system-sidekiq', 'system-sphinx', 'zync', 'zync-database']
threescale_amp_sa_template: "'{\"apiVersion\": \"v1\",\"kind\": \"ServiceAccount\",\"imagePullSecrets\": [{\"name\": \"{{ threescale_pull_secret_name }}\"}],\"metadata\": {\"name\": \"amp\"}}'"
threescale_patch_file_dir: "/tmp/3scale26-patch-files"
- threescale_route_creator_role: "'{\"apiVersion\": \"authorization.openshift.io/v1\", \"kind\": \"ClusterRole\", \"metadata\": {\"name\": \"3scale-route-creator\"},\"rules\": [{\"apiGroups\": [\"route.openshift.io\"],\"attributeRestrictions\": null,\"resources\": [\"routes\",\"routes/custom-host\"],\"verbs\": [\"create\",\"get\",\"list\",\"patch\",\"update\"]}]}'"
- name: Create patch file directory
file:
@@ -696,13 +695,3 @@
file:
path: "{{ threescale_patch_file_dir }}"
state: absent
-
- - name: Create route creator clusterrole
- shell: echo {{ threescale_route_creator_role }} | oc create -f -
- register: clusterrole_route_create_exists
- failed_when: clusterrole_route_create_exists.stderr and 'AlreadyExists' not in clusterrole_route_create_exists.stderr
-
- - name: Add role 3scale-route-creator to customer-admin user
- shell: oc adm policy add-role-to-user 3scale-route-creator customer-admin
- register: customer_admin_rolebinding_exists
- failed_when: customer_admin_rolebinding_exists.stderr
diff --git a/playbooks/upgrades/install_user_rhsso.yml b/playbooks/upgrades/install_user_rhsso.yml
deleted file mode 100644
index baa08874..00000000
--- a/playbooks/upgrades/install_user_rhsso.yml
+++ /dev/null
@@ -1,47 +0,0 @@
----
-- hosts: localhost
- gather_facts: true
- tasks:
- - block:
- - name: Include vars from rhsso
- include_vars: "../../roles/rhsso/defaults/main.yml"
-
- - name: "Update KeycloakRealm CRD"
- shell: "oc replace -f {{ rhsso_operator_resources }}/crds/KeycloakRealm_crd.yaml -n {{ eval_rhsso_namespace }}"
-
- - name: Find out original customer-admin password
- shell: "oc get secret customer-admin-user-credentials -n {{ eval_rhsso_namespace }} --template='{{ '{{' }} index .data \"password\" {{ '}}' }}' | base64 --decode"
- register: customer_admin_password_output
-
- - name: Install user rhsso
- include_role:
- name: rhsso
- tasks_from: install_sso.yml
- vars:
- sso_namespace: "{{ eval_user_rhsso_namespace }}"
- sso_namespace_display_name: "User Facing Red Hat Single Sign-On"
- rhsso_provision_immediately: true
-
- - name: Setup IDP and customer-admin permissions in master realm
- include_role:
- name: rhsso-user
- tasks_from: setup-master-realm.yml
- vars:
- openshift_master_url: "{{ hostvars['EVAL_VARS']['openshift_master_url'] }}"
- rhsso_evals_admin_password: "{{ customer_admin_password_output.stdout }}"
-
- - name: Setup backup for user rhsso
- include_role:
- name: rhsso
- tasks_from: backup.yaml
- vars:
- sso_namespace: "{{ eval_user_rhsso_namespace }}"
- tags: ['user_rhsso']
- when: backup_restore_install | default(false) | bool
- - name: apply {{ eval_user_rhsso_namespace }}/view role to {{ rhsso_evals_admin_username }} user
- shell: "oc adm policy add-role-to-user view {{rhsso_evals_admin_username}} -n {{ eval_user_rhsso_namespace }}"
- register: policy_cmd
- failed_when: policy_cmd.rc != 0
-
- tags: ['user_rhsso']
- when: user_rhsso | default(true) | bool
\ No newline at end of file
diff --git a/playbooks/upgrades/upgrade.yaml b/playbooks/upgrades/upgrade.yaml
index db7174d9..a58f75eb 100644
--- a/playbooks/upgrades/upgrade.yaml
+++ b/playbooks/upgrades/upgrade.yaml
@@ -18,60 +18,130 @@
tasks_from: upgrade
vars:
from_versions:
- - "release-1.5.1"
+ - "release-1.5.2"
+ - set_fact:
+ upgrade_webapp_image: quay.io/integreatly/tutorial-web-app:{{ webapp_version }}
+ upgrade_webapp_operator_image: quay.io/integreatly/tutorial-web-app-operator:{{ webapp_operator_release_tag }}
+ upgrade_fuse_image_tag: 1.7
+
+ # Mobile Security Service
+ - name: Uninstall Mobile Security Service
+ include_role:
+ name: mobile_security_service
+ tasks_from: uninstall_mss.yml
+ vars:
+ mss_namespace: "{{ eval_mobile_security_service_namespace }}"
+
+ # Managed Service Broker
+ - name: Expose vars
+ include_vars: "../../roles/rhsso/defaults/main.yml"
+ -
+ name: Set eval_app_host var
+ set_fact:
+ eval_app_host: "{{ hostvars['EVAL_VARS']['eval_app_host'] }}"
+ - include_role:
+ name: msbroker
+ tasks_from: upgrade
+ vars:
+ route_suffix: "{{ eval_app_host }}"
+ sso_realm: "{{ rhsso_realm }}"
+
+ # Solution Explorer
+ - name: Webapp upgrade
+ include_role:
+ name: webapp
+ tasks_from: upgrade
+ vars:
+ rhsso_openshift_master_config_path: "{{ eval_openshift_master_config_path }}"
+ rhsso_namespace: "{{ eval_rhsso_namespace }}"
+ openshift_master_url: "{{ hostvars['EVAL_VARS']['openshift_master_url'] | replace('https://', '') }}"
+ openshift_asset_url: "{{ hostvars['EVAL_VARS']['openshift_asset_url'] | replace('https://', '') }}"
+
+ - name: SSO upgrade
+ include_role:
+ name: rhsso
+ tasks_from: upgrade
+
+ - name: Fuse Online upgrade
+ include_role:
+ name: fuse_managed
+ tasks_from: upgrade
+
+ - name: AMQ Online upgrade
+ include_role:
+ name: enmasse
+
+ - name: Update image streams
+ include_role:
+ name: images
+ tasks_from: import
+ with_items:
+ - 3scale
+ - apicurito
+ - codeready
+ - enmasse
+ - fuse
+ - launcher
+ - sso
+ - webapp
+ loop_control:
+ loop_var: images_source_namespace
+
+ # Launcher
+ - name: Launcher upgrade
+ include_role:
+ name: launcher
+ tasks_from: upgrade
+ when: launcher
-- hosts: localhost
- gather_facts: yes
- tasks:
# Monitoring upgrade
- - name: Expose monitoring vars
+ - name: Expose vars
include_vars: "../../roles/middleware_monitoring_config/defaults/main.yml"
- - name: Expose 3scale vars
- include_vars: "../../roles/3scale/defaults/main.yml"
-
# Grafana
- include_role:
name: middleware_monitoring
tasks_from: upgrade/grafana
- when: target_version.stdout == "release-1.5.1"
# Prometheus & Alertmanager
- include_role:
name: middleware_monitoring
tasks_from: upgrade/prometheus
- when: target_version.stdout == "release-1.5.1"
# Pull the trigger on the monitoring upgrade
- include_role:
name: middleware_monitoring
tasks_from: upgrade/trigger
- when: target_version.stdout == "release-1.5.1"
# End monitoring upgrade
- name: Recreate any CRs for alerts to ensure we have the latest
include_role:
name: middleware_monitoring_config
tasks_from: upgrade.yml
- vars:
- mdc: false
- mobile_security_service: false
- ups: false
- - name: Apply latest backup monitoring alerts
- include_role:
+ # Backup alerts upgrade
+ - include_role:
name: backup
- tasks_from: monitoring
+ tasks_from: upgrade
- - name: Upgrade keycloak-operator
- include_role:
- name: rhsso
- tasks_from: upgrade.yaml
+ # Fuse
+ - include_role:
+ name: fuse
+ tasks_from: upgrade
+ vars:
+ old_fuse_tag: "application-templates-2.1.fuse-730065-redhat-00002"
+ fuse_image_tag: "{{ upgrade_fuse_image_tag }}"
- - name: Upgrade keycloak-operator for user-sso
- include_role:
- name: rhsso-user
- tasks_from: upgrade.yaml
+ - include_role:
+ name: fuse_managed
+ tasks_from: upgrade
+ vars:
+ fuse_image_tag: "{{ upgrade_fuse_image_tag }}"
+
+ # Namespaces
+ - include_role:
+ name: namespace
+ tasks_from: upgrade
#Update product version (should always be last)
- import_playbook: "../generate-customisation-inventory.yml"
diff --git a/roles/3scale/defaults/main.yml b/roles/3scale/defaults/main.yml
index a1466426..65d3f86f 100644
--- a/roles/3scale/defaults/main.yml
+++ b/roles/3scale/defaults/main.yml
@@ -50,100 +50,119 @@ threescale_pull_secret_name: "threescale-registry-auth"
threescale_resources:
- name: apicast-production
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 50m
memory: 50Mi
replicas: 2
- name: apicast-staging
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 5m
memory: 32Mi
replicas: 2
- name: backend-cron
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 5m
memory: 20Mi
- name: backend-listener
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 50m
memory: 300Mi
replicas: 2
- name: backend-redis
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 100m
memory: 10Mi
limits:
memory: 0
- name: backend-worker
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 15m
memory: 30Mi
replicas: 2
- name: system-app
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 5m
memory: 600Mi
- name: system-memcache
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 5m
memory: 10Mi
- name: system-mysql
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 25m
memory: 512Mi
- name: system-redis
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 15m
memory: 30Mi
limits:
memory: 0
- name: system-sidekiq
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 10m
memory: 50Mi
replicas: 2
- name: system-sphinx
kind: dc
- resources:
- requests:
+ resources:
+ limits:
+ cpu: 1
+ memory: 512Mi
+ requests:
cpu: 8m
memory: 250Mi
- name: zync
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 15m
memory: 120M
replicas: 2
- name: zync-database
kind: dc
- resources:
- requests:
+ resources:
+ requests:
cpu: 5m
memory: 60M
+- name: zync-que
+ kind: dc
+ resources:
+ requests:
+ cpu: 25m
# Routes
threescale_route_system_developer: system-developer
threescale_route_system_master: system-master
threescale_route_system_provider: system-provider
+
+threescale_image_streams:
+ - amp-apicast:latest
+ - amp-backend:latest
+ - amp-system:latest
+ - amp-zync:latest
+ - zync-database-postgresql:latest
+ - system-mysql:latest
+ - system-memcached:latest
+ - system-redis:latest
+ - backend-redis:latest
diff --git a/roles/3scale/tasks/_wt2_route.yml b/roles/3scale/tasks/_wt2_route.yml
deleted file mode 100644
index 1e63b60f..00000000
--- a/roles/3scale/tasks/_wt2_route.yml
+++ /dev/null
@@ -1,32 +0,0 @@
----
-
-- set_fact:
- w2_route_name: "wt2-{{ username }}"
-
-- name: Check WT2 route exists
- shell: oc get route/{{ w2_route_name }} -n {{ threescale_namespace }}
- register: wt2_route_cmd
- failed_when: false
-
-- name: Create WT2 route
- shell:
- cmd: |
- cat <<EOF | oc apply -n {{ threescale_namespace }} -f -
- apiVersion: v1
- kind: Route
- metadata:
- labels:
- app: 3scale
- name: "{{ w2_route_name }}"
- spec:
- host: "{{ w2_route_name }}-{{ threescale_namespace }}.{{ threescale_route_suffix }}"
- port:
- targetPort: gateway
- tls:
- insecureEdgeTerminationPolicy: None
- termination: edge
- to:
- kind: Service
- name: apicast-staging
- EOF
- when: wt2_route_cmd.stderr != '' and 'NotFound' in wt2_route_cmd.stderr
diff --git a/roles/3scale/tasks/install.yml b/roles/3scale/tasks/install.yml
index 54f87246..d7a779d9 100644
--- a/roles/3scale/tasks/install.yml
+++ b/roles/3scale/tasks/install.yml
@@ -54,9 +54,6 @@
- import_tasks: resources.yml
-- import_tasks: routes.yml
- when: not enable_wildcard_route
-
- name: "Setup pv-based storage"
block:
- name: "Check for storage class: {{ threescale_pvc_rwx_storageclassname }}"
diff --git a/roles/3scale/tasks/new_limits.yml b/roles/3scale/tasks/new_limits.yml
index 932a0410..17164b26 100644
--- a/roles/3scale/tasks/new_limits.yml
+++ b/roles/3scale/tasks/new_limits.yml
@@ -2,6 +2,7 @@
- name: Delete limit range (if exists)
shell: oc delete limitrange 3scale-core-resource-limits -n {{ threescale_namespace }}
register: delete_limits_cmd
+ changed_when: delete_limits_cmd.rc == 0 and 'deleted' in delete_limits_cmd.stderr
failed_when: delete_limits_cmd.stderr != '' and 'not found' not in delete_limits_cmd.stderr
- name: Apply resource overrides for 3scale
@@ -10,3 +11,4 @@
vars:
ns: "{{ threescale_namespace }}"
resources: "{{ threescale_resources }}"
+ when: (threescale_resources | d([], true) | length) > 0
diff --git a/roles/3scale/tasks/routes.yml b/roles/3scale/tasks/routes.yml
deleted file mode 100644
index d72273a7..00000000
--- a/roles/3scale/tasks/routes.yml
+++ /dev/null
@@ -1,13 +0,0 @@
----
-
-#Walkthrough 2 routes
-- name: Create evaluation admin user wt2 route
- include_tasks: _wt2_route.yml
- vars:
- username: "{{ rhsso_evals_admin_username }}"
-
-- name: Seed evaluation users wt2 routes
- include_tasks: _wt2_route.yml
- vars:
- username: "{{ rhsso_seed_users_name_format|format(item|int) }}"
- with_sequence: count={{ rhsso_seed_users_count }}
diff --git a/roles/3scale/tasks/upgrade.yml b/roles/3scale/tasks/upgrade.yml
deleted file mode 100644
index 52690e5c..00000000
--- a/roles/3scale/tasks/upgrade.yml
+++ /dev/null
@@ -1,226 +0,0 @@
-#upgrade
--
- name: "patch amp-system imagestream"
- shell: |
- oc patch -n {{ threescale_namespace }} imagestream/amp-system --type=json -p '[
- {
- "op": "add",
- "path": "/spec/tags/-",
- "value": {
- "annotations": {"openshift.io/display-name": "AMP system 2.5.0"},
- "from": { "kind": "DockerImage", "name": "registry.access.redhat.com/3scale-amp25/system"},
- "name": "2.5.0",
- "referencePolicy": {"type": "Source"}
- }
- }
- ]'
--
- name: "patch amp-apicast imagestream"
- shell: |
- oc patch -n {{ threescale_namespace }} imagestream/amp-apicast --type=json -p '[
- {
- "op": "add",
- "path": "/spec/tags/-",
- "value": {
- "annotations": {"openshift.io/display-name": "AMP APIcast 2.5.0"},
- "from": { "kind": "DockerImage", "name": "registry.access.redhat.com/3scale-amp25/apicast-gateway"},
- "name": "2.5.0",
- "referencePolicy": {"type": "Source"}
- }
- }
- ]'
--
- name: "patch amp-backend imagestream"
- shell: |
- oc patch -n {{ threescale_namespace }} imagestream/amp-backend --type=json -p '[
- {
- "op": "add",
- "path": "/spec/tags/-",
- "value": {
- "annotations": {"openshift.io/display-name": "AMP Backend 2.5.0"},
- "from": { "kind": "DockerImage", "name": "registry.access.redhat.com/3scale-amp25/backend"},
- "name": "2.5.0",
- "referencePolicy": {"type": "Source"}
- }
- }
- ]'
--
- name: "patch amp-zync imagestream"
- shell: |
- oc patch -n {{ threescale_namespace }} imagestream/amp-zync --type=json -p '[
- {
- "op": "add",
- "path": "/spec/tags/-",
- "value": {
- "annotations": {"openshift.io/display-name": "AMP Zync 2.5.0"},
- "from": { "kind": "DockerImage", "name": "registry.access.redhat.com/3scale-amp25/zync"},
- "name": "2.5.0",
- "referencePolicy": {"type": "Source"}
- }
- }
- ]'
--
- name: "set AMP_RELEASE env var"
- shell: "oc set env dc/system-app AMP_RELEASE=2.5.0 -n {{ threescale_namespace }}"
-
-# patch configmap system
--
- name: "download system > rolling_updates.yml from configmap"
- shell: |
- oc get configmap system -n {{ threescale_namespace }} -o jsonpath="{$.data.rolling_updates\.yml}" > /tmp/rolling_updates.yml
-
--
- name: "check file is not modified already"
- shell: "cat /tmp/rolling_updates.yml | grep policy_registry | wc -l"
- register: rolling_update_modified
--
- when: rolling_update_modified.stdout == "0"
- block:
- -
- name: "modify rolling_updates.yml on local file system"
- shell: |
- echo -e " policy_registry: true\n service_mesh_integration: true" >> /tmp/rolling_updates.yml
-
- -
- name: "get new rolling_updates.yml value"
- shell: sed "s/\$/\\\n/" /tmp/rolling_updates.yml
- register: new_rolling_updates_value
-
- -
- name: "apply changes to configmap"
- shell: |
- oc patch configmap system -n {{ threescale_namespace }} --type=json -p '[{"op": "add", "path": "/data/rolling_updates.yml", "value": "{{ new_rolling_updates_value.stdout }}"}]'
-
--
- name: "get mysql password"
- shell: 'oc set env dc/system-mysql -n {{ threescale_namespace }} --list | grep MYSQL_PASSWORD | cut -f2 -d='
- register: mysql_password
-
--
- name: "get mysql username"
- shell: 'oc set env dc/system-mysql -n {{ threescale_namespace }} --list | grep MYSQL_USER | cut -f2 -d='
- register: mysql_user
-
--
- name: "patch mysql username into system-database secret"
- shell: |
- oc patch secret/system-database -n {{ threescale_namespace }} -p "{\"stringData\": {\"DB_USER\": \"{{ mysql_user.stdout }}\"}}"
-
--
- name: "patch mysql password into system-database secret"
- shell: |
- oc patch secret/system-database -n {{ threescale_namespace }} -p "{\"stringData\": {\"DB_PASSWORD\": \"{{ mysql_password.stdout }}\"}}"
-
--
- name: "get index of mysql_user env var in system-mysql dc"
- shell: 'oc set env dc/system-mysql -n {{ threescale_namespace }} --list | grep -v "deploymentconfigs" | grep MYSQL_USER -B100 | grep -v MYSQL_USER | wc -l'
- register: mysql_user_index
-
--
- name: "get index of mysql_password env var in system-mysql dc"
- shell: 'oc set env dc/system-mysql -n {{ threescale_namespace }} --list | grep -v "deploymentconfigs" | grep MYSQL_PASSWORD -B100 | grep -v MYSQL_PASSWORD | wc -l'
- register: mysql_password_index
-
--
- name: "patch dc/system-mysql to read env vars from secret"
- shell: |
- oc patch -n {{ threescale_namespace }} dc system-mysql --type=json --patch '[
- {
- "op": "replace",
- "path": "/spec/template/spec/containers/0/env/{{ mysql_user_index.stdout }}",
- "value": {
- "name": "MYSQL_USER",
- "valueFrom": {
- "secretKeyRef": {"key": "DB_USER", "name": "system-database"}
- }
- }
- },
- {
- "op": "replace",
- "path": "/spec/template/spec/containers/0/env/{{ mysql_password_index.stdout }}",
- "value": {
- "name": "MYSQL_PASSWORD",
- "valueFrom": {
- "secretKeyRef": {"key": "DB_PASSWORD", "name": "system-database"}
- }
- }
- }
- ]'
-
--
- name: "scale down zync"
- shell: "oc scale --replicas 0 dc/zync -n {{ threescale_namespace }}"
-
--
- name: "backup zync-database"
- shell: |
- oc rsh -n {{ threescale_namespace }} $(oc get pods -n {{ threescale_namespace }} -l 'deploymentConfig=zync-database' -o jsonpath="{$.items[0].metadata.name}") bash -c 'pg_dumpall' > /tmp/zync-database-backup-for-2.5.raw
-
--
- name: "add new postgresql tag to image stream"
- shell: "oc tag -n {{ threescale_namespace }} --source=docker registry.access.redhat.com/rhscl/postgresql-10-rhel7 postgresql:10 --insecure=true"
-
--
- name: "update postgresql image tag in zync-database dc"
- shell: |
- oc patch -n {{ threescale_namespace }} dc/zync-database --type=json --patch '[
- {
- "op": "replace",
- "path": "/spec/triggers/1/imageChangeParams/from/name",
- "value": "postgresql:10"
- }
- ]'
--
- name: "wait for new pod to come up"
- shell: |
- oc get pods -n {{ threescale_namespace }} -l 'deploymentConfig=zync-database' -o jsonpath="{$.items[0].status.containerStatuses[0].ready}"
- register: zync_database_pod_phase
- until: zync_database_pod_phase.stdout == "true"
- retries: 50
- delay: 5
-
--
- name: "restore previous backup"
- shell: |
- cat /tmp/zync-database-backup-for-2.5.raw | oc rsh -n {{ threescale_namespace }} $(oc get pods -n {{ threescale_namespace }} -l 'deploymentConfig=zync-database' -o jsonpath="{$.items[0].metadata.name}") bash -c 'psql -d postgres -f -'
-
--
- name: "scale up zync"
- shell: |
- oc scale dc zync --replicas 1 -n {{ threescale_namespace }}
-
--
- name: "wait for zync to start"
- shell: |
- oc get pods -n {{ threescale_namespace }} -l 'deploymentConfig=zync' -o jsonpath="{$.items[0].status.containerStatuses[0].ready}"
- register: zync_pod_ready
- until: zync_pod_ready.stdout == "true"
- retries: 50
- delay: 5
--
- name: "clean up old image tag"
- shell: oc tag -n {{ threescale_namespace }} -d postgresql:9.5
- register: delete_tag
- failed_when: delete_tag.stderr != '' and 'NotFound' not in delete_tag.stderr
-
--
- name: "upgrade non imagestream deployments"
- include_role:
- name: images
- tasks_from: bump_deployment
- vars:
- ns: "{{ threescale_namespace }}"
- dc: "{{ item }}"
- with_items:
- - system-mysql
- - system-memcache
- - system-redis
- - backend-redis
-
-- name: Create 3scale redis secret
- include_role:
- name: backup
- tasks_from: _create_redis_secret.yml
- vars:
- secret_name: '{{ threescale_backup_redis_secret }}'
- secret_redis_host: 'system-redis.{{ threescale_namespace }}.svc'
\ No newline at end of file
diff --git a/roles/3scale/tasks/upgrade_images.yml b/roles/3scale/tasks/upgrade_images.yml
new file mode 100644
index 00000000..5da39e61
--- /dev/null
+++ b/roles/3scale/tasks/upgrade_images.yml
@@ -0,0 +1,7 @@
+---
+
+- name: Import Image Streams
+ shell: oc import-image {{ patch_image_stream_item }} -n {{ threescale_namespace }}
+ with_items: "{{ threescale_image_streams }}"
+ loop_control:
+ loop_var: patch_image_stream_item
diff --git a/roles/amq_streams/tasks/upgrade_images.yml b/roles/amq_streams/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/amq_streams/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/apicurito/defaults/main.yml b/roles/apicurito/defaults/main.yml
index 3747d36e..345b7288 100644
--- a/roles/apicurito/defaults/main.yml
+++ b/roles/apicurito/defaults/main.yml
@@ -4,4 +4,10 @@ apicurito_template: apicurito
apicurito_template_namespace: openshift
apicurito_template_file: /tmp/apicurito-template.yaml
apicurito_template_file_format: yaml
-apicurito_route_hostname: apicurito-app
\ No newline at end of file
+apicurito_route_hostname: apicurito-app
+
+apicurito_resources: []
+
+apicurito_image_streams:
+ - apicurito-ui:1.2
+ - fuse-apicurito-generator:1.2
diff --git a/roles/apicurito/tasks/new_limits.yml b/roles/apicurito/tasks/new_limits.yml
new file mode 100644
index 00000000..0f00f1c8
--- /dev/null
+++ b/roles/apicurito/tasks/new_limits.yml
@@ -0,0 +1,9 @@
+---
+
+- name: Apply resource overrides for apicurito
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ apicurito_namespace }}"
+ resources: "{{ apicurito_resources }}"
+ when: (apicurito_resources | d([], true) | length) > 0
diff --git a/roles/apicurito/tasks/upgrade_images.yml b/roles/apicurito/tasks/upgrade_images.yml
new file mode 100644
index 00000000..2ee96003
--- /dev/null
+++ b/roles/apicurito/tasks/upgrade_images.yml
@@ -0,0 +1,7 @@
+---
+
+- name: Import Image Streams
+ shell: oc import-image {{ patch_image_stream_item }} -n openshift
+ with_items: "{{ apicurito_image_streams }}"
+ loop_control:
+ loop_var: patch_image_stream_item
\ No newline at end of file
diff --git a/roles/backup/tasks/monitoring.yml b/roles/backup/tasks/monitoring.yml
index 7f2bf26c..a92aceb7 100644
--- a/roles/backup/tasks/monitoring.yml
+++ b/roles/backup/tasks/monitoring.yml
@@ -11,6 +11,6 @@
expected_cronjobs: "{{ backup_expected_cronjobs }}"
- name: Create CronJob Alerts
- shell: oc apply -f /tmp/backup-monitoring-alerts.yml -n {{ monitoring_namespace }}
+ shell: oc create -f /tmp/backup-monitoring-alerts.yml -n {{ monitoring_namespace }}
register: create_alerts
- failed_when: create_alerts.rc != 0
+ failed_when: create_alerts.stderr != '' and 'AlreadyExists' not in create_alerts.stderr
diff --git a/roles/backup/tasks/upgrade.yaml b/roles/backup/tasks/upgrade.yaml
index 4e5daa73..df15ff04 100644
--- a/roles/backup/tasks/upgrade.yaml
+++ b/roles/backup/tasks/upgrade.yaml
@@ -9,36 +9,3 @@
name: backup
tasks_from: monitoring
when: delete_alerts_cmd.rc == 0
-
-# Backup namespace cronjobs
-- name: get all cronjobs
- shell: "oc get cronjobs -o custom-columns=NAME:{.metadata.name} --no-headers -n {{ backup_namespace }}"
- register: cronjobs_result
-
-- set_fact:
- cronjobs: "{{ cronjobs_result.stdout.splitlines() }}"
-
-- name: "patch all cronjobs in {{ backup_namespace }} namespace"
- shell: "oc patch cronjob {{ item }} -n {{ backup_namespace }} --patch='[{\"op\": \"add\", \"path\": \"/spec/jobTemplate/spec/template/spec/containers/0/image\", \"value\": \"{{ upgrade_backup_container_image }}\"}]' --type=json"
- register: upgrade_cronjob
- failed_when: upgrade_cronjob.stderr != '' and 'not patched' not in upgrade_cronjob.stderr
- with_items: "{{ cronjobs }}"
-
-# Cluster SSO namespace cronjobs
-- name: Change backup CronJob name in Keycloak CR to include namespace suffix
- shell: "oc patch keycloak rhsso -n {{ rhsso_namespace }} --type json -p '[{\"op\":\"replace\",\"path\":\"/spec/backups/0/image_tag\",\"value\":\"{{ upgrade_backup_container_tag }}\"}]'"
- register: patch
- failed_when: patch.rc != 0
-
-# User SSO namespace cronjobs
-- name: Change backup CronJob name in Keycloak CR to include namespace suffix
- shell: "oc patch keycloak rhsso -n {{ user_rhsso_namespace }} --type json -p '[{\"op\":\"replace\",\"path\":\"/spec/backups/0/image_tag\",\"value\":\"{{ upgrade_backup_container_tag }}\"}]'"
- register: patch
- failed_when: patch.rc != 0
-
-# Verify cronjobs
-- name: Verify all middleware cronjobs use new image
- shell: "oc get cronjobs --selector='monitoring-key=middleware' -o custom-columns=NAME:{.spec.jobTemplate.spec.template.spec.containers[0].image} --no-headers --all-namespaces | sed 's|{{ upgrade_backup_container_image }}||g'"
- register: result
- until: result.stdout | replace('\n', '') | length == 0
- changed_when: False
diff --git a/roles/backup/tasks/upgrade_images.yml b/roles/backup/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/backup/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/backup/templates/backup-monitoring-alerts.yml.j2 b/roles/backup/templates/backup-monitoring-alerts.yml.j2
index 4e288cf9..c556c3cc 100644
--- a/roles/backup/templates/backup-monitoring-alerts.yml.j2
+++ b/roles/backup/templates/backup-monitoring-alerts.yml.j2
@@ -12,6 +12,7 @@ spec:
{% for check in [{ 'time': '300', 'severity': 'warning' }, { 'time': '600', 'severity': 'critical' }] %}
- alert: JobRunningTimeExceeded
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Job {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.job {{ '}}' }} has been running for longer than {{ check['time'] }} seconds
expr: |
time() - (max(kube_job_status_active * ON(job) GROUP_RIGHT() kube_job_labels{label_monitoring_key="middleware"}) BY (job) * ON(job) GROUP_RIGHT() max(kube_job_status_start_time * ON(job) GROUP_RIGHT() kube_job_labels{label_monitoring_key="middleware"}) BY (job, namespace, label_cronjob_name) > 0) > {{ check['time'] }}
@@ -20,6 +21,7 @@ spec:
{% endfor %}
- alert: CronJobSuspended
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: CronJob {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.cronjob {{ '}}' }} is suspended
expr: |
kube_cronjob_labels{ label_monitoring_key="middleware" } * ON (cronjob) GROUP_RIGHT() kube_cronjob_spec_suspend > 0
@@ -28,11 +30,13 @@ spec:
severity: critical
- alert: CronJobNotRunInThreshold
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: CronJob {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.label_cronjob_name {{ '}}' }} has not started a Job in 25 hours
expr: |
(time() - (max( kube_job_status_start_time * ON(job) GROUP_RIGHT() kube_job_labels{label_monitoring_key="middleware"} ) BY (job, label_cronjob_name) == ON(label_cronjob_name) GROUP_LEFT() max( kube_job_status_start_time * ON(job) GROUP_RIGHT() kube_job_labels{label_monitoring_key="middleware"} ) BY (label_cronjob_name))) > 60*60*25
- alert: CronJobsFailed
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: 'Job {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.job {{ '}}' }} has failed'
expr: >
clamp_max(max(kube_job_status_start_time * ON(job) GROUP_RIGHT() kube_job_labels{label_cronjob_name!=""} ) BY (job, label_cronjob_name, namespace) == ON(label_cronjob_name) GROUP_LEFT() max(kube_job_status_start_time * ON(job) GROUP_RIGHT() kube_job_labels{label_cronjob_name!=""}) BY (label_cronjob_name), 1) * ON(job) GROUP_LEFT() kube_job_status_failed > 0
@@ -43,6 +47,7 @@ spec:
{% for cronjob in cronjobs %}
- alert: CronJobExists_{{ namespace}}_{{ cronjob }}
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: CronJob {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.cronjob {{ '}}' }} does not exist
expr: |
absent(kube_cronjob_info{cronjob="{{ cronjob }}", namespace="{{ namespace }}"})
diff --git a/roles/code-ready/tasks/upgrade.yml b/roles/code-ready/tasks/upgrade.yml
deleted file mode 100644
index a9326469..00000000
--- a/roles/code-ready/tasks/upgrade.yml
+++ /dev/null
@@ -1,25 +0,0 @@
--
- name: "patch codeready deployment"
- shell: "oc patch deployment codeready --patch='{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"codeready\",\"imagePullPolicy\":\"Always\" }]}}}}' -n {{ che_namespace }}"
- register: patch_codeready_result
- failed_when: patch_codeready_result.stderr != ""
-
--
- name: "patch postgres deployment"
- shell: "oc patch deployment postgres --patch='{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"postgres\", \"image\":\"registry.access.redhat.com/rhscl/postgresql-96-rhel7:latest\", \"imagePullPolicy\":\"Always\"}]}}}}' -n {{ che_namespace }}"
- register: patch_postgres_result
- failed_when: patch_postgres_result.stderr != ""
-
--
- name: "redeploy codeready"
- shell: "oc patch deployment codeready --patch '{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"last-restart\":\"`date +'%s'`\"}}}}}' -n {{ che_namespace }}"
- register: redeploy_codeready_result
- failed_when: redeploy_codeready_result.stderr != ""
- when: '"not patched" in patch_codeready_result.stdout'
-
--
- name: "redeploy postgres"
- shell: "oc patch deployment postgres --patch '{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"last-restart\":\"`date +'%s'`\"}}}}}' -n {{ che_namespace }}"
- register: redeploy_postgres_result
- failed_when: redeploy_postgres_result.stderr != ""
- when: '"not patched" in patch_postgres_result.stdout'
diff --git a/roles/code-ready/defaults/main.yml b/roles/codeready/defaults/main.yml
similarity index 74%
rename from roles/code-ready/defaults/main.yml
rename to roles/codeready/defaults/main.yml
index 6c852d87..2145cab4 100644
--- a/roles/code-ready/defaults/main.yml
+++ b/roles/codeready/defaults/main.yml
@@ -6,6 +6,9 @@ che_namespace: "{{ eval_che_namespace | default('codeready') }}"
che_display_name: "Red Hat CodeReady"
che_template_folder: /tmp/che-templates
che_validate_certs: "{{ eval_sso_validate_certs | default('true') }}"
+che_deployment_name: "codeready"
+che_postgres_deployment_name: "postgres"
+che_operator_deployment_name: "codeready-operator"
#server template vars
che_protocol: https
@@ -21,9 +24,9 @@ che_persistent_volume: true
che_persistent_volume_size: 2Gi
che_persistent_volume_storageclassname: efs
-#postgre
-che_postgre_image_name: 'docker.io/eclipse/che-postgres'
-che_postgre_image_tag: '6.9.0'
+#images
+che_operator_image_name: 'registry.redhat.io/codeready-workspaces/server-operator-rhel8:1.2'
+che_postgre_image_name: 'registry.access.redhat.com/rhscl/postgresql-96-rhel7:latest'
#keycloak
che_keycloak_user: admin
@@ -38,3 +41,6 @@ che_keycloak_port: 443
codeready_backup_postgres_secret: 'codeready-postgres-secret'
codeready_postgres_cronjob_name: 'codeready-postgres-backup'
codeready_pv_cronjob_name: 'codeready-pv-backup'
+
+#scaling
+codeready_resources: []
diff --git a/roles/code-ready/tasks/backup.yml b/roles/codeready/tasks/backup.yml
similarity index 100%
rename from roles/code-ready/tasks/backup.yml
rename to roles/codeready/tasks/backup.yml
diff --git a/roles/code-ready/tasks/download_installer.yml b/roles/codeready/tasks/download_installer.yml
similarity index 77%
rename from roles/code-ready/tasks/download_installer.yml
rename to roles/codeready/tasks/download_installer.yml
index 9f26cee5..021241a1 100644
--- a/roles/code-ready/tasks/download_installer.yml
+++ b/roles/codeready/tasks/download_installer.yml
@@ -1,6 +1,6 @@
---
- set_fact:
- codeready_install_dir: /tmp/code-ready-{{ che_version }}
+ codeready_install_dir: /tmp/codeready-{{ che_version }}
- set_fact:
codeready_install_scripts_dir: "{{ codeready_install_dir }}/operator-installer"
@@ -14,4 +14,4 @@
when: codeready_installer_downloaded is not defined
- set_fact:
- codeready_installer_downloaded: true
\ No newline at end of file
+ codeready_installer_downloaded: true
diff --git a/roles/code-ready/tasks/install.yml b/roles/codeready/tasks/install.yml
similarity index 100%
rename from roles/code-ready/tasks/install.yml
rename to roles/codeready/tasks/install.yml
diff --git a/roles/code-ready/tasks/keycloak-client.yml b/roles/codeready/tasks/keycloak-client.yml
similarity index 100%
rename from roles/code-ready/tasks/keycloak-client.yml
rename to roles/codeready/tasks/keycloak-client.yml
diff --git a/roles/code-ready/tasks/main.yaml b/roles/codeready/tasks/main.yaml
similarity index 100%
rename from roles/code-ready/tasks/main.yaml
rename to roles/codeready/tasks/main.yaml
diff --git a/roles/codeready/tasks/new_limits.yml b/roles/codeready/tasks/new_limits.yml
new file mode 100644
index 00000000..4dab4ef3
--- /dev/null
+++ b/roles/codeready/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for codeready
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ che_namespace }}"
+ resources: "{{ codeready_resources }}"
+ when: (codeready_resources | d([], true) | length) > 0
diff --git a/roles/code-ready/tasks/uninstall.yml b/roles/codeready/tasks/uninstall.yml
similarity index 100%
rename from roles/code-ready/tasks/uninstall.yml
rename to roles/codeready/tasks/uninstall.yml
diff --git a/roles/code-ready/tasks/upgrade_1.0_to_1.2.yml b/roles/codeready/tasks/upgrade_1.0_to_1.2.yml
similarity index 100%
rename from roles/code-ready/tasks/upgrade_1.0_to_1.2.yml
rename to roles/codeready/tasks/upgrade_1.0_to_1.2.yml
diff --git a/roles/codeready/tasks/upgrade_images.yml b/roles/codeready/tasks/upgrade_images.yml
new file mode 100644
index 00000000..77e97ee4
--- /dev/null
+++ b/roles/codeready/tasks/upgrade_images.yml
@@ -0,0 +1,30 @@
+---
+- name: "patch {{ che_deployment_name }} deployment"
+ shell: 'oc patch deployment {{ che_deployment_name }} --patch=''{"spec":{"template":{"spec":{"containers":[{"name":"{{ che_deployment_name }}","imagePullPolicy":"Always" }]}}}}'' -n {{ che_namespace }}'
+ register: patch_codeready_result
+ failed_when: patch_codeready_result.stderr != ""
+
+- name: "patch {{ che_postgres_deployment_name }} deployment"
+ shell: 'oc patch deployment {{ che_postgres_deployment_name }} --patch=''{"spec":{"template":{"spec":{"containers":[{"name":"{{ che_postgres_deployment_name }}", "image":"{{ che_postgre_image_name }}", "imagePullPolicy":"Always"}]}}}}'' -n {{ che_namespace }}'
+ register: patch_postgres_result
+ failed_when: patch_postgres_result.stderr != ""
+
+- name: Patch {{ che_operator_deployment_name }} Deployment
+ shell: oc patch deployment {{ che_operator_deployment_name }} --patch='{"spec":{"template":{"spec":{"containers":[{"name":"{{ che_operator_deployment_name }}","image":"{{ che_operator_image_name }}"}]}}}}' --namespace {{ che_namespace }}
+ register: codeready_operator_image_patch_result
+ failed_when: codeready_operator_image_patch_result.stderr != ""
+
+- name: "redeploy {{ che_deployment_name }}"
+ shell: 'oc patch deployment {{ che_deployment_name }} --patch ''{"spec":{"template":{"metadata":{"annotations":{"last-restart":"`date +''%s''`"}}}}}'' -n {{ che_namespace }}'
+ register: redeploy_codeready_result
+ failed_when: redeploy_codeready_result.stderr != ""
+
+- name: "redeploy {{ che_postgres_deployment_name }}"
+ shell: 'oc patch deployment {{ che_postgres_deployment_name }} --patch ''{"spec":{"template":{"metadata":{"annotations":{"last-restart":"`date +''%s''`"}}}}}'' -n {{ che_namespace }}'
+ register: redeploy_postgres_result
+ failed_when: redeploy_postgres_result.stderr != ""
+
+- name: "redeploy {{ che_operator_deployment_name }}"
+ shell: 'oc patch deployment {{ che_operator_deployment_name }} --patch ''{"spec":{"template":{"metadata":{"annotations":{"last-restart":"`date +''%s''`"}}}}}'' -n {{ che_namespace }}'
+ register: redeploy_codeready_operator_result
+ failed_when: redeploy_codeready_operator_result.stderr != ""
diff --git a/roles/code-ready/templates/config.yaml b/roles/codeready/templates/config.yaml
similarity index 100%
rename from roles/code-ready/templates/config.yaml
rename to roles/codeready/templates/config.yaml
diff --git a/roles/code-ready/templates/keycloak/client.json b/roles/codeready/templates/keycloak/client.json
similarity index 100%
rename from roles/code-ready/templates/keycloak/client.json
rename to roles/codeready/templates/keycloak/client.json
diff --git a/roles/customisation/tasks/main.yaml b/roles/customisation/tasks/main.yaml
index 4f784b5b..9454ff29 100644
--- a/roles/customisation/tasks/main.yaml
+++ b/roles/customisation/tasks/main.yaml
@@ -95,11 +95,11 @@
- name: Get codeready secure route
shell: oc get route/codeready -o template --template \{\{.spec.host\}\} -n {{ che_namespace }}
register: che_secure_route
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- set_fact:
che_secure_route: "https://{{che_secure_route.stdout}}"
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- name: set che component
set_fact:
@@ -107,11 +107,11 @@
- name: codeready
version: "{{che_version}}"
host: "{{che_secure_route}}"
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- set_fact:
component_manifests: "{{component_manifests}} + {{che_manifest}}"
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- name: Find encrypted RH-SSO route
shell: for route in $(oc get routes -n {{ rhsso_namespace}} | awk '{print $1}' | grep 'sso' | grep -v 'NAME'); do term=$(oc get route $route -n {{ rhsso_namespace }} -o template --template \{\{.spec.tls.termination\}\}); if [ "$term" == "edge" ] || [ "$term" == "reencrypt" ]; then echo $route; break; fi; done
diff --git a/roles/datasync/tasks/main.yml b/roles/datasync/tasks/main.yml
index 74621c1d..9a4ee3f4 100644
--- a/roles/datasync/tasks/main.yml
+++ b/roles/datasync/tasks/main.yml
@@ -1,6 +1,10 @@
---
- name: "Install Mobile Developer Services Data Sync Application template"
- shell: oc apply -f {{ datasync_app_template }} -n {{ datasync_template_namespace }}
+ shell: oc create -f {{ datasync_app_template }} -n {{ datasync_template_namespace }}
+ register: data_sync_app
+ failed_when: data_sync_app.stderr != '' and 'AlreadyExists' not in data_sync_app.stderr
- name: "Install Mobile Developer Services Data Sync Showcase template"
- shell: oc apply -f {{ datasync_showcase_template }} -n {{ datasync_template_namespace }}
\ No newline at end of file
+ shell: oc create -f {{ datasync_showcase_template }} -n {{ datasync_template_namespace }}
+ register: data_sync_showcase
+ failed_when: data_sync_showcase.stderr != '' and 'AlreadyExists' not in data_sync_showcase.stderr
\ No newline at end of file
diff --git a/roles/enmasse/defaults/main.yml b/roles/enmasse/defaults/main.yml
index a8639d6e..fdaa4816 100644
--- a/roles/enmasse/defaults/main.yml
+++ b/roles/enmasse/defaults/main.yml
@@ -11,13 +11,9 @@ enmasse_clean_artifacts: true
enmasse_backup_postgres_secret: 'enmasse-postgres-secret'
enmasse_postgres_cronjob_name: 'enmasse-postgres-backup'
enmasse_pv_cronjob_name: 'enmasse-pv-backup'
+enmasse_postgresql_image: 'postgresql:9.6'
enmasse_resources:
- - name: postgresql
- kind: dc
- resources:
- requests:
- memory: 50Mi
- name: api-server
kind: deploy
resources:
@@ -28,14 +24,13 @@ enmasse_resources:
resources:
requests:
memory: 75M
- # keycloak operator undoes any externally made changes
- # - name: keycloak
- # kind: deploy
- # resources:
- # requests:
- # memory: 750M
+ - name: postgresql
+ kind: dc
+ resources:
+ requests:
+ memory: 50Mi
- name: service-broker
kind: deploy
resources:
requests:
- memory: 200M
+ memory: 200Mi
diff --git a/roles/enmasse/tasks/new_limits.yml b/roles/enmasse/tasks/new_limits.yml
index 3031b4b8..e30c9185 100644
--- a/roles/enmasse/tasks/new_limits.yml
+++ b/roles/enmasse/tasks/new_limits.yml
@@ -5,4 +5,4 @@
vars:
ns: "{{ enmasse_namespace }}"
resources: "{{ enmasse_resources }}"
-
+ when: (enmasse_resources | d([], true) | length) > 0
diff --git a/roles/enmasse/tasks/upgrade.yml b/roles/enmasse/tasks/upgrade_images.yml
similarity index 59%
rename from roles/enmasse/tasks/upgrade.yml
rename to roles/enmasse/tasks/upgrade_images.yml
index a70b15ef..38685c5e 100644
--- a/roles/enmasse/tasks/upgrade.yml
+++ b/roles/enmasse/tasks/upgrade_images.yml
@@ -1,16 +1,22 @@
--
- name: "get all deployments"
+---
+- name: "get all deployments"
shell: "oc get deployments -o custom-columns=NAME:{.metadata.name} --no-headers -n {{ enmasse_namespace }}"
register: deployments_result
failed_when: deployments_result.stderr != ""
--
- set_fact:
+- set_fact:
deployments: "{{ deployments_result.stdout.splitlines() }}"
--
- name: "patch all deployments"
+- name: "patch all deployments"
shell: oc patch deployment {{ item }} -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"last-restart\":\"`date +'%s'`\"}}}}}" -n {{ enmasse_namespace }}
register: patch_result
failed_when: patch_result.stderr != ""
- with_items: "{{ deployments }}"
\ No newline at end of file
+ with_items: "{{ deployments }}"
+
+- name: "Postgresql update the imagestreams"
+ shell: oc import-image {{ enmasse_postgresql_image }} -n openshift
+ register: result
+ until: result.stdout
+ retries: 50
+ delay: 1
+ failed_when: not result.stdout
diff --git a/roles/fuse/defaults/main.yml b/roles/fuse/defaults/main.yml
index 3c10423b..528cf409 100644
--- a/roles/fuse/defaults/main.yml
+++ b/roles/fuse/defaults/main.yml
@@ -5,4 +5,4 @@ fuse_templates_url: https://raw.githubusercontent.com/jboss-fuse/application-tem
msbroker_namespace: "{{ eval_msbroker_namespace | default('managed-service-broker')}}"
-fuse_pull_secret_name: "syndesis-pull-secret"
\ No newline at end of file
+fuse_pull_secret_name: "syndesis-pull-secret"
diff --git a/roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml b/roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml
index 725cc289..7a0aca3b 100644
--- a/roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml
+++ b/roles/fuse/tasks/_upgrade_fuse_online_imagestreams.yml
@@ -21,34 +21,40 @@
- "s2i"
- name: Add new tag {{ fuse_upgrade_image_tag }} to Fuse online server image
- shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-server:1.4-14 fuse-ignite-server:{{ fuse_upgrade_image_tag }} -n openshift"
+ shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-server:1.4-17 fuse-ignite-server:{{ fuse_upgrade_image_tag }} -n openshift"
- name: Add new tag {{ fuse_upgrade_image_tag }} to fuse-ignite-ui image
- shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-ui:1.4-6 fuse-ignite-ui:{{ fuse_upgrade_image_tag }} -n openshift"
+ shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-ui:1.4-9 fuse-ignite-ui:{{ fuse_upgrade_image_tag }} -n openshift"
+
+- name: Add new tag {{ fuse_upgrade_image_tag }} to Fuse online meta image
+ shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-meta:1.4-16 fuse-ignite-meta:{{ fuse_upgrade_image_tag }} -n openshift"
+
+- name: Add new tag {{ fuse_upgrade_image_tag }} to fuse-ignite-s2i image
+ shell: "oc tag --source docker {{ fuse_registry }}/fuse7/fuse-ignite-s2i:1.4-16 fuse-ignite-s2i:{{ fuse_upgrade_image_tag }} -n openshift"
- name: Import new fuse-ignite-server image
- shell: "oc import-image fuse-ignite-server:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-server:1.4-14 -n openshift"
+ shell: "oc import-image fuse-ignite-server:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-server:1.4-17 -n openshift"
register: import_image_result
until: import_image_result.rc == 0
retries: 5
delay: 5
- name: Import new fuse-ignite-meta image
- shell: "oc import-image fuse-ignite-meta:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-meta:1.4-13 -n openshift"
+ shell: "oc import-image fuse-ignite-meta:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-meta:1.4-16 -n openshift"
register: import_image_result
until: import_image_result.rc == 0
retries: 5
delay: 5
- name: Import new fuse-ignite-s2i image
- shell: "oc import-image fuse-ignite-s2i:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-s2i:1.4-13 -n openshift"
+ shell: "oc import-image fuse-ignite-s2i:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-s2i:1.4-16 -n openshift"
register: import_image_result
until: import_image_result.rc == 0
retries: 5
delay: 5
- name: Import new fuse-ignite-ui image
- shell: "oc import-image fuse-ignite-ui:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-ui:1.4-6 -n openshift"
+ shell: "oc import-image fuse-ignite-ui:{{ fuse_upgrade_image_tag }} --confirm='true' {{ fuse_registry }}/fuse7/fuse-ignite-ui:1.4-9 -n openshift"
register: import_image_result
until: import_image_result.rc == 0
retries: 5
@@ -61,5 +67,4 @@
shell: "oc patch imagestream/prometheus -n openshift --type json -p '[{\"op\": \"add\", \"path\": \"/spec/tags/0/from/name\", \"value\": \"{{ fuse_registry }}/openshift3/prometheus:v3.9.25\"}]'"
- name: Patch postgres exporter imagestream
- shell: "oc patch imagestream/postgres_exporter -n openshift --type json -p '[{\"op\": \"add\", \"path\": \"/spec/tags/0/from/name\", \"value\": \"{{ fuse_registry }}/fuse7-tech-preview/fuse-postgres-exporter:1.4-4\"}]'"
-
+ shell: "oc patch imagestream/postgres_exporter -n openshift --type json -p '[{\"op\": \"add\", \"path\": \"/spec/tags/0/from/name\", \"value\": \"{{ fuse_registry }}/fuse7-tech-preview/fuse-postgres-exporter:1.4-4\"}]'"
\ No newline at end of file
diff --git a/roles/fuse/tasks/main.yml b/roles/fuse/tasks/main.yml
index 5a77c1a9..94436f74 100644
--- a/roles/fuse/tasks/main.yml
+++ b/roles/fuse/tasks/main.yml
@@ -6,7 +6,6 @@
failed_when: create_fuse_imagestream_cmd.stderr != '' and 'AlreadyExists' not in create_fuse_imagestream_cmd.stderr
changed_when: create_fuse_imagestream_cmd.rc == 0
with_items:
- - "{{ fuse_online_imagestream_resources }}"
- "{{ fuse_on_openshift_imagestreams_url }}"
# SETUP FUSE ON OPENSHIFT TEMPLATES
@@ -45,4 +44,4 @@
- "fuse-apicurito.yml"
register: create_fuse_console_template_cmd
failed_when: create_fuse_console_template_cmd.stderr != '' and 'AlreadyExists' not in create_fuse_console_template_cmd.stderr
- changed_when: create_fuse_console_template_cmd.rc == 0
\ No newline at end of file
+ changed_when: create_fuse_console_template_cmd.rc == 0
diff --git a/roles/fuse/tasks/uninstall.yml b/roles/fuse/tasks/uninstall.yml
index c949df78..43e83277 100644
--- a/roles/fuse/tasks/uninstall.yml
+++ b/roles/fuse/tasks/uninstall.yml
@@ -1,15 +1,13 @@
---
-# CLEANUP FUSE OPERATOR AND FUSE ON OPENSHIFT IMAGE STREAMS
+# CLEANUP FUSE ON OPENSHIFT IMAGE STREAMS
- name: Delete Fuse image streams
shell: "oc delete -f {{ item }} -n openshift"
register: delete_fuse_imagestream_cmd
failed_when: delete_fuse_imagestream_cmd.stderr != '' and 'NotFound' not in delete_fuse_imagestream_cmd.stderr
changed_when: delete_fuse_imagestream_cmd.rc == 0
with_items:
- - "{{ fuse_online_imagestream_resources }}"
- "{{ fuse_on_openshift_imagestreams_url }}"
-
# CLEANUP FUSE ON OPENSHIFT TEMPLATES
- name: Delete Fuse quickstart templates
shell: oc delete -f {{ fuse_templates_url }}/{{ item }} -n openshift
diff --git a/roles/fuse/tasks/upgrade.yml b/roles/fuse/tasks/upgrade.yml
index b108e62a..2cb5f433 100644
--- a/roles/fuse/tasks/upgrade.yml
+++ b/roles/fuse/tasks/upgrade.yml
@@ -3,15 +3,6 @@
old_fuse_templates_url: "https://raw.githubusercontent.com/jboss-fuse/application-templates/{{ old_fuse_tag }}/quickstarts"
old_fuse_resource_base: "https://raw.githubusercontent.com/jboss-fuse/application-templates/{{ old_fuse_tag }}"
-- name: Update Fuse Online image streams
- include: _upgrade_fuse_online_imagestreams.yml
-
-- name: Create new Fuse on Openshift images
- shell: "oc replace --force -f {{ fuse_on_openshift_imagestreams_url }} -n openshift"
- register: create_fuse_imagestream_cmd
- failed_when: create_fuse_imagestream_cmd.stderr != '' and 'AlreadyExists' not in create_fuse_imagestream_cmd.stderr
- changed_when: create_fuse_imagestream_cmd.rc == 0
-
# Delete old templates
- name: Delete old fuse quickstart templates
shell: oc delete -f {{ old_fuse_templates_url }}/{{ item }} -n openshift
@@ -78,8 +69,6 @@
- name: Create Fuse Console templates
shell: oc replace --force -f {{ fuse_resource_base }}/{{ item }} -n openshift
with_items:
- - "fis-console-cluster-template.json"
- - "fis-console-namespace-template.json"
- "fuse-apicurito.yml"
register: create_fuse_console_template_cmd
failed_when: create_fuse_console_template_cmd.stderr != '' and 'AlreadyExists' not in create_fuse_console_template_cmd.stderr
@@ -92,8 +81,4 @@
name: imagestream_pull_secret
vars:
namespace: "{{ msbroker_namespace }}"
- product_ns_pull_secret_name: "{{ fuse_pull_secret_name }}"
-
-- name: Update the fuse operator resources url for msbroker
- shell: "oc set env deployment/msb FUSE_OPERATOR_RESOURCES_URL={{ fuse_online_operator_resources }} -n {{ msbroker_namespace }}"
-
\ No newline at end of file
+ product_ns_pull_secret_name: "{{ fuse_pull_secret_name }}"
\ No newline at end of file
diff --git a/roles/fuse/tasks/upgrade_images.yml b/roles/fuse/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/fuse/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/fuse_managed/defaults/main.yml b/roles/fuse_managed/defaults/main.yml
index b5e258a8..2fabb5f2 100644
--- a/roles/fuse_managed/defaults/main.yml
+++ b/roles/fuse_managed/defaults/main.yml
@@ -3,4 +3,44 @@ fuse_namespace: "{{ eval_managed_fuse_namespace }}"
fuse_cr_name: fuse-managed
fuse_backup_postgres_secret_name: fuse-postgres-auth
-fuse_pull_secret_name: "syndesis-pull-secret"
\ No newline at end of file
+fuse_pull_secret_name: "syndesis-pull-secret"
+
+fuse_resources: []
+
+fuse_image_streams:
+ - name: "fuse-ignite-server"
+ image: "registry.redhat.io/fuse7/fuse-ignite-server:1.4-17"
+ tag: "1.7"
+ namespace: "openshift"
+ - name: "fuse-ignite-ui"
+ image: "registry.redhat.io/fuse7/fuse-ignite-ui:1.4-9"
+ tag: "1.7"
+ namespace: "openshift"
+ - name: "fuse-ignite-meta"
+ image: "registry.redhat.io/fuse7/fuse-ignite-meta:1.4-16"
+ tag: "1.7"
+ namespace: "openshift"
+ - name: "fuse-ignite-s2i"
+ image: "registry.redhat.io/fuse7/fuse-ignite-s2i:1.4-16"
+ tag: "1.7"
+ namespace: "openshift"
+ - name: "postgres_exporter"
+ image: "registry.redhat.io/fuse7-tech-preview/fuse-postgres-exporter:1.4-4"
+ tag: "v0.4.7"
+ namespace: "openshift"
+ - name: "oauth-proxy"
+ image: "registry.redhat.io/openshift4/ose-oauth-proxy:4.1"
+ tag: "v1.1.0"
+ namespace: "openshift"
+ - name: "prometheus"
+ image: "registry.access.redhat.com/openshift3/prometheus:v3.9.25"
+ tag: "v2.1.0"
+ namespace: "openshift"
+ - name: "jboss-amq-63"
+ image: "registry.access.redhat.com/jboss-amq-6/amq63-openshift:1.3"
+ tag: "1.3"
+ namespace: "{{ fuse_namespace }}"
+ - name: "fuse-online-operator"
+ image: "registry.redhat.io/fuse7/fuse-online-operator:1.4-16"
+ tag: "1.7"
+ namespace: "{{ fuse_namespace }}"
diff --git a/roles/fuse_managed/tasks/main.yml b/roles/fuse_managed/tasks/main.yml
index 6c470363..b60c7c33 100644
--- a/roles/fuse_managed/tasks/main.yml
+++ b/roles/fuse_managed/tasks/main.yml
@@ -35,20 +35,29 @@
namespace: "{{ fuse_namespace }}"
product_ns_pull_secret_name: "{{ fuse_pull_secret_name }}"
-- name: Create Syndesis CRD
- shell: oc apply -f {{ fuse_online_crd_resources }}
-
-- name: Create Fuse image streams
- shell: "oc replace --force -f {{ fuse_online_imagestream_resources }} -n openshift"
- register: fuse_create_imagestream
- failed_when: fuse_create_imagestream.stderr != '' and 'AlreadyExists' not in fuse_create_imagestream.stderr
- changed_when: fuse_create_imagestream.rc == 0
-
-- name: Create fuse Operator resources in {{ fuse_namespace }}
- shell: oc apply -f {{ fuse_online_operator_resources }} -n {{ fuse_namespace }}
-
-- name: TODO-REMOVE-THIS-AFTER-FUSE-ONLINE-INCLUDES-THESE-ARGS Patch args on operator to include monitoring resources addon
- shell: oc patch dc/syndesis-operator -p='{"spec":{"template":{"spec":{"containers":[{"name":"syndesis-operator","args":["-a","/conf/addons"]}]}}}}}' -n {{ fuse_namespace }}
+- name: Link imagestream pull secret for images
+ shell: oc secrets link default {{ fuse_pull_secret_name }} --for=pull -n {{ fuse_namespace }}
+
+- name: Download fuse binary
+ get_url:
+ url: https://github.com/jboss-fuse/fuse-clients/releases/download/{{ fuse_online_release_tag }}/syndesis-{{ fuse_online_release_tag }}-linux-64bit.tar.gz
+ dest: /tmp/fuse-binary-archive
+
+- name: Create directory for extraction
+ file:
+ path: /tmp/fuse-binary
+ state: directory
+
+- name: Extract fuse binary
+ unarchive:
+ src: /tmp/fuse-binary-archive
+ dest: /tmp/fuse-binary
+
+- name: Run operator cluster install
+ shell: "/tmp/fuse-binary/syndesis-operator install cluster --namespace {{ fuse_namespace }}"
+
+- name: Run operator install
+ shell: "/tmp/fuse-binary/syndesis-operator install operator --namespace {{ fuse_namespace }}"
- template:
src: syndesis-customresource.yml.j2
diff --git a/roles/fuse_managed/tasks/new_limits.yml b/roles/fuse_managed/tasks/new_limits.yml
new file mode 100644
index 00000000..01481e37
--- /dev/null
+++ b/roles/fuse_managed/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for fuse
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ fuse_namespace }}"
+ resources: "{{ fuse_resources }}"
+ when: (fuse_resources | d([], true) | length) > 0
diff --git a/roles/fuse_managed/tasks/uninstall.yml b/roles/fuse_managed/tasks/uninstall.yml
index d857657e..ad2b594f 100644
--- a/roles/fuse_managed/tasks/uninstall.yml
+++ b/roles/fuse_managed/tasks/uninstall.yml
@@ -1,11 +1,4 @@
---
-- name: Delete Fuse image streams
- shell: "oc delete -f {{ fuse_online_imagestream_resources }} -n openshift"
- register: fuse_delete_imagestream
- failed_when: fuse_delete_imagestream.stderr != '' and 'NotFound' not in fuse_delete_imagestream.stderr
- changed_when: fuse_delete_imagestream.rc == 0
- when: fuse_delete_imagestreams | default(true) | bool
-
- name: Remove the Fuse Managed namespace
shell: oc delete project {{ fuse_namespace }}
failed_when: false
diff --git a/roles/fuse_managed/tasks/upgrade.yml b/roles/fuse_managed/tasks/upgrade.yml
index 61e8b8ec..89e77de2 100644
--- a/roles/fuse_managed/tasks/upgrade.yml
+++ b/roles/fuse_managed/tasks/upgrade.yml
@@ -1,104 +1,23 @@
---
-- set_fact:
- fuse_online_template: https://raw.githubusercontent.com/syndesisio/fuse-online-install/{{ fuse_online_release_tag }}/resources/fuse-online-template.yml
-
-# Used to pull images from registry.redhat.io
-- name: Expose vars
- include_vars: "{{ role_path }}/../imagestream_pull_secret/defaults/main.yml"
-- include_role:
- name: imagestream_pull_secret
- vars:
- namespace: "{{ fuse_namespace }}"
- product_ns_pull_secret_name: "{{ fuse_pull_secret_name }}"
-
-- include_role:
- name: fuse
- tasks_from: _upgrade_fuse_online_imagestreams
-
-- name: Update fuse operator resources in {{ fuse_namespace }}
- shell: oc replace --force -f {{ fuse_online_operator_resources }} -n {{ fuse_namespace }}
-
-- name: TODO-REMOVE-THIS-AFTER-FUSE-ONLINE-INCLUDES-THESE-ARGS Patch args on operator to include monitoring resources addon
- shell: oc patch dc/syndesis-operator -p='{"spec":{"template":{"spec":{"containers":[{"name":"syndesis-operator","args":["-a","/conf/addons"]}]}}}}}' -n {{ fuse_namespace }}
-
-# Upgrade syndesis resources. This should bypass the upgrade phase of the operator
-- name: Update and create new resources added to the fuse online template
- block:
- - name: Get syndesis global config secret params
- shell: oc get secret syndesis-global-config -o template --template \{\{.data.params\}\} -n {{ fuse_namespace }} | base64 --decode > /tmp/syndesis-params.yaml
- - name: Format syndesis parameter file
- shell: "sed -i 's/^/-p /' /tmp/syndesis-params.yaml"
- - shell: sed -i -z "s/\\n/ /g" /tmp/syndesis-params.yaml
- - name: Get syndesis parameters
- shell: "cat /tmp/syndesis-params.yaml"
- register: syndesis_parameters
- - name: Create new resources added to the fuse online template
- shell: "oc process -n {{ fuse_namespace }} -f {{ fuse_online_template }} -p SAR_PROJECT={{ fuse_namespace }} -p MAX_INTEGRATIONS_PER_USER=0 {{ syndesis_parameters.stdout }} | oc apply -n {{ fuse_namespace }} -f -"
- register: update_fuse_online_template_resources
- failed_when: update_fuse_online_template_resources.stderr != '' and 'Warning' not in update_fuse_online_template_resources.stderr and 'metadata.resourceVersion' not in update_fuse_online_template_resources.stderr
-
-- name: Verify Fuse upgrade succeeded
- shell: oc get pods -n {{ fuse_namespace }} --selector="app=syndesis" -o jsonpath='{.items[*].status.containerStatuses[?(@.ready==true)].ready}' | wc -w
- register: fuse_verify_result
- until: fuse_verify_result.stdout.find("8") != -1
- retries: 50
- delay: 10
- changed_when: False
-
-# Link Syndesis pull secret to service accounts
-- name: Get Syndesis service accounts
- shell: oc get serviceaccounts -n {{ fuse_namespace }} | grep syndesis | awk '{print $1}'
- register: fuse_serviceaccounts
-
-- name: Link syndesis-pull-secret to fuse service accounts for image pull
- shell: "oc secrets link {{ item }} {{ fuse_pull_secret_name }} --for=pull -n {{ fuse_namespace }}"
- with_items: "{{ fuse_serviceaccounts.stdout_lines }}"
-
-- name: Link syndesis-pull-secret to builder service account
- shell: "oc secrets link builder {{ fuse_pull_secret_name }} --for=pull,mount -n {{ fuse_namespace }}"
-
-- name: Expose controllers via 3Scale
- shell: "oc set env dc syndesis-server CONTROLLERS_EXPOSE_VIA3SCALE=true -n {{ fuse_namespace }}"
-
-# Update Syndesis CR to 1.4
-- name: Update version of Fuse custom resource to 1.4
- shell: "oc patch syndesis/fuse-managed --type=json --patch='[{\"op\": \"replace\", \"path\": \"/status/version\", \"value\": \"1.4\"}]' -n {{ fuse_namespace }}"
-
-- name: Get fuse monitoring resources
- get_url:
- url: "https://raw.githubusercontent.com/syndesisio/syndesis/1.7.x/install/addons/{{ item }}"
- dest: "/tmp/{{ item }}"
- with_items:
- - syndesis-db-dashboard.yml
- - syndesis-db-prometheus-rule.yml
- - syndesis-db-servicemonitor.yml
- - syndesis-integration-dashboard.yml
- - syndesis-integrations-service.yml
- - syndesis-integrations-servicemonitor.yml
- - syndesis-jvm-dashboard.yml
- - syndesis-legacy-ui-serviceaccount.yml
- - syndesis-legacy-ui.yml
- - syndesis-meta-servicemonitor.yml
- - syndesis-rest-api-dashboard.yml
- - syndesis-rest-api-prometheus-rule.yml
- - syndesis-server-servicemonitor.yml
-
-- name: Create fuse monitoring resources
- shell: "oc create -f /tmp/{{ item }} -n {{ fuse_namespace }}"
- register: fuse_monitoring_resource
- failed_when: fuse_monitoring_resource.stderr != '' and 'already exists' not in fuse_monitoring_resource.stderr
- with_items:
- - syndesis-db-dashboard.yml
- - syndesis-db-prometheus-rule.yml
- - syndesis-db-servicemonitor.yml
- - syndesis-integration-dashboard.yml
- - syndesis-integrations-service.yml
- - syndesis-integrations-servicemonitor.yml
- - syndesis-jvm-dashboard.yml
- - syndesis-legacy-ui-serviceaccount.yml
- - syndesis-legacy-ui.yml
- - syndesis-meta-servicemonitor.yml
- - syndesis-rest-api-dashboard.yml
- - syndesis-rest-api-prometheus-rule.yml
- - syndesis-server-servicemonitor.yml
\ No newline at end of file
+- debug: msg="Upgrade work to be carried out in INTLY-4092, the process has changed in latest fuse release"
+# - name: Download fuse binary
+# get_url:
+# url: https://github.com/jboss-fuse/fuse-clients/releases/download/{{ fuse_online_release_tag }}/syndesis-{{ fuse_online_release_tag }}-linux-64bit.tar.gz
+# dest: /tmp/fuse-binary-archive
+
+# - name: Create directory for extraction
+# file:
+# path: /tmp/fuse-binary
+# state: directory
+
+# - name: Extract fuse binary
+# unarchive:
+# src: /tmp/fuse-binary-archive
+# dest: /tmp/fuse-binary
+
+# - name: Run operator cluster install
+# shell: "/tmp/fuse-binary/syndesis-operator install cluster --namespace {{ fuse_namespace }}"
+
+# - name: Run operator install
+# shell: "/tmp/fuse-binary/syndesis-operator install operator --namespace {{ fuse_namespace }}"
\ No newline at end of file
diff --git a/roles/fuse_managed/tasks/upgrade_images.yml b/roles/fuse_managed/tasks/upgrade_images.yml
new file mode 100644
index 00000000..9d2fa05f
--- /dev/null
+++ b/roles/fuse_managed/tasks/upgrade_images.yml
@@ -0,0 +1,12 @@
+---
+- name: Add new tag to Fuse online images
+ shell: "oc tag --source docker {{ patch_image_stream_item.image }} {{ patch_image_stream_item.name }}:{{ patch_image_stream_item.tag }} -n {{ patch_image_stream_item.namespace }}"
+ with_items: "{{ fuse_image_streams }}"
+ loop_control:
+ loop_var: patch_image_stream_item
+
+- name: Import new Fuse online images
+ shell: "oc import-image {{ patch_image_stream_item.name }}:{{ patch_image_stream_item.tag }} --confirm='true' {{ patch_image_stream_item.image }} -n {{ patch_image_stream_item.namespace }}"
+ with_items: "{{ fuse_image_streams }}"
+ loop_control:
+ loop_var: patch_image_stream_item
diff --git a/roles/fuse_managed/templates/syndesis-customresource.yml.j2 b/roles/fuse_managed/templates/syndesis-customresource.yml.j2
index 62691845..aa7ad1a1 100644
--- a/roles/fuse_managed/templates/syndesis-customresource.yml.j2
+++ b/roles/fuse_managed/templates/syndesis-customresource.yml.j2
@@ -4,7 +4,7 @@ metadata:
name: {{ fuse_cr_name }}
spec:
components:
- db:
+ database:
resources: {}
meta:
resources: {}
@@ -16,6 +16,6 @@ spec:
resources: {}
upgrade:
resources: {}
- imageStreamNamespace: openshift
- integration:
- limit: 0
+ addons:
+ ops:
+ enabled: "true"
diff --git a/roles/gitea/tasks/upgrade_images.yml b/roles/gitea/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/gitea/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/launcher/defaults/main.yml b/roles/launcher/defaults/main.yml
index 62ba76e2..2e8c0bb2 100644
--- a/roles/launcher/defaults/main.yml
+++ b/roles/launcher/defaults/main.yml
@@ -33,4 +33,6 @@ github_repo_description: "Generated by Red Hat Developer Launch"
launcher_backup_postgres_secret_name: "launcher-postgres-auth"
launcher_frontend_route_name: "launcher"
-launcher_frontend_protocol: "https"
\ No newline at end of file
+launcher_frontend_protocol: "https"
+
+launcher_resources: []
diff --git a/roles/launcher/tasks/new_limits.yml b/roles/launcher/tasks/new_limits.yml
new file mode 100644
index 00000000..255119c5
--- /dev/null
+++ b/roles/launcher/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for launcher
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ launcher_namespace }}"
+ resources: "{{ launcher_resources }}"
+ when: (launcher_resources | d([], true) | length) > 0
diff --git a/roles/launcher/tasks/upgrade_images.yml b/roles/launcher/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/launcher/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/launcher/tasks/upgrade_sso_7.2_to_7.3.yml b/roles/launcher/tasks/upgrade_sso_7.2_to_7.3.yml
deleted file mode 100644
index 2d2a888c..00000000
--- a/roles/launcher/tasks/upgrade_sso_7.2_to_7.3.yml
+++ /dev/null
@@ -1,91 +0,0 @@
----
-- name: "Export the existing launcher-sso deploymentconfig"
- shell: "oc get deploymentconfigs launcher-sso -n {{ launcher_namespace }} --export -o json > /tmp/sso_7.2_deploymentconfig.json"
-
-- name: "Capture the environment variables in the launcher-sso deploymentconfig"
- shell: 'jq ".spec.template.spec.containers[0].env" /tmp/sso_7.2_deploymentconfig.json'
- register: launcher_sso_environment_vars
-
-- name: "Patch the launcher-sso-ping service with the serving-cert-secret-name annotation"
- shell: 'oc annotate service launcher-sso-ping "service.alpha.openshift.io/serving-cert-secret-name"="sso-x509-jgroups-secret" --overwrite -n {{ launcher_namespace }}'
-
-- name: "Copy over the Keycloak 7.3 deploymentconfig template"
- template:
- src: sso_7.3_deploymentconfig.json
- dest: /tmp/sso_7.3_deploymentconfig.json
-
-- name: "Delete the existing launcher-sso deploymentconfig"
- shell: "oc delete deploymentconfigs launcher-sso -n {{ launcher_namespace }}"
-
-- name: "Recreate the launcher-sso deploymentconfig"
- shell: "oc apply -f /tmp/sso_7.3_deploymentconfig.json -n {{ launcher_namespace }}"
-
-- name: "Wait for deploymentconfig launcher-sso readiness"
- shell: "oc get dc/launcher-sso -o jsonpath='{.status.availableReplicas}' -n {{ launcher_namespace }}"
- register: launcher_sso_replicas
- until: launcher_sso_replicas.stdout == "1"
- retries: 50
- delay: 10
- failed_when: launcher_sso_replicas.stderr
- changed_when: False
-
-- name: Retrieve launcher SSO Admin Password
- shell: "oc get dc/launcher-sso \
- -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name==\"SSO_ADMIN_PASSWORD\")].value}' \
- -n {{ launcher_namespace }}"
- register: launcher_admin_password_cmd
-
-- set_fact:
- launcher_sso_password: "{{ launcher_admin_password_cmd.stdout }}"
-
-- name: Find encrypted RH-SSO route
- shell: for route in $(oc get routes -n {{ launcher_namespace }} | awk '{print $1}' | grep 'sso' | grep -v 'NAME'); do term=$(oc get route $route -n {{ launcher_namespace }} -o template --template \{\{.spec.tls.termination\}\}); if [ "$term" == "edge" ] || [ "$term" == "reencrypt" ]; then echo $route; break; fi; done
- register: rhsso_secure_route_name
- retries: 60
- delay: 5
- failed_when: rhsso_secure_route_name.stdout == ''
- until: rhsso_secure_route_name.stdout != ''
- when: launcher
-
-- name: Get RH-SSO secure route
- local_action: command oc get route/{{ rhsso_secure_route_name.stdout }} -o template --template \{\{.spec.host\}\} -n {{ launcher_namespace }}
- register: rhsso_secure_route
-
-- set_fact:
- launcher_sso_route: "{{ rhsso_secure_route.stdout }}"
-
-- name: Generate Launcher SSO auth token for admin user on {{ launcher_sso_route }}
- uri:
- url: "https://{{ launcher_sso_route }}/auth/realms/master/protocol/openid-connect/token"
- method: POST
- body: "client_id=admin-cli&username={{ launcher_sso_username }}&password={{ launcher_sso_password }}&grant_type=password"
- validate_certs: "{{ launcher_sso_validate_certs }}"
- register: launcher_sso_auth_response
- retries: 60
- delay: 5
- until: launcher_sso_auth_response.status == 200
-
-- name: Retrieve the GitHub identity provider config from Launcher SSO
- uri:
- url: "https://{{ launcher_sso_route }}/auth/admin/realms/{{ launcher_sso_realm }}/identity-provider/instances/github"
- method: GET
- validate_certs: "{{ launcher_sso_validate_certs }}"
- headers:
- Authorization: "Bearer {{ launcher_sso_auth_response.json.access_token }}"
- status_code: [200]
- register: github_idp_config_output
-
-- name: Update the GitHub identity provider config var
- shell: "echo '{{ github_idp_config_output.json | to_json }}' | jq '.config.defaultScope = \"{{ launcher_github_default_scopes }}\"'"
- register: github_idp_config
-
-- name: Update GitHub identity provider default scopes in Launcher SSO
- uri:
- url: "https://{{ launcher_sso_route }}/auth/admin/realms/{{ launcher_sso_realm }}/identity-provider/instances/github"
- method: PUT
- body: "{{ github_idp_config.stdout }}"
- validate_certs: "{{ launcher_sso_validate_certs }}"
- body_format: json
- headers:
- Authorization: "Bearer {{ launcher_sso_auth_response.json.access_token }}"
- status_code: [204]
diff --git a/roles/launcher/templates/sso_7.3_deploymentconfig.json b/roles/launcher/templates/sso_7.3_deploymentconfig.json
deleted file mode 100644
index c20967c4..00000000
--- a/roles/launcher/templates/sso_7.3_deploymentconfig.json
+++ /dev/null
@@ -1,131 +0,0 @@
-{
- "kind": "DeploymentConfig",
- "apiVersion": "v1",
- "metadata": {
- "name": "launcher-sso",
- "labels": {
- "application": "launcher-sso"
- }
- },
- "spec": {
- "strategy": {
- "type": "Recreate"
- },
- "triggers": [
- {
- "type": "ImageChange",
- "imageChangeParams": {
- "automatic": true,
- "containerNames": [
- "launcher-sso"
- ],
- "from": {
- "kind": "ImageStreamTag",
- "namespace": "openshift",
- "name": "redhat-sso73-openshift:1.0"
- }
- }
- },
- {
- "type": "ConfigChange"
- }
- ],
- "replicas": 1,
- "selector": {
- "deploymentConfig": "launcher-sso"
- },
- "template": {
- "metadata": {
- "name": "launcher-sso",
- "labels": {
- "deploymentConfig": "launcher-sso",
- "application": "launcher-sso"
- }
- },
- "spec": {
- "terminationGracePeriodSeconds": 75,
- "containers": [
- {
- "name": "launcher-sso",
- "image": "launcher-sso",
- "imagePullPolicy": "Always",
- "resources": {
- "limits": {
- "memory": "1Gi"
- }
- },
- "volumeMounts": [
- {
- "name": "sso-x509-https-volume",
- "mountPath": "/etc/x509/https",
- "readOnly": true
- },
- {
- "name": "sso-x509-jgroups-volume",
- "mountPath": "/etc/x509/jgroups",
- "readOnly": true
- }
- ],
- "livenessProbe": {
- "exec": {
- "command": [
- "/bin/bash",
- "-c",
- "/opt/eap/bin/livenessProbe.sh"
- ]
- },
- "initialDelaySeconds": 60
- },
- "readinessProbe": {
- "exec": {
- "command": [
- "/bin/bash",
- "-c",
- "/opt/eap/bin/readinessProbe.sh"
- ]
- }
- },
- "ports": [
- {
- "name": "jolokia",
- "containerPort": 8778,
- "protocol": "TCP"
- },
- {
- "name": "http",
- "containerPort": 8080,
- "protocol": "TCP"
- },
- {
- "name": "https",
- "containerPort": 8443,
- "protocol": "TCP"
- },
- {
- "name": "ping",
- "containerPort": 8888,
- "protocol": "TCP"
- }
- ],
- "env": {{ launcher_sso_environment_vars.stdout }}
- }
- ],
- "volumes": [
- {
- "name": "sso-x509-https-volume",
- "secret": {
- "secretName": "sso-x509-https-secret"
- }
- },
- {
- "name": "sso-x509-jgroups-volume",
- "secret": {
- "secretName": "sso-x509-jgroups-secret"
- }
- }
- ]
- }
- }
- }
-}
-
diff --git a/roles/mdc/defaults/main.yml b/roles/mdc/defaults/main.yml
index bb4edd92..9198d559 100644
--- a/roles/mdc/defaults/main.yml
+++ b/roles/mdc/defaults/main.yml
@@ -3,7 +3,7 @@ mdc_template_dir: /tmp
mdc_namespace: "{{ eval_mdc_namespace | default('mobile-developer-console') }}"
mdc_name: mdc
mdc_oauth_client_id: mobile-developer-console
-mdc_resources:
+mdc_resource_items:
- "{{ mdc_operator_resources }}/service_account.yaml"
- "{{ mdc_operator_resources }}/role.yaml"
- "{{ mdc_operator_resources }}/role_binding.yaml"
@@ -22,7 +22,7 @@ sync_documentation_url: "{{ documentation_url }}/sync.html"
mss_documentation_url: "{{ documentation_url }}/mss.html"
#monitor
-mdc_monitor_resources:
+mdc_monitor_resource_items:
- "{{ mdc_operator_resources }}/monitor/service_monitor.yaml"
- "{{ mdc_operator_resources }}/monitor/mdc_service_monitor.yaml"
mdc_monitor_templates:
@@ -30,3 +30,6 @@ mdc_monitor_templates:
- operator_prometheus_rules
- mdc_prometheus_rules
- mdc_grafana_dashboard
+
+#scaling
+mdc_resources: []
diff --git a/roles/mdc/tasks/install-operator.yml b/roles/mdc/tasks/install-operator.yml
index 8a0cc0ca..bc1f50ff 100644
--- a/roles/mdc/tasks/install-operator.yml
+++ b/roles/mdc/tasks/install-operator.yml
@@ -11,7 +11,7 @@
- name: Install mdc resources
shell: "oc apply -f {{ item }} -n {{ mdc_namespace }}"
- with_items: "{{ mdc_resources }}"
+ with_items: "{{ mdc_resource_items }}"
- set_fact:
mdc_openshift_host: "{{ openshift_master_url }}"
diff --git a/roles/mdc/tasks/monitoring.yml b/roles/mdc/tasks/monitoring.yml
index 2ef22038..045ceb69 100644
--- a/roles/mdc/tasks/monitoring.yml
+++ b/roles/mdc/tasks/monitoring.yml
@@ -1,7 +1,7 @@
---
- name: Create Service Monitor resource
shell: "oc apply -f {{ item }} -n {{ mdc_namespace }}"
- with_items: "{{ mdc_monitor_resources }}"
+ with_items: "{{ mdc_monitor_resource_items }}"
register: output
failed_when: output.stderr != '' and 'already exists' not in output.stderr
diff --git a/roles/mdc/tasks/new_limits.yml b/roles/mdc/tasks/new_limits.yml
new file mode 100644
index 00000000..5128e155
--- /dev/null
+++ b/roles/mdc/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for mdc
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ mdc_namespace }}"
+ resources: "{{ mdc_resources }}"
+ when: (mdc_resources | d([], true) | length) > 0
diff --git a/roles/mdc/tasks/uninstall.yml b/roles/mdc/tasks/uninstall.yml
index 5a369d23..fc17ba39 100644
--- a/roles/mdc/tasks/uninstall.yml
+++ b/roles/mdc/tasks/uninstall.yml
@@ -4,7 +4,6 @@
register: mdc_resource_list_cmd
failed_when: false
with_items:
- - oauthclient
- mobiledeveloperconsole
- keycloakrealm
@@ -18,6 +17,11 @@
failed_when: output.stderr != '' and 'not found' not in output.stderr
with_items: "{{ mdc_resource_list }}"
+- name: Delete mdc oauthclient
+ shell: "oc delete oauthclient {{ mdc_oauth_client_id }}"
+ register: output
+ failed_when: output.stderr != '' and 'not found' not in output.stderr
+
- name: "Wait for resources to be removed"
shell: oc get {{ item }} -n {{ mdc_namespace }}
register: result
diff --git a/roles/mdc/tasks/upgrade_images.yml b/roles/mdc/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/mdc/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/mdc/templates/mdc_prometheus_rules.yaml.j2 b/roles/mdc/templates/mdc_prometheus_rules.yaml.j2
index 624a3c60..43c3db6c 100644
--- a/roles/mdc/templates/mdc_prometheus_rules.yaml.j2
+++ b/roles/mdc/templates/mdc_prometheus_rules.yaml.j2
@@ -16,8 +16,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The MDC has been down for more than 5 minutes. "
- summary: "The mobile-developer-console is down. For more information see on the MDC at https://github.com/aerogear/mobile-developer-console"
+ description: "The MDC has been down for more than 5 minutes."
+ summary: "The mobile-developer-console is down."
sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-mdc.adoc"
- alert: MobileDeveloperConsoleDown
expr: absent(kube_endpoint_address_available{endpoint="{{ mdc_name }}-mdc"} >= 1)
@@ -25,8 +25,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The MDC admin console has been down for more than 5 minutes. "
- summary: "The mobile-developer-console admin console endpoint has been unavailable for more that 5 minutes. For more information see on the MDC at https://github.com/aerogear/mobile-developer-console"
+ description: "The MDC admin console has been down for more than 5 minutes."
+ summary: "The mobile-developer-console admin console endpoint has been unavailable for more that 5 minutes."
sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-mdc.adoc"
- alert: MobileDeveloperConsolePodCPUHigh
expr: "(rate(process_cpu_seconds_total{job='{{ mdc_name }}-mdc'}[1m])) > (((kube_pod_container_resource_limits_cpu_cores{namespace='{{ mdc_namespace }}',container='mdc'})/100)*90)"
@@ -34,8 +34,8 @@ spec:
labels:
severity: warning
annotations:
- description: "The MDC pod has been at 90% CPU usage for more than 5 minutes"
- summary: "The mobile-developer-console is reporting high cpu usage for more that 5 minutes. For more information see on the MDC at https://github.com/aerogear/mobile-developer-console"
+ description: "The MDC pod has been at 90% CPU usage for more than 5 minutes."
+ summary: "The mobile-developer-console is reporting high cpu usage for more that 5 minutes."
sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-mdc.adoc"
- alert: MobileDeveloperConsolePodMemoryHigh
expr: "(process_resident_memory_bytes{job='{{ mdc_name }}-mdc'}) > (((kube_pod_container_resource_limits_memory_bytes{namespace='{{ mdc_namespace }}',container='mdc'})/100)*90)"
@@ -43,6 +43,6 @@ spec:
labels:
severity: warning
annotations:
- description: "The MDC pod has been at 90% memory usage for more than 5 minutes"
- summary: "The mobile-developer-console is reporting high memory usage for more that 5 minutes. For more information see on the MDC at https://github.com/aerogear/mobile-developer-console"
- sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-mdc.adoc"
\ No newline at end of file
+ description: "The MDC pod has been at 90% memory usage for more than 5 minutes."
+ summary: "The mobile-developer-console is reporting high memory usage for more that 5 minutes."
+ sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-mdc.adoc"
diff --git a/roles/mdc/templates/operator_prometheus_rules.yaml.j2 b/roles/mdc/templates/operator_prometheus_rules.yaml.j2
index 1e7898c8..d49e7f15 100644
--- a/roles/mdc/templates/operator_prometheus_rules.yaml.j2
+++ b/roles/mdc/templates/operator_prometheus_rules.yaml.j2
@@ -17,5 +17,5 @@ spec:
severity: critical
annotations:
description: "The MDC Operator has been down for more than 5 minutes."
- summary: "The MDC Operator is down. For more information on the MDC Operator, see https://github.com/aerogear/mobile-developer-console-operator"
+ summary: "The MDC Operator is down."
sop_url: "https://github.com/aerogear/mobile-developer-console-operator/blob/{{ mdc_operator_release_tag }}/SOP/SOP-operator.adoc"
diff --git a/roles/middleware_monitoring/defaults/main.yml b/roles/middleware_monitoring/defaults/main.yml
index ec8acabd..f9fcad24 100644
--- a/roles/middleware_monitoring/defaults/main.yml
+++ b/roles/middleware_monitoring/defaults/main.yml
@@ -7,7 +7,7 @@ monitoring_prometheus_retention: 45d
monitoring_prometheus_storage_request: 10Gi
# Resources to create via the oc tool.
-monitoring_resources:
+monitoring_resource_items:
- "{{ middleware_monitoring_operator_resources }}/crds/BlackboxTarget.yaml"
- "{{ middleware_monitoring_operator_resources }}/crds/ApplicationMonitoring.yaml"
- "{{ middleware_monitoring_operator_resources }}/operator_roles/service_account.yaml"
@@ -40,3 +40,5 @@ monitoring_resource_templates_pre:
monitoring_resource_templates_post:
# Application monitoring resources
- "application_monitoring_cr.yml"
+
+monitoring_resources: []
\ No newline at end of file
diff --git a/roles/middleware_monitoring/tasks/main.yml b/roles/middleware_monitoring/tasks/main.yml
index 4a7dfe06..8c79adac 100644
--- a/roles/middleware_monitoring/tasks/main.yml
+++ b/roles/middleware_monitoring/tasks/main.yml
@@ -15,7 +15,7 @@
- name: Create required operator resources
shell: "oc apply -f {{ item }} -n {{ monitoring_namespace }}"
register: monitoring_resource_create
- with_items: "{{ monitoring_resources }}"
+ with_items: "{{ monitoring_resource_items }}"
- include: ./create_resource_from_template.yml
with_items: "{{ monitoring_resource_templates_post }}"
\ No newline at end of file
diff --git a/roles/middleware_monitoring/tasks/new_limits.yml b/roles/middleware_monitoring/tasks/new_limits.yml
new file mode 100644
index 00000000..9637f247
--- /dev/null
+++ b/roles/middleware_monitoring/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for middleware monitoring
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ monitoring_namespace }}"
+ resources: "{{ monitoring_resources }}"
+ when: (monitoring_resources | d([], true) | length) > 0
diff --git a/roles/middleware_monitoring/tasks/uninstall.yml b/roles/middleware_monitoring/tasks/uninstall.yml
index 779c3585..d470cf99 100644
--- a/roles/middleware_monitoring/tasks/uninstall.yml
+++ b/roles/middleware_monitoring/tasks/uninstall.yml
@@ -9,7 +9,7 @@
shell: "oc delete -f {{ item }} -n {{ monitoring_namespace }}"
register: monitoring_resource_delete
failed_when: monitoring_resource_delete.stderr != '' and 'NotFound' not in monitoring_resource_delete.stderr and "no matches for kind" not in monitoring_resource_delete.stderr
- with_items: "{{ monitoring_resources }}"
+ with_items: "{{ monitoring_resource_items }}"
- name: Delete monitoring namespace
shell: "oc delete project {{ monitoring_namespace }}"
diff --git a/roles/middleware_monitoring/tasks/upgrade/grafana.yml b/roles/middleware_monitoring/tasks/upgrade/grafana.yml
index 3bcc248f..5de22e0a 100644
--- a/roles/middleware_monitoring/tasks/upgrade/grafana.yml
+++ b/roles/middleware_monitoring/tasks/upgrade/grafana.yml
@@ -9,6 +9,7 @@
shell: "oc get deployment grafana-deployment -n {{ monitoring_namespace }}"
register: get_deployment_cmd
failed_when: get_deployment_cmd.rc == 0
+ until: get_deployment_cmd.rc != 0
changed_when: "'NotFound' in get_deployment_cmd.stderr"
retries: 10
delay: 5
@@ -36,3 +37,24 @@
register: delete_cmd
failed_when: delete_cmd.stderr != '' and 'NotFound' not in delete_cmd.stderr
changed_when: delete_cmd.rc == 0
+
+- name: Recreate resources from latest templates
+ include: ./create_resource_from_template.yml
+ with_items:
+ - "grafana-proxy-clusterrole.yml"
+ - "grafana-proxy-clusterrole_binding.yml"
+ - "grafana_cluster_role.yml"
+ - "grafana_cluster_role_binding.yml"
+
+- name: Upgrade CRDs
+ include: ./apply_resource_from_template.yml
+ with_items:
+ - "grafana_crd.yml"
+ - "grafana_dashboard_crd.yml"
+ - "grafana_datasource_crd.yml"
+
+- name: Include rhsso vars
+ include_vars: ../../../rhsso/defaults/main.yml
+
+- name: Label the keycloak dashboard for the grafana operator to discover it
+ shell: "oc label grafanadashboard keycloak monitoring-key=middleware -n {{ rhsso_namespace }} --overwrite"
\ No newline at end of file
diff --git a/roles/middleware_monitoring/tasks/upgrade/prometheus.yml b/roles/middleware_monitoring/tasks/upgrade/prometheus.yml
index e3052721..a97178ac 100644
--- a/roles/middleware_monitoring/tasks/upgrade/prometheus.yml
+++ b/roles/middleware_monitoring/tasks/upgrade/prometheus.yml
@@ -35,33 +35,6 @@
failed_when: po_role_apply_cmd.stderr != '' and 'Warning' not in po_role_apply_cmd.stderr
changed_when: po_role_apply_cmd.rc == 0
-- name: Delete serviceaccounts
- shell: "oc delete serviceaccount {{ item }} -n {{ monitoring_namespace }}"
- register: delete_cmd
- failed_when: delete_cmd.stderr != '' and 'NotFound' not in delete_cmd.stderr
- changed_when: delete_cmd.rc == 0
- with_items:
- - alertmanager
- - prometheus-application-monitoring
-
-- name: Delete routes
- shell: "oc delete route {{ item }} -n {{ monitoring_namespace }}"
- register: delete_cmd
- failed_when: delete_cmd.stderr != '' and 'NotFound' not in delete_cmd.stderr
- changed_when: delete_cmd.rc == 0
- with_items:
- - alertmanager-route
- - prometheus-route
-
-- name: Delete services
- shell: "oc delete service {{ item }} -n {{ monitoring_namespace }}"
- register: delete_cmd
- failed_when: delete_cmd.stderr != '' and 'NotFound' not in delete_cmd.stderr
- changed_when: delete_cmd.rc == 0
- with_items:
- - alertmanager-service
- - prometheus-service
-
- name: Scale up the prometheus operator
shell: "oc scale deployment/prometheus-operator --replicas 1 -n {{ monitoring_namespace }}"
register: po_scale_up_cmd
diff --git a/roles/middleware_monitoring/tasks/upgrade/trigger.yml b/roles/middleware_monitoring/tasks/upgrade/trigger.yml
index 75dd8b5f..fcf1d439 100644
--- a/roles/middleware_monitoring/tasks/upgrade/trigger.yml
+++ b/roles/middleware_monitoring/tasks/upgrade/trigger.yml
@@ -10,7 +10,15 @@
register: aom_role_apply_cmd
failed_when: aom_role_apply_cmd.stderr != '' and 'Warning' not in aom_role_apply_cmd.stderr
changed_when: aom_role_apply_cmd.rc == 0
- with_items: "{{ monitoring_resources }}"
+ with_items: "{{ monitoring_resource_items }}"
+
+- name: Upgrade the prometheus operator roles
+ include: ./apply_resource_from_template.yml
+ with_items:
+ - "prometheus_operator_cluster_role.yml"
+
+- name: Label additional scrape config secret
+ shell: "oc label secret {{ additional_scrape_config_name }} monitoring-key={{ monitoring_label_value }} -n {{ monitoring_namespace }} --overwrite"
- name: Delete the lockfile
shell: "oc delete configmap application-monitoring-operator-lock -n {{ monitoring_namespace }}"
@@ -25,6 +33,11 @@
- set_fact:
aom_cr_name: "{{ get_amo_cr_name_cmd.stdout | trim }}"
+- name: Apply latest application monitoring CR changes
+ include: ./apply_resource_from_template.yml
+ with_items:
+ - "application_monitoring_cr_upgrade.yml"
+
- name: Remove the status from the CR to trigger a reconcile
shell: oc patch applicationmonitoring {{ aom_cr_name }} -n {{ monitoring_namespace }} --type json -p '[{"op":"replace", "path":"/status/phase", "value":0}]'
register: reconcile_cmd
@@ -41,3 +54,50 @@
register: rollout_cmd
failed_when: rollout_cmd.rc != 0
changed_when: rollout_cmd.rc == 0
+
+- name: Wait for Grafana to become available
+ shell: "oc get grafanas grafana -n {{ monitoring_namespace }}"
+ register: get_grafana_cmd
+ failed_when: get_grafana_cmd.rc != 0
+ changed_when: get_grafana_cmd.rc == 0
+ until: "'not found' not in get_grafana_cmd.stdout"
+ retries: 10
+ delay: 5
+
+- name: Patch grafana dashboard selector
+ register: grafana_patch
+ failed_when: grafana_patch.stderr != '' and 'not patched' not in grafana_patch.stderr
+ changed_when: grafana_patch.rc == 0
+ shell: |
+ oc patch -n {{ monitoring_namespace }} grafanas grafana --type=json -p '[
+ {
+ "op": "replace",
+ "path": "/spec/dashboardLabelSelector",
+ "value": [
+ {
+ "matchExpressions": [
+ {
+ "key": "monitoring-key",
+ "operator": "In",
+ "values": ["middleware"]
+ }
+ ]
+ },
+ {
+ "matchLabels": {
+ "app": "syndesis"
+ }
+ }
+ ]}]'
+
+- name: remove prometheus rolebinding
+ include: ./delete_resource_from_template.yml
+ with_items:
+ - "prometheus_cluster_role.yml"
+ - "prometheus_cluster_role_binding.yml"
+
+- name: recreate prometheus rolebinding
+ include: ./create_resource_from_template.yml
+ with_items:
+ - "prometheus_cluster_role.yml"
+ - "prometheus_cluster_role_binding.yml"
diff --git a/roles/middleware_monitoring/tasks/upgrade_images.yml b/roles/middleware_monitoring/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/middleware_monitoring/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2 b/roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2
index 05c774d1..8f613584 100644
--- a/roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2
+++ b/roles/middleware_monitoring/templates/alert_manager_cluster_role_binding.yml.j2
@@ -7,7 +7,7 @@ roleRef:
name: alertmanager-application-monitoring
subjects:
- kind: ServiceAccount
- name: alertmanager
+ name: alertmanager-service-account
namespace: "{{ monitoring_namespace }}"
userNames:
-- system:serviceaccount:{{ monitoring_namespace }}:alertmanager
+- system:serviceaccount:{{ monitoring_namespace }}:alertmanager-service-account
diff --git a/roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2 b/roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2
index f2d0fe9a..08dc41ab 100644
--- a/roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2
+++ b/roles/middleware_monitoring/templates/application_monitoring_cr.yml.j2
@@ -7,4 +7,6 @@ spec:
additionalScrapeConfigSecretName: "integreatly-additional-scrape-configs"
additionalScrapeConfigSecretKey: "integreatly-additional.yaml"
prometheusRetention: {{ monitoring_prometheus_retention }}
- prometheusStorageRequest: {{ monitoring_prometheus_storage_request }}
\ No newline at end of file
+ prometheusStorageRequest: {{ monitoring_prometheus_storage_request }}
+ prometheusInstanceNamespaces: {{ monitoring_namespace }}
+ alertmanagerInstanceNamespaces: {{ monitoring_namespace }}
\ No newline at end of file
diff --git a/roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2 b/roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2
index 7f7303e8..f8f5a99b 100644
--- a/roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2
+++ b/roles/middleware_monitoring/templates/prometheus_cluster_role_binding.yml.j2
@@ -8,7 +8,7 @@ roleRef:
name: prometheus-application-monitoring
subjects:
- kind: ServiceAccount
- name: prometheus-application-monitoring
+ name: prometheus-service-account
namespace: "{{ monitoring_namespace }}"
userNames:
-- system:serviceaccount:{{ monitoring_namespace }}:prometheus-application-monitoring
+- system:serviceaccount:{{ monitoring_namespace }}:prometheus-service-account
diff --git a/roles/middleware_monitoring_config/defaults/main.yml b/roles/middleware_monitoring_config/defaults/main.yml
index 4db908d3..4ccfbf1e 100644
--- a/roles/middleware_monitoring_config/defaults/main.yml
+++ b/roles/middleware_monitoring_config/defaults/main.yml
@@ -25,7 +25,23 @@ monitoring_grafanadashboards_resource_templates:
# PrometheusRules template variables
monitoring_prometheusrules_resource_templates:
- kube_state_metrics_alerts.yml
+- kube_state_metrics_3scale_alerts.yml
+- kube_state_metrics_fuse_online_alerts.yml
# BlackboxTargets template variables
monitoring_blackboxtargets_resource_templates:
-- blackboxtargets.yml
\ No newline at end of file
+- blackboxtargets.yml
+
+# Alertmanager template variables
+alertmanager_to_email: "{{ cluster_mailto | default('[email protected]') }}"
+
+# Sengrid Defaults
+smtp_smarthost: smtp.sendgrid.net:587
+smtp_auth_username: apikey
+smtp_auth_password: "{{ sendgrid_api_key | default('') }}"
+
+# DMS
+dms_webhook_url: "{{ deadmanssnitch_url | default('') }}"
+
+# pagerduty
+pd_service_key: "{{ pagerduty_integration_key | default('') }}"
diff --git a/roles/middleware_monitoring_config/tasks/create_alertmanager.yml b/roles/middleware_monitoring_config/tasks/create_alertmanager.yml
new file mode 100644
index 00000000..7be564d9
--- /dev/null
+++ b/roles/middleware_monitoring_config/tasks/create_alertmanager.yml
@@ -0,0 +1,27 @@
+---
+
+- name: Configure AlertManager
+ block:
+ - name: get alertmanager route for alertmanager config
+ shell: oc get route alertmanager-route -o template --template \{\{.spec.host\}\} -n {{ middleware_monitoring_namespace }}
+ register: alertmanager
+
+ - set_fact:
+ alertmanager_route: "{{alertmanager.stdout}}"
+ when: alertmanager
+
+ - name: Generate custom alertmanager config
+ template:
+ src: "alertmanager.yml.j2"
+ dest: /tmp/alertmanager.yaml
+
+ - name: Create and apply alertmanager-application-monitoring secret
+ shell: oc create secret generic alertmanager-application-monitoring --from-file=/tmp/alertmanager.yaml --dry-run -o yaml | oc apply -n {{ middleware_monitoring_namespace }} -f -
+
+ - name: Remove generated alertmanager template
+ file: path='/tmp/alertmanager.yaml' state=absent
+
+ # Once any of these are present, update alertmanager secret
+ when: smtp_auth_password != '' or
+ dms_webhook_url != '' or
+ pd_service_key != ''
\ No newline at end of file
diff --git a/roles/middleware_monitoring_config/tasks/kube_state_metrics_alerts.yml b/roles/middleware_monitoring_config/tasks/create_alerts.yml
similarity index 100%
rename from roles/middleware_monitoring_config/tasks/kube_state_metrics_alerts.yml
rename to roles/middleware_monitoring_config/tasks/create_alerts.yml
diff --git a/roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml b/roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml
index efc578ed..423d4e64 100644
--- a/roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml
+++ b/roles/middleware_monitoring_config/tasks/get_blackbox_routes.yml
@@ -65,10 +65,3 @@
shell: oc get route unifiedpush-unifiedpush-proxy -o template --template \{\{.spec.host\}\} -n {{ eval_ups_namespace }}
register: ups_ups_route
when: ups | default(true) | bool
-
-- name: Get Mobile Security Service routes for blackbox targets
- block:
- - name: get mss route
- shell: oc get route route -o template --template \{\{.spec.host\}\} -n {{ eval_mobile_security_service_namespace }}
- register: mss_mss_route
- when: mobile_security_service | default(true) | bool
diff --git a/roles/middleware_monitoring_config/tasks/main.yml b/roles/middleware_monitoring_config/tasks/main.yml
index e8101ff8..989df184 100644
--- a/roles/middleware_monitoring_config/tasks/main.yml
+++ b/roles/middleware_monitoring_config/tasks/main.yml
@@ -1,7 +1,9 @@
---
- include_tasks: ./create_grafanadashboards.yml
-- include_tasks: ./kube_state_metrics_alerts.yml
+- include_tasks: ./create_alerts.yml
- include_tasks: ./prometheus_additional_scrape_config.yml
- include_tasks: ./create_blackboxtargets.yml
- include_tasks: ./patch_blackboxtargets_config.yml
when: not eval_self_signed_certs|bool
+- include_tasks: ./create_alertmanager.yml
+
diff --git a/roles/middleware_monitoring_config/tasks/upgrade.yml b/roles/middleware_monitoring_config/tasks/upgrade.yml
index 972e0446..f4d153f8 100644
--- a/roles/middleware_monitoring_config/tasks/upgrade.yml
+++ b/roles/middleware_monitoring_config/tasks/upgrade.yml
@@ -1,6 +1,6 @@
---
# Apply latest CR definitions
-- include_tasks: ./kube_state_metrics_alerts.yml
+- include_tasks: ./create_alerts.yml
- include_tasks: ./create_grafanadashboards.yml
- include_tasks: ./create_blackboxtargets.yml
- include_tasks: ./prometheus_additional_scrape_config.yml
diff --git a/roles/middleware_monitoring_config/templates/alertmanager.yml.j2 b/roles/middleware_monitoring_config/templates/alertmanager.yml.j2
new file mode 100644
index 00000000..6870530f
--- /dev/null
+++ b/roles/middleware_monitoring_config/templates/alertmanager.yml.j2
@@ -0,0 +1,49 @@
+global:
+ resolve_timeout: 5m
+{% if smtp_auth_password %}
+ smtp_smarthost: {{ smtp_smarthost }}
+ smtp_from: noreply@{{ alertmanager_route }}
+ smtp_auth_username: {{ smtp_auth_username }}
+ smtp_auth_password: {{ smtp_auth_password }}
+{% endif %}
+route:
+ group_wait: 30s
+ group_interval: 5m
+ repeat_interval: 12h
+ receiver: default
+ routes:
+ - match:
+ severity: critical
+ receiver: critical
+ - match:
+ alertname: DeadMansSwitch
+ repeat_interval: 5m
+{% if dms_webhook_url %}
+ receiver: deadmansswitch
+{% endif %}
+receivers:
+- name: default
+ email_configs:
+ - send_resolved: true
+ to: {{ alertmanager_to_email }}
+- name: critical
+{% if pd_service_key %}
+ pagerduty_configs:
+ - service_key: {{ pd_service_key }}
+{% endif %}
+ email_configs:
+ - send_resolved: true
+ to: {{ alertmanager_to_email }}
+{% if dms_webhook_url %}
+- name: deadmansswitch
+ webhook_configs:
+ - url: "{{ dms_webhook_url }}"
+{% endif %}
+inhibit_rules:
+- source_match:
+ alertname: 'JobRunningTimeExceeded'
+ severity: 'critical'
+ target_match:
+ alertname: 'JobRunningTimeExceeded'
+ severity: 'warning'
+ equal: ['alertname', 'job', 'label_cronjob_name']
\ No newline at end of file
diff --git a/roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2 b/roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2
index eb18da6c..0e3805a0 100644
--- a/roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2
+++ b/roles/middleware_monitoring_config/templates/blackboxtargets.yml.j2
@@ -64,9 +64,3 @@ spec:
url: https://{{ ups_ups_route.stdout }}/rest/auth/config/
module: http_2xx
{% endif %}
-{% if mobile_security_service | bool %}
- # Mobile Security Service targets
- - service: mss-ui
- url: https://{{ mss_mss_route.stdout }}/api/healthz
- module: http_2xx
-{% endif %}
diff --git a/roles/middleware_monitoring_config/templates/kube_state_metrics_3scale_alerts.yml.j2 b/roles/middleware_monitoring_config/templates/kube_state_metrics_3scale_alerts.yml.j2
new file mode 100644
index 00000000..ad0b66ef
--- /dev/null
+++ b/roles/middleware_monitoring_config/templates/kube_state_metrics_3scale_alerts.yml.j2
@@ -0,0 +1,156 @@
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ monitoring-key: "{{monitoring_label_value}}"
+ name: ksm-3scale-alerts
+spec:
+ groups:
+ - name: 3scale.rules
+ rules:
+ - alert: ThreeScaleApicastStagingPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale apicast-staging has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"apicast-staging.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleApicastProductionPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale apicast-production has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"apicast-production.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleBackendWorkerPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale backend-worker has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"backend-worker.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleBackendListenerPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale backend-listener has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"backend-listener.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleBackendRedisPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale backend-redis has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"backend-redis.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleSystemRedisPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale system-redis has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"system-redis.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleSystemMySQLPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale system-mysql has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"system-mysql.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleSystemAppPod
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale system-app has no pods in a ready state.
+ expr: |
+ absent(kube_pod_status_ready{namespace="{{ threescale_namespace }}", condition="true", pod=~"system-app-.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleAdminUIBBT
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale Admin UI Blackbox Target: If this console is unavailable,
+ the client is unable to configure or administer their API setup.
+ expr: |
+ absent(probe_success{job="blackbox", service="3scale-admin-ui"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleDeveloperUIBBT
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale Developer UI Blackbox Target: If this console is
+ unavailable, the clients developers are unable signup or perform
+ API management.
+ expr: >
+ absent(probe_success{job="blackbox",
+ service="3scale-developer-console-ui"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScaleSystemAdminUIBBT
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ 3Scale System Admin UI Blackbox Target: If this console is
+ unavailable, the client is unable to perform Account Management,
+ Analytics or Billing.
+ expr: >
+ absent(probe_success{job="blackbox",
+ service="3scale-system-admin-ui"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: ThreeScalePodCount
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected at least 15 pods.
+ expr: |
+ absent(sum(kube_pod_status_ready{condition="true",namespace="{{ threescale_namespace }}"}) >= 15)
+ for: 5m
+ labels:
+ severity: warning
+ - alert: ThreeScalePodHighMemory
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available memory for longer than 15 minutes.
+ scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/3scale-scaling.md
+ expr: |
+ sum by(container) (label_replace(container_memory_usage_bytes{container_name!="",namespace="{{ threescale_namespace }}"}, "container", "$1", "container_name", "(.*)")) / sum by(container) (kube_pod_container_resource_limits_memory_bytes{namespace="{{ threescale_namespace }}"}) * 100 > 90
+ for: 15m
+ labels:
+ severity: warning
+ - alert: ThreeScalePodHighCPU
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available CPU for longer than 15 minutes.
+ scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/3scale-scaling.md
+ expr: |
+ sum(label_replace(namespace_pod_name_container_name:container_cpu_usage_seconds_total:sum_rate{namespace="{{ threescale_namespace }}"}, 'container', '$1', 'container_name', '(.*)')) by (container) / sum(kube_pod_container_resource_limits_cpu_cores{namespace="{{ threescale_namespace }}"}) by (container) * 100 > 90
+ for: 15m
+ labels:
+ severity: warning
diff --git a/roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2 b/roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2
index 1e6e4e8f..1a256739 100644
--- a/roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2
+++ b/roles/middleware_monitoring_config/templates/kube_state_metrics_alerts.yml.j2
@@ -10,6 +10,7 @@ spec:
rules:
- alert: KubePodCrashLooping
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.pod {{ '}}' }} ({{ '{{' }} $labels.container {{ '}}' }}) is restarting {{ '{{' }} printf "%.2f" $value {{ '}}' }} times every 5 minutes; for the last 15 minutes.
expr: |
rate(kube_pod_container_status_restarts_total{job="kube-state-metrics"}[15m]) * on (namespace, namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"} * 60 * 5 > 0
@@ -18,6 +19,7 @@ spec:
severity: critical
- alert: ESPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Expected at least Elastic Search 1 pods in namespace {{ '{{' }} $labels.namespace {{ '}}' }}.
expr: |
(1 - absent(kube_pod_status_ready{condition="true",namespace="openshift-logging", pod=~"logging-es-data-master-.*"}))
@@ -26,6 +28,7 @@ spec:
severity: warning
- alert: ESNotReady
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Not all Elastic Search replication controllers are in a ready state.
expr: |
count(kube_replicationcontroller_status_ready_replicas{namespace="openshift-logging", replicationcontroller=~"logging-es-data-master-.*"}) != sum(kube_replicationcontroller_status_ready_replicas{namespace="openshift-logging", replicationcontroller=~"logging-es-data-master-.*"})
@@ -34,6 +37,7 @@ spec:
severity: warning
- alert: KubePodNotReady
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.pod {{ '}}' }} has been in a non-ready state for longer than 15 minutes.
expr: |
sum by(pod, namespace) (kube_pod_status_phase{phase=~"Pending|Unknown"} * on (namespace, namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) > 0
@@ -42,6 +46,7 @@ spec:
severity: critical
- alert: KubePodImagePullBackOff
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.pod {{ '}}' }} has been unable to pull it's image for longer than 5 minutes.
expr: |
(kube_pod_container_status_waiting_reason{reason="ImagePullBackOff"} * on (namespace, namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) > 0
@@ -50,6 +55,7 @@ spec:
severity: critical
- alert: KubePodBadConfig
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.pod {{ '}}' }} has been unable to start due to a bad configuration for longer than 5 minutes.
expr: |
(kube_pod_container_status_waiting_reason{reason="CreateContainerConfigError"} * on (namespace, namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) > 0
@@ -58,22 +64,16 @@ spec:
severity: critical
- alert: KubePodStuckCreating
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod {{ '{{' }} $labels.namespace {{ '}}' }}/{{ '{{' }} $labels.pod {{ '}}' }} has been trying to start for longer than 15 minutes - this could indicate a configuration error.
expr: |
(kube_pod_container_status_waiting_reason{reason="ContainerCreating"} * on (namespace, namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) > 0
for: 15m
labels:
severity: critical
- - alert: ThreeScalePodCount
- annotations:
- message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 15 pods.
- expr: |
- (1-absent(kube_pod_status_ready{condition="true", namespace="{{ threescale_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ threescale_namespace }}"}) != 15
- for: 5m
- labels:
- severity: critical
- alert: AMQOnlinePodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected at least 6 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{eval_enmasse_namespace}}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{eval_enmasse_namespace}}"}) < 6
@@ -82,6 +82,7 @@ spec:
severity: critical
- alert: FuseOnlinePodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected at least 8 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{eval_managed_fuse_namespace}}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{eval_managed_fuse_namespace}}"}) < 8
@@ -91,6 +92,7 @@ spec:
- alert: ApicuritoPodCount
annotations:
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 2 pods.
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_apicurito_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_apicurito_namespace }}"}) != 2
for: 5m
@@ -98,6 +100,7 @@ spec:
severity: critical
- alert: CodeReadyPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected at least 2 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_che_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_che_namespace }}"}) < 2
@@ -106,6 +109,7 @@ spec:
severity: critical
- alert: LauncherPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 6 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_launcher_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_launcher_namespace }}"}) != 6
@@ -114,6 +118,7 @@ spec:
severity: critical
- alert: ManagedServiceBrokerPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 1 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_msbroker_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_msbroker_namespace }}"}) != 1
@@ -122,6 +127,7 @@ spec:
severity: critical
- alert: MiddlewareMonitoringPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 6 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_middleware_monitoring_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_middleware_monitoring_namespace }}"}) != 6
@@ -130,6 +136,7 @@ spec:
severity: critical
- alert: SSOPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 3 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_rhsso_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_rhsso_namespace }}"}) != 3
@@ -138,32 +145,16 @@ spec:
severity: critical
- alert: SolutionExplorerPodCount
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: Pod count for namespace {{ '{{' }} $labels.namespace {{ '}}' }} is {{ '{{' }} printf "%.0f" $value {{ '}}' }}. Expected exactly 2 pods.
expr: |
(1-absent(kube_pod_status_ready{condition="true", namespace="{{ eval_webapp_namespace }}"})) or sum(kube_pod_status_ready{condition="true", namespace="{{ eval_webapp_namespace }}"}) != 2
for: 5m
labels:
severity: critical
- - alert: ThreeScalePodHighMemory
- annotations:
- message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available memory for longer than 15 minutes.
- scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/3scale-scaling.md
- expr: |
- sum by(container) (label_replace(container_memory_usage_bytes{container_name!="",namespace="{{ threescale_namespace }}"}, "container", "$1", "container_name", "(.*)")) / sum by(container) (kube_pod_container_resource_limits_memory_bytes{namespace="{{ threescale_namespace }}"}) * 100 > 90
- for: 15m
- labels:
- severity: warning
- - alert: ThreeScalePodHighCPU
- annotations:
- message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available CPU for longer than 15 minutes.
- scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/3scale-scaling.md
- expr: |
- sum(label_replace(namespace_pod_name_container_name:container_cpu_usage_seconds_total:sum_rate{namespace="{{ threescale_namespace }}"}, 'container', '$1', 'container_name', '(.*)')) by (container) / sum(kube_pod_container_resource_limits_cpu_cores{namespace="{{ threescale_namespace }}"}) by (container) * 100 > 90
- for: 15m
- labels:
- severity: warning
- alert: SSOPodHighMemory
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available memory for longer than 15 minutes.
scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/sso-scaling.md
expr: |
@@ -173,6 +164,7 @@ spec:
severity: warning
- alert: AMQOnlinePodHighMemory
annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
message: The {{ '{{' }} $labels.container {{ '}}' }} pod has been using {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of available memory for longer than 15 minutes.
scaling_plan: https://github.com/integr8ly/middleware-load-testing/blob/master/sops/amq-scaling.md
expr: |
@@ -183,6 +175,7 @@ spec:
- alert: ClusterSchedulableMemoryLow
annotations:
message: The cluster has {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of memory requested and unavailable for scheduling for longer than 15 minutes.
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts/Cluster_Schedulable_Resources_Low.asciidoc
expr: |
((sum(sum by(node) (sum by(pod, node) (kube_pod_container_resource_requests_memory_bytes * on(node) group_left() (sum by(node) (kube_node_labels{label_node_role_kubernetes_io_compute="true"} == 1))) * on(pod) group_left() (sum by(pod) (kube_pod_status_phase{phase="Running"}) == 1)))) / ((sum((kube_node_labels{label_node_role_kubernetes_io_compute="true"} == 1) * on(node) group_left() (sum by(node) (kube_node_status_allocatable_memory_bytes)))))) * 100 > 85
for: 15m
@@ -191,6 +184,7 @@ spec:
- alert: ClusterSchedulableCPULow
annotations:
message: The cluster has {{ '{{' }} printf "%.0f" $value {{ '}}' }}% of CPU cores requested and unavailable for scheduling for longer than 15 minutes.
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts/Cluster_Schedulable_Resources_Low.asciidoc
expr: |
((sum(sum by(node) (sum by(pod, node) (kube_pod_container_resource_requests_cpu_cores * on(node) group_left() (sum by(node) (kube_node_labels{label_node_role_kubernetes_io_compute="true"} == 1))) * on(pod) group_left() (sum by(pod) (kube_pod_status_phase{phase="Running"}) == 1)))) / ((sum((kube_node_labels{label_node_role_kubernetes_io_compute="true"} == 1) * on(node) group_left() (sum by(node) (kube_node_status_allocatable_cpu_cores)))))) * 100 > 85
for: 15m
@@ -199,6 +193,7 @@ spec:
- alert: PVCStorageAvailable
annotations:
message: The {{ '{{' }} $labels.persistentvolumeclaim {{ '}}' }} PVC has has been {{ '{{' }} printf "%.0f" $value {{ '}}' }}% full for longer than 15 minutes.
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts/Cluster_Schedulable_Resources_Low.asciidoc
expr: |
((sum by(persistentvolumeclaim, namespace) (kubelet_volume_stats_used_bytes) * on ( namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) / (sum by(persistentvolumeclaim, namespace) (kube_persistentvolumeclaim_resource_requests_storage_bytes) * on ( namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"})) * 100 > 85
for: 15m
@@ -207,8 +202,33 @@ spec:
- alert: PVCStorageMetricsAvailable
annotations:
message: PVC storage metrics are not available
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts/Cluster_Schedulable_Resources_Low.asciidoc
expr: |
absent(kubelet_volume_stats_available_bytes) == 1
for: 15m
labels:
severity: warning
+ - alert: PVCStorageWillFillIn4Days
+ annotations:
+ message: The {{ '{{' }} $labels.persistentvolumeclaim {{ '}}' }} PVC will run of disk space in the next 4 days.
+ expr: |
+ (sum by(persistentvolumeclaim, namespace) (predict_linear(kubelet_volume_stats_available_bytes{job="kubelet"}[6h], 4 * 24 * 3600)) * on(namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) <= 0
+ for: 15m
+ labels:
+ severity: warning
+ - alert: PVCStorageWillFillIn4Hours
+ annotations:
+ message: The {{ '{{' }} $labels.persistentvolumeclaim {{ '}}' }} PVC will run of disk space in the next 4 hours.
+ expr: |
+ (sum by(persistentvolumeclaim, namespace) (predict_linear(kubelet_volume_stats_available_bytes{job="kubelet"}[1h], 4 * 3600)) * on(namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) <= 0
+ for: 15m
+ labels:
+ severity: critical
+ - alert: PersistentVolumeErrors
+ annotations:
+ message: The PVC {{ '{{' }} $labels.persistentvolumeclaim {{ '}}' }} is in status {{ '{{' }} $labels.phase {{ '}}' }} in namespace {{ '{{' }} $labels.namespace {{ '}}' }}
+ expr: |
+ (sum by(persistentvolumeclaim, namespace, phase) (kube_persistentvolumeclaim_status_phase{phase=~"Failed|Pending|Lost"}) * on ( namespace) group_left(label_monitoring_key) kube_namespace_labels{label_monitoring_key="middleware"}) > 0
+ for: 15m
+ labels:
+ severity: critical
diff --git a/roles/middleware_monitoring_config/templates/kube_state_metrics_fuse_online_alerts.yml.j2 b/roles/middleware_monitoring_config/templates/kube_state_metrics_fuse_online_alerts.yml.j2
new file mode 100644
index 00000000..b1204ef5
--- /dev/null
+++ b/roles/middleware_monitoring_config/templates/kube_state_metrics_fuse_online_alerts.yml.j2
@@ -0,0 +1,30 @@
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+ labels:
+ monitoring-key: "{{monitoring_label_value}}"
+ name: ksm-fuse-online-alerts
+spec:
+ groups:
+ - name: fuseOnline.rules
+ rules:
+ - alert: FuseOnlineSyndesisServerInstanceDown
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ Fuse Online Syndesis Server instance {{ "{{" }}$labels.pod{{ "}}" }} in namespace {{ "{{" }}$labels.namespace{{ "}}" }} is down.
+ expr: >
+ absent(kube_pod_status_ready{namespace="{{ eval_managed_fuse_namespace }}", condition="true", pod=~"syndesis-server-.*"})
+ for: 5m
+ labels:
+ severity: critical
+ - alert: FuseOnlineSyndesisUIInstanceDown
+ annotations:
+ sop_url: https://github.com/RHCloudServices/integreatly-help/blob/master/sops/alerts_and_troubleshooting.md
+ message: >-
+ Fuse Online Syndesis UI instance {{ "{{" }}$labels.pod{{ "}}" }} in namespace {{ "{{" }}$labels.namespace{{ "}}" }} is down.
+ expr: >
+ absent(kube_pod_status_ready{namespace="{{ eval_managed_fuse_namespace }}", condition="true", pod=~"syndesis-ui-.*"})
+ for: 5m
+ labels:
+ severity: critical
diff --git a/roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2 b/roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2
index aad3f6f4..a01e3e88 100644
--- a/roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2
+++ b/roles/middleware_monitoring_config/templates/resources-by-namespace.yml.j2
@@ -94,7 +94,7 @@ spec:
},
"yaxes": [
{
- "format": "percentunit",
+ "format": "short",
"label": null,
"logBase": 1,
"max": null,
diff --git a/roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2 b/roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2
index 0d14036d..e79d4780 100644
--- a/roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2
+++ b/roles/middleware_monitoring_config/templates/resources-by-pod.yml.j2
@@ -109,7 +109,7 @@ spec:
},
"yaxes": [
{
- "format": "percentunit",
+ "format": "short",
"label": null,
"logBase": 1,
"max": null,
diff --git a/roles/mobile_security_service/OWNERS b/roles/mobile_security_service/OWNERS
deleted file mode 100644
index f9172588..00000000
--- a/roles/mobile_security_service/OWNERS
+++ /dev/null
@@ -1,5 +0,0 @@
-# See the OWNERS docs: https://git.k8s.io/community/contributors/guide/owners.md
-
-approvers:
-- wei-lee
-- grdryn
diff --git a/roles/mobile_security_service/defaults/main.yml b/roles/mobile_security_service/defaults/main.yml
index a5297422..cc41b7b1 100644
--- a/roles/mobile_security_service/defaults/main.yml
+++ b/roles/mobile_security_service/defaults/main.yml
@@ -1,23 +1 @@
-mobile_security_service_name: "mobile-security-service"
mobile_security_service_namespace: "{{ eval_mobile_security_service_namespace | default('mobile-security-service') }}"
-mobile_security_service_display_name: "Mobile Security Service"
-
-mobile_security_service_operator_resource_items:
- - "{{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityservice_crd.yaml"
- - "{{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityservicedb_crd.yaml"
- - "{{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityserviceapp_crd.yaml"
- - "{{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityservicebackup_crd.yaml"
- - "{{ mobile_security_service_operator_resources }}/cluster_role.yaml"
- - "{{ mobile_security_service_operator_resources }}/service_account.yaml"
-
-mobile_security_service_monitoring_resource_items:
- - "{{ mobile_security_service_operator_resources }}/monitor/service_monitor.yaml"
- - "{{ mobile_security_service_operator_resources }}/monitor/mss_service_monitor.yaml"
-
-mobile_security_service_monitoring_resource_templates:
- - mss_grafana_dashboard
- - mss_operator_grafana_dashboard
- - mss_operator_prometheus_rule
- - mss_prometheus_rule
-
-mdc_namespace: "{{ eval_mdc_namespace | default('mobile-developer-console') }}"
diff --git a/roles/mobile_security_service/tasks/backup.yml b/roles/mobile_security_service/tasks/backup.yml
deleted file mode 100644
index 128680c4..00000000
--- a/roles/mobile_security_service/tasks/backup.yml
+++ /dev/null
@@ -1,18 +0,0 @@
----
-- name: Create ServiceAccount and role binding
- include_role:
- name: backup
- tasks_from: _setup_service_account.yml
- vars:
- binding_name: mss-backup-binding
- serviceaccount_namespace: '{{ mobile_security_service_namespace }}'
-
-- name: Generate Mobile Security Service Backup CR template
- template:
- src: "backup_cr.yml.j2"
- dest: /tmp/mobile_security_service_backup.yml
-
-- name: Apply Mobile Security Service Backup CR
- shell: "oc apply -f /tmp/mobile_security_service_backup.yml -n {{ mobile_security_service_namespace }}"
- register: mobile_security_service_backup_cmd
- failed_when: mobile_security_service_backup_cmd.stderr != '' and 'AlreadyExists' not in mobile_security_service_backup_cmd.stderr
diff --git a/roles/mobile_security_service/tasks/main.yml b/roles/mobile_security_service/tasks/main.yml
deleted file mode 100644
index f2a4f73c..00000000
--- a/roles/mobile_security_service/tasks/main.yml
+++ /dev/null
@@ -1,81 +0,0 @@
----
-- name: Create {{ mobile_security_service_namespace }} namespace
- include_role:
- name: namespace
- vars:
- name: "{{ mobile_security_service_namespace }}"
- display_name: "{{ mobile_security_service_display_name }}"
-
-- name: Add labels to namespace
- shell: oc patch ns {{ mobile_security_service_namespace }} --patch '{"metadata":{"labels":{"{{ monitoring_label_name }}":"{{ monitoring_label_value }}", "integreatly-middleware-service":"true"}}}'
- register: namespace_patch
- failed_when: namespace_patch.stderr != '' and 'not patched' not in namespace_patch.stderr
- changed_when: namespace_patch.rc == 0
-
-- name: Create Mobile Security Service Operator Resources
- shell: "oc create -f {{ item }} -n {{ mobile_security_service_namespace }}"
- with_items: "{{ mobile_security_service_operator_resource_items }}"
- register: mobile_security_service_operator_resource_cmd
- failed_when: mobile_security_service_operator_resource_cmd.stderr != '' and 'AlreadyExists' not in mobile_security_service_operator_resource_cmd.stderr
-
-- name: Copy cluster role binding template
- template:
- src: cluster_role_binding.yml.j2
- dest: /tmp/mss_cluster_role_binding.yml
-
-- name: Create cluster role binding
- shell: "oc create -f /tmp/mss_cluster_role_binding.yml -n {{ mobile_security_service_namespace }}"
- register: mss_role_binding_cmd
- failed_when: mss_role_binding_cmd.stderr != '' and 'AlreadyExists' not in mss_role_binding_cmd.stderr
-
-- name: Delete cluster role binding template
- file:
- path: /tmp/mss_cluster_role_binding.yml
- state: absent
-
-- name: Generate Mobile Security Service operator template
- template:
- src: "operator.yml.j2"
- dest: /tmp/mobile-security-service-operator.yml
-
-- name: Create Mobile Security Service Operator
- shell: "oc create -f /tmp/mobile-security-service-operator.yml -n {{ mobile_security_service_namespace }}"
- register: mobile_security_service_operator_cmd
- failed_when: mobile_security_service_operator_cmd.stderr != '' and 'AlreadyExists' not in mobile_security_service_operator_cmd.stderr
-
-- name: "Wait for Operator pod to be ready"
- shell: "oc get pods --namespace={{ mobile_security_service_namespace }} --selector=name=mobile-security-service-operator -o jsonpath='{.items[*].status.containerStatuses[?(@.ready==true)].ready}' | wc -w"
- register: mobile_security_service_operator_result
- until: mobile_security_service_operator_result.stdout.find("1") != -1
- retries: 50
- delay: 10
-
-- name: "Delete operator Template File"
- file: path=/tmp/mobile-security-service-operator.yml state=absent
-
-- name: Create Mobile Security Service DB custom resource
- shell: oc create -f {{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityservicedb_cr.yaml -n {{ mobile_security_service_namespace }}
- register: create_mobile_security_service_db_custom_resource_cmd
- failed_when: create_mobile_security_service_db_custom_resource_cmd.stderr != '' and 'AlreadyExists' not in create_mobile_security_service_db_custom_resource_cmd.stderr
- changed_when: create_mobile_security_service_db_custom_resource_cmd.rc == 0
-
-- name: "Wait for Mobile Security Service DB pods to be ready"
- shell: "oc get pods --namespace={{ mobile_security_service_namespace }} --selector=name=mobilesecurityservicedb -o jsonpath='{.items[*].status.containerStatuses[?(@.ready==true)].ready}' | wc -w"
- register: mobile_security_service_db_result
- until: mobile_security_service_db_result.stdout.find("1") != -1
- retries: 50
- delay: 10
-
-- name: Create Mobile Security Service custom resource
- shell: oc create -f {{ mobile_security_service_operator_resources }}/crds/mobile-security-service_v1alpha1_mobilesecurityservice_cr.yaml -n {{ mobile_security_service_namespace }}
- register: create_mobile_security_service_custom_resource_cmd
- failed_when: create_mobile_security_service_custom_resource_cmd.stderr != '' and 'AlreadyExists' not in create_mobile_security_service_custom_resource_cmd.stderr
- changed_when: create_mobile_security_service_custom_resource_cmd.rc == 0
-
-- name: "Wait for Mobile Security Service pods to be ready"
- shell: "oc get pods --namespace={{ mobile_security_service_namespace }} --selector=name=mobilesecurityservice -o jsonpath='{.items[*].status.containerStatuses[?(@.ready==true)].ready}' | wc -w"
- register: mobile_security_service_result
- until: mobile_security_service_result.stdout.find("2") != -1
- retries: 50
- delay: 10
-
diff --git a/roles/mobile_security_service/tasks/monitoring.yml b/roles/mobile_security_service/tasks/monitoring.yml
deleted file mode 100644
index 661edd2f..00000000
--- a/roles/mobile_security_service/tasks/monitoring.yml
+++ /dev/null
@@ -1,30 +0,0 @@
----
-- name: Check {{ mobile_security_service_namespace }} namespace exists
- shell: oc get namespace {{ mobile_security_service_namespace }}
- register: mss_namespace_check
- failed_when: mss_namespace_check.stderr != '' and 'not found' not in mss_namespace_check.stderr
-
-
-- name: Create Mobile Security Service Operator Resources
- shell: "oc apply -f {{ item }} -n {{ mobile_security_service_namespace }}"
- with_items: "{{ mobile_security_service_monitoring_resource_items }}"
- register: mobile_security_service_monitoring_resource_cmd
- failed_when: mobile_security_service_monitoring_resource_cmd.stderr != '' and 'AlreadyExists' not in mobile_security_service_monitoring_resource_cmd.stderr
-
-- name: Copy Mobile Security Service Monitoring Resource Templates
- template:
- src: "{{ item }}.yml.j2"
- dest: "/tmp/{{ item }}.yml"
- with_items: "{{ mobile_security_service_monitoring_resource_templates }}"
-
-- name: Create Mobile Security Service Monitoring Resources
- shell: "oc apply -f /tmp/{{ item }}.yml -n {{ mobile_security_service_namespace }}"
- with_items: "{{ mobile_security_service_monitoring_resource_templates }}"
- register: mss_monitoring_resources_create_cmd
- failed_when: mss_monitoring_resources_create_cmd.stderr != '' and 'AlreadyExists' not in mss_monitoring_resources_create_cmd.stderr
-
-- name: Delete Mobile Security Service Monitoring Resource Templates
- file:
- path: /tmp/{{ item }}.yml
- state: absent
- with_items: "{{ mobile_security_service_monitoring_resource_templates }}"
\ No newline at end of file
diff --git a/roles/mobile_security_service/tasks/upgrade.yml b/roles/mobile_security_service/tasks/upgrade.yml
deleted file mode 100644
index b08c4ca1..00000000
--- a/roles/mobile_security_service/tasks/upgrade.yml
+++ /dev/null
@@ -1,29 +0,0 @@
----
-- name: Check if there is an existing installation of the Mobile Security Service
- shell: oc get namespace {{ mobile_security_service_namespace }}
- register: namespace_exists
- failed_when: namespace_exists.stderr != '' and 'NotFound' not in namespace_exists.stderr
-
-
-- name: Install Mobile Security Service
- include_role:
- name: mobile_security_service
- tags: ['mobile_security_service']
- when: namespace_exists.rc != 0 and mobile_security_service
-
-- name: Upgrade Mobile Security Service
- block:
- - name: Generate Mobile Security Service operator template
- template:
- src: "operator.yml.j2"
- dest: /tmp/mobile-security-service-operator.yml
- - name: Patch operator deployment
- shell: "oc apply -f /tmp/mobile-security-service-operator.yml -n {{ mobile_security_service_namespace }}"
- - name: Remove operator template
- file:
- path: "/tmp/mobile-security-service-operator.yml"
- state: absent
- - name: Patch the service image
- shell: "oc patch deployment mobile-security-service -n {{ mobile_security_service_namespace }} --type json -p '[{\"op\": \"replace\", \"path\": \"/spec/template/spec/containers/1/image\", \"value\": \"quay.io/aerogear/mobile-security-service:{{ mss_version }}\"}]'
-"
- when: namespace_exists.rc == 0 and mobile_security_service
\ No newline at end of file
diff --git a/roles/mobile_security_service/templates/backup_cr.yml.j2 b/roles/mobile_security_service/templates/backup_cr.yml.j2
deleted file mode 100644
index 042fff20..00000000
--- a/roles/mobile_security_service/templates/backup_cr.yml.j2
+++ /dev/null
@@ -1,10 +0,0 @@
-apiVersion: mobile-security-service.aerogear.org/v1alpha1
-kind: MobileSecurityServiceBackup
-metadata:
- name: mobile-security-service-backup
-spec:
- image: {{ backup_image }}
- awsCredentialsSecretName: {{ aws_s3_backup_secret_name }}
- productName: mss
- awsCredentialsSecretNamespace: {{ backup_namespace }}
- schedule: "{{ backup_schedule }}"
diff --git a/roles/mobile_security_service/templates/cluster_role_binding.yml.j2 b/roles/mobile_security_service/templates/cluster_role_binding.yml.j2
deleted file mode 100644
index 2b35120f..00000000
--- a/roles/mobile_security_service/templates/cluster_role_binding.yml.j2
+++ /dev/null
@@ -1,12 +0,0 @@
-kind: ClusterRoleBinding
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
- name: mobile-security-service-operator
-subjects:
-- kind: ServiceAccount
- name: mobile-security-service-operator
- namespace: "{{ mobile_security_service_namespace }}"
-roleRef:
- kind: ClusterRole
- name: mobile-security-service-operator
- apiGroup: rbac.authorization.k8s.io
\ No newline at end of file
diff --git a/roles/mobile_security_service/templates/mss_grafana_dashboard.yml.j2 b/roles/mobile_security_service/templates/mss_grafana_dashboard.yml.j2
deleted file mode 100644
index c9c3d001..00000000
--- a/roles/mobile_security_service/templates/mss_grafana_dashboard.yml.j2
+++ /dev/null
@@ -1,1099 +0,0 @@
-apiVersion: integreatly.org/v1alpha1
-kind: GrafanaDashboard
-metadata:
- name: mobile-security-service-application
- labels:
- monitoring-key: middleware
- prometheus: application-monitoring
-spec:
- selector:
- matchLabels:
- name: mobilesecurityservice
- json: |
- {
- "__requires": [
- {
- "type": "grafana",
- "id": "grafana",
- "name": "Grafana",
- "version": "4.3.2"
- },
- {
- "type": "panel",
- "id": "graph",
- "name": "Graph",
- "version": ""
- },
- {
- "type": "datasource",
- "id": "prometheus",
- "name": "Prometheus",
- "version": "1.0.0"
- },
- {
- "type": "panel",
- "id": "singlestat",
- "name": "Singlestat",
- "version": ""
- }
- ],
- "annotations": {
- "list": [
- {
- "builtIn": 1,
- "datasource": "-- Grafana --",
- "enable": true,
- "hide": true,
- "iconColor": "rgba(0, 211, 255, 1)",
- "name": "Annotations & Alerts",
- "type": "dashboard"
- }
- ]
- },
- "description": "Application metrics",
- "editable": true,
- "gnetId": null,
- "graphTooltip": 0,
- "links": [],
- "panels": [
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 0
- },
- "id": 9,
- "panels": [],
- "repeat": null,
- "title": "Uptime",
- "type": "row"
- },
- {
- "aliasColors": {},
- "bars": true,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 3,
- "y": 1
- },
- "id": 1,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [
- {
- "type": "dashboard"
- }
- ],
- "nullPointMode": "null",
- "percentage": true,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "kube_endpoint_address_available{endpoint='mobile-security-service-application'}",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 2,
- "legendFormat": "Admin Console - Uptime",
- "metric": "",
- "refId": "A",
- "step": 2
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Admin Console - Uptime",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "none",
- "label": null,
- "logBase": null,
- "max": 1.5,
- "min": 0,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": null,
- "max": 2,
- "min": 0,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "aliasColors": {},
- "bars": true,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 3,
- "y": 9
- },
- "id": 12,
- "legend": {
- "alignAsTable": false,
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": true,
- "targets": [
- {
- "expr": "kube_endpoint_address_available{namespace='{{ mobile_security_service_namespace }}',endpoint='mobile-security-service-db'}",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "legendFormat": "Postgres Database - Uptime",
- "refId": "A"
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Mobile Security Service Application Database - Uptime",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "aliasColors": {},
- "bars": true,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 3,
- "y": 17
- },
- "id": 14,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": true,
- "targets": [
- {
- "expr": "kube_endpoint_address_available{namespace='{{ mobile_security_service_namespace }}',endpoint='mobile-security-service-application'}",
- "format": "time_series",
- "intervalFactor": 1,
- "legendFormat": "Mobile Security Service Application - Uptime",
- "refId": "A"
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Mobile Security Service Application - Uptime",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 25
- },
- "id": 10,
- "panels": [],
- "repeat": null,
- "title": "Resources",
- "type": "row"
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 0,
- "y": 26
- },
- "id": 4,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "kube_pod_container_resource_limits_memory_bytes{namespace='{{ mobile_security_service_namespace }}', container='application'}",
- "format": "time_series",
- "intervalFactor": 1,
- "legendFormat": "Virtual Memory",
- "refId": "A"
- },
- {
- "expr": "kube_pod_container_resource_requests_memory_bytes{namespace='{{ mobile_security_service_namespace }}', container='application'}",
- "format": "time_series",
- "intervalFactor": 2,
- "legendFormat": "Memory Usage",
- "refId": "B",
- "step": 2
- },
- {
- "expr": "kube_pod_container_resource_limits_memory_bytes{namespace='{{ mobile_security_service_namespace }}', container='application'}",
- "format": "time_series",
- "intervalFactor": 2,
- "legendFormat": "Max Memory Allocation",
- "refId": "C",
- "step": 2
- },
- {
- "expr": "((kube_pod_container_resource_limits_memory_bytes{namespace='{{ mobile_security_service_namespace }}', container='application'})/100)*90",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 2,
- "legendFormat": "90% of Max Memory Allocation",
- "refId": "D",
- "step": 2
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Memory Usage",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "bytes",
- "label": null,
- "logBase": 2,
- "max": null,
- "min": 0,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 42
- },
- "id": 20,
- "panels": [],
- "title": "API HTTP Monitoring",
- "type": "row"
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 9,
- "w": 24,
- "x": 0,
- "y": 43
- },
- "id": 28,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "(api_requests_duration_seconds{job=\"mobile-security-service-application\"})",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "legendFormat": "{{ '{{' }}method{{ '}}' }} {{ '{{' }}path{{ '}}' }}",
- "refId": "A"
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "API Requests Duration",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "s",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": "0",
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 8,
- "x": 0,
- "y": 52
- },
- "id": 30,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "rate(api_requests_failure_total[30m])",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "legendFormat": "Errors {{ '{{' }}code{{ '}}' }} - {{ '{{' }}method{{ '}}' }} {{ '{{' }}path{{ '}}' }}",
- "refId": "A"
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Rate of HTTP Errors Per Second",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "decimals": null,
- "format": "short",
- "label": "Rate of HTTP Errors/Second",
- "logBase": 1,
- "max": null,
- "min": "0",
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 10,
- "x": 8,
- "y": 52
- },
- "id": 32,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "rate(api_requests_total[1h])",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "legendFormat": "{{ '{{' }}method{{ '}}' }} - {{ '{{' }}code{{ '}}' }} - {{ '{{' }}path{{ '}}' }}",
- "refId": "A"
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Rate of HTTP Responses",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "short",
- "label": "Rate of HTTP Responses/Second",
- "logBase": 1,
- "max": null,
- "min": "0",
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "cacheTimeout": null,
- "colorBackground": false,
- "colorValue": false,
- "colors": [
- "#299c46",
- "rgba(237, 129, 40, 0.89)",
- "#d44a3a"
- ],
- "datasource": "Prometheus",
- "format": "s",
- "gauge": {
- "maxValue": 40,
- "minValue": 0,
- "show": true,
- "thresholdLabels": true,
- "thresholdMarkers": true
- },
- "gridPos": {
- "h": 4,
- "w": 3,
- "x": 18,
- "y": 52
- },
- "id": 26,
- "interval": null,
- "links": [],
- "mappingType": 1,
- "mappingTypes": [
- {
- "name": "value to text",
- "value": 1
- },
- {
- "name": "range to text",
- "value": 2
- }
- ],
- "maxDataPoints": 100,
- "nullPointMode": "connected",
- "nullText": null,
- "postfix": "",
- "postfixFontSize": "50%",
- "prefix": "",
- "prefixFontSize": "50%",
- "rangeMaps": [
- {
- "from": "null",
- "text": "N/A",
- "to": "null"
- }
- ],
- "sparkline": {
- "fillColor": "rgba(31, 118, 189, 0.18)",
- "full": false,
- "lineColor": "rgb(31, 120, 193)",
- "show": true
- },
- "tableColumn": "",
- "targets": [
- {
- "expr": "max(api_requests_duration_seconds{job='mobile-security-service-application'})",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "refId": "A"
- }
- ],
- "thresholds": "25,30",
- "title": "Max API Requests Duration",
- "type": "singlestat",
- "valueFontSize": "80%",
- "valueMaps": [
- {
- "op": "=",
- "text": "N/A",
- "value": "null"
- }
- ],
- "valueName": "max"
- },
- {
- "cacheTimeout": null,
- "colorBackground": false,
- "colorValue": false,
- "colors": [
- "#299c46",
- "rgba(237, 129, 40, 0.89)",
- "#d44a3a"
- ],
- "datasource": "Prometheus",
- "format": "s",
- "gauge": {
- "maxValue": 40,
- "minValue": 0,
- "show": true,
- "thresholdLabels": true,
- "thresholdMarkers": true
- },
- "gridPos": {
- "h": 4,
- "w": 3,
- "x": 21,
- "y": 52
- },
- "id": 34,
- "interval": null,
- "links": [],
- "mappingType": 1,
- "mappingTypes": [
- {
- "name": "value to text",
- "value": 1
- },
- {
- "name": "range to text",
- "value": 2
- }
- ],
- "maxDataPoints": 100,
- "nullPointMode": "connected",
- "nullText": null,
- "postfix": "",
- "postfixFontSize": "50%",
- "prefix": "",
- "prefixFontSize": "50%",
- "rangeMaps": [
- {
- "from": "null",
- "text": "N/A",
- "to": "null"
- }
- ],
- "sparkline": {
- "fillColor": "rgba(31, 118, 189, 0.18)",
- "full": false,
- "lineColor": "rgb(31, 120, 193)",
- "show": true
- },
- "tableColumn": "",
- "targets": [
- {
- "expr": "min(api_requests_duration_seconds{job='mobile-security-service-application'})",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "refId": "A"
- }
- ],
- "thresholds": "25,30",
- "title": "Minium API Requests Duration Seconds",
- "type": "singlestat",
- "valueFontSize": "80%",
- "valueMaps": [
- {
- "op": "=",
- "text": "N/A",
- "value": "null"
- }
- ],
- "valueName": "min"
- },
- {
- "cacheTimeout": null,
- "colorBackground": false,
- "colorValue": false,
- "colors": [
- "#299c46",
- "rgba(237, 129, 40, 0.89)",
- "#d44a3a"
- ],
- "datasource": "Prometheus",
- "format": "s",
- "gauge": {
- "maxValue": 40,
- "minValue": 0,
- "show": true,
- "thresholdLabels": true,
- "thresholdMarkers": true
- },
- "gridPos": {
- "h": 4,
- "w": 3,
- "x": 18,
- "y": 56
- },
- "id": 24,
- "interval": null,
- "links": [],
- "mappingType": 1,
- "mappingTypes": [
- {
- "name": "value to text",
- "value": 1
- },
- {
- "name": "range to text",
- "value": 2
- }
- ],
- "maxDataPoints": 100,
- "nullPointMode": "connected",
- "nullText": null,
- "postfix": "",
- "postfixFontSize": "50%",
- "prefix": "",
- "prefixFontSize": "50%",
- "rangeMaps": [
- {
- "from": "null",
- "text": "N/A",
- "to": "null"
- }
- ],
- "sparkline": {
- "fillColor": "rgba(31, 118, 189, 0.18)",
- "full": false,
- "lineColor": "rgb(31, 120, 193)",
- "show": true
- },
- "tableColumn": "",
- "targets": [
- {
- "expr": "avg(api_requests_duration_seconds{job='mobile-security-service-application'})",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "refId": "A"
- }
- ],
- "thresholds": "25, 30",
- "title": "Average API Requests Duration",
- "type": "singlestat",
- "valueFontSize": "80%",
- "valueMaps": [
- {
- "op": "=",
- "text": "N/A",
- "value": "null"
- }
- ],
- "valueName": "avg"
- },
- {
- "cacheTimeout": null,
- "colorBackground": false,
- "colorValue": false,
- "colors": [
- "#299c46",
- "rgba(237, 129, 40, 0.89)",
- "#d44a3a"
- ],
- "datasource": "Prometheus",
- "format": "none",
- "gauge": {
- "maxValue": 40,
- "minValue": 0,
- "show": true,
- "thresholdLabels": true,
- "thresholdMarkers": true
- },
- "gridPos": {
- "h": 4,
- "w": 3,
- "x": 21,
- "y": 56
- },
- "id": 22,
- "interval": null,
- "links": [],
- "mappingType": 1,
- "mappingTypes": [
- {
- "name": "value to text",
- "value": 1
- },
- {
- "name": "range to text",
- "value": 2
- }
- ],
- "maxDataPoints": 100,
- "nullPointMode": "connected",
- "nullText": null,
- "postfix": "",
- "postfixFontSize": "50%",
- "prefix": "",
- "prefixFontSize": "50%",
- "rangeMaps": [
- {
- "from": "null",
- "text": "N/A",
- "to": "null"
- }
- ],
- "sparkline": {
- "fillColor": "rgba(31, 118, 189, 0.18)",
- "full": false,
- "lineColor": "rgb(31, 120, 193)",
- "show": true
- },
- "tableColumn": "",
- "targets": [
- {
- "expr": "avg(api_requests_in_flight{job='mobile-security-service-application'})",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 1,
- "legendFormat": "",
- "refId": "A"
- }
- ],
- "thresholds": "15 ,20",
- "title": "Average API requests in flight",
- "type": "singlestat",
- "valueFontSize": "80%",
- "valueMaps": [
- {
- "op": "=",
- "text": "N/A",
- "value": "null"
- }
- ],
- "valueName": "avg"
- }
- ],
- "refresh": "10s",
- "schemaVersion": 16,
- "style": "dark",
- "tags": [],
- "templating": {
- "list": []
- },
- "time": {
- "from": "now/d",
- "to": "now"
- },
- "timepicker": {
- "refresh_intervals": [
- "5s",
- "10s",
- "30s",
- "1m",
- "5m",
- "15m",
- "30m",
- "1h",
- "2h",
- "1d"
- ],
- "time_options": [
- "5m",
- "15m",
- "1h",
- "6h",
- "12h",
- "24h",
- "2d",
- "7d",
- "30d"
- ]
- },
- "timezone": "browser",
- "title": "Mobile Security Service Application",
- "version": 1
- }
- name: mobilesecurityserviceapplication.json
diff --git a/roles/mobile_security_service/templates/mss_operator_grafana_dashboard.yml.j2 b/roles/mobile_security_service/templates/mss_operator_grafana_dashboard.yml.j2
deleted file mode 100644
index 70043a6f..00000000
--- a/roles/mobile_security_service/templates/mss_operator_grafana_dashboard.yml.j2
+++ /dev/null
@@ -1,418 +0,0 @@
-apiVersion: integreatly.org/v1alpha1
-kind: GrafanaDashboard
-metadata:
- name: mobile-security-service-operator
- labels:
- monitoring-key: middleware
- prometheus: application-monitoring
-spec:
- selector:
- matchLabels:
- name: mobile-security-service-operator
- json: |
- {
- "__requires": [
- {
- "type": "grafana",
- "id": "grafana",
- "name": "Grafana",
- "version": "4.3.2"
- },
- {
- "type": "panel",
- "id": "graph",
- "name": "Graph",
- "version": ""
- },
- {
- "type": "datasource",
- "id": "prometheus",
- "name": "Prometheus",
- "version": "1.0.0"
- },
- {
- "type": "panel",
- "id": "singlestat",
- "name": "Singlestat",
- "version": ""
- }
- ],
- "annotations": {
- "list": [
- {
- "builtIn": 1,
- "datasource": "-- Grafana --",
- "enable": true,
- "hide": true,
- "iconColor": "rgba(0, 211, 255, 1)",
- "name": "Annotations & Alerts",
- "type": "dashboard"
- }
- ]
- },
- "description": "Application metrics",
- "editable": true,
- "gnetId": null,
- "graphTooltip": 0,
- "links": [],
- "panels": [
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 0
- },
- "id": 9,
- "panels": [],
- "repeat": null,
- "title": "Uptime",
- "type": "row"
- },
- {
- "aliasColors": {},
- "bars": true,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 3,
- "y": 1
- },
- "id": 1,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [
- {
- "type": "dashboard"
- }
- ],
- "nullPointMode": "null",
- "percentage": true,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "kube_endpoint_address_available{namespace='{{ mobile_security_service_namespace }}',endpoint='mobile-security-service-operator'}",
- "format": "time_series",
- "hide": false,
- "intervalFactor": 2,
- "legendFormat": "{{ '{{' }}service{{ '}}' }} - Uptime",
- "metric": "",
- "refId": "A",
- "step": 2
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Uptime",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "none",
- "label": null,
- "logBase": null,
- "max": 1.5,
- "min": 0,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": null,
- "max": 2,
- "min": 0,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 9
- },
- "id": 10,
- "panels": [],
- "repeat": null,
- "title": "Resources",
- "type": "row"
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 0,
- "y": 10
- },
- "id": 4,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "process_virtual_memory_bytes{namespace='{{ mobile_security_service_namespace }}',job='mobile-security-service-operator'}",
- "format": "time_series",
- "intervalFactor": 1,
- "legendFormat": "Virtual Memory",
- "refId": "A"
- },
- {
- "expr": "process_resident_memory_bytes{namespace='{{ mobile_security_service_namespace }}',job='mobile-security-service-operator'}",
- "format": "time_series",
- "intervalFactor": 2,
- "legendFormat": "Memory Usage",
- "refId": "B",
- "step": 2
- },
- {
- "expr": "kube_pod_container_resource_limits_memory_bytes{namespace='{{ mobile_security_service_namespace }}',container='mobile-security-service-operator'}",
- "format": "time_series",
- "intervalFactor": 2,
- "legendFormat": "Max Memory Allocation",
- "refId": "C",
- "step": 2
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "Memory Usage",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "bytes",
- "label": null,
- "logBase": 2,
- "max": null,
- "min": 0,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- },
- {
- "aliasColors": {},
- "bars": false,
- "dashLength": 10,
- "dashes": false,
- "datasource": "Prometheus",
- "fill": 1,
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 0,
- "y": 18
- },
- "id": 2,
- "legend": {
- "avg": false,
- "current": false,
- "max": false,
- "min": false,
- "show": true,
- "total": false,
- "values": false
- },
- "lines": true,
- "linewidth": 1,
- "links": [],
- "nullPointMode": "null",
- "percentage": false,
- "pointradius": 5,
- "points": false,
- "renderer": "flot",
- "seriesOverrides": [],
- "spaceLength": 10,
- "stack": false,
- "steppedLine": false,
- "targets": [
- {
- "expr": "sum(rate(process_cpu_seconds_total{namespace='{{ mobile_security_service_namespace }}',job='mobile-security-service-operator'}[1m]))*1000",
- "format": "time_series",
- "interval": "",
- "intervalFactor": 2,
- "legendFormat": "Mobile Security Service Operator- CPU Usage in Millicores",
- "refId": "A",
- "step": 2
- },
- {
- "expr": "(kube_pod_container_resource_limits_cpu_cores{namespace='{{ mobile_security_service_namespace }}',container='mobile-security-service-operator'})*1000",
- "format": "time_series",
- "interval": "",
- "intervalFactor": 2,
- "legendFormat": "Maximum Limit of Millicores",
- "refId": "B",
- "step": 2
- }
- ],
- "thresholds": [],
- "timeFrom": null,
- "timeRegions": [],
- "timeShift": null,
- "title": "CPU Usage",
- "tooltip": {
- "shared": true,
- "sort": 0,
- "value_type": "individual"
- },
- "transparent": false,
- "type": "graph",
- "xaxis": {
- "buckets": null,
- "mode": "time",
- "name": null,
- "show": true,
- "values": []
- },
- "yaxes": [
- {
- "format": "short",
- "label": "Millicores",
- "logBase": 10,
- "max": null,
- "min": null,
- "show": true
- },
- {
- "format": "short",
- "label": null,
- "logBase": 1,
- "max": null,
- "min": null,
- "show": true
- }
- ],
- "yaxis": {
- "align": false,
- "alignLevel": null
- }
- }
- ],
- "refresh": "10s",
- "schemaVersion": 16,
- "style": "dark",
- "tags": [],
- "templating": {
- "list": []
- },
- "time": {
- "from": "now/d",
- "to": "now"
- },
- "timepicker": {
- "refresh_intervals": [
- "5s",
- "10s",
- "30s",
- "1m",
- "5m",
- "15m",
- "30m",
- "1h",
- "2h",
- "1d"
- ],
- "time_options": [
- "5m",
- "15m",
- "1h",
- "6h",
- "12h",
- "24h",
- "2d",
- "7d",
- "30d"
- ]
- },
- "timezone": "browser",
- "title": "Mobile Security Service Operator",
- "version": 2
- }
- name: mobilesecurityservice.json
diff --git a/roles/mobile_security_service/templates/mss_operator_prometheus_rule.yml.j2 b/roles/mobile_security_service/templates/mss_operator_prometheus_rule.yml.j2
deleted file mode 100644
index 6571cb70..00000000
--- a/roles/mobile_security_service/templates/mss_operator_prometheus_rule.yml.j2
+++ /dev/null
@@ -1,26 +0,0 @@
-# Monitor Service (Metrics)
-apiVersion: monitoring.coreos.com/v1
-kind: PrometheusRule
-metadata:
- labels:
- monitoring-key: middleware
- prometheus: application-monitoring
- role: alert-rules
- name: application-monitoring
-spec:
- selector:
- matchLabels:
- name: mobile-security-service-operator
- groups:
- - name: general.rules
- rules:
- - alert: MobileSecurityServiceOperatorDown
- expr: absent(up{job="mobile-security-service-operator"} == 1)
- for: 5m
- labels:
- severity: critical
- annotations:
- description: "The mobile-security-service-operator has been down for more than 5 minutes. "
- summary: "The mobile-security-service-operator is down. For more information see on the MSS operator https://github.com/aerogear/mobile-security-service-operator"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-operator.adoc"
-
diff --git a/roles/mobile_security_service/templates/mss_prometheus_rule.yml.j2 b/roles/mobile_security_service/templates/mss_prometheus_rule.yml.j2
deleted file mode 100644
index ca4dba4d..00000000
--- a/roles/mobile_security_service/templates/mss_prometheus_rule.yml.j2
+++ /dev/null
@@ -1,88 +0,0 @@
-# Monitor Service (Metrics)
-apiVersion: monitoring.coreos.com/v1
-kind: PrometheusRule
-metadata:
- labels:
- monitoring-key: middleware
- prometheus: application-monitoring
- role: alert-rules
- name: mobile-security-service
-spec:
- selector:
- matchLabels:
- app: mobilesecurityservice
- groups:
- - name: general.rules
- rules:
- - alert: MobileSecurityServiceDown
- expr: absent(kube_pod_container_status_running{namespace="{{ mobile_security_service_namespace }}",container="application"}>=1)
- for: 5m
- labels:
- severity: critical
- annotations:
- description: "The mobile-security-service has been down for more than 5 minutes. "
- summary: "The mobile-security-service is down. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServiceConsoleDown
- expr: absent(kube_endpoint_address_available{endpoint="mobile-security-service-application"} >= 1)
- for: 5m
- labels:
- severity: critical
- annotations:
- description: "The mobile-security-service admin console has been down for more than 5 minutes. "
- summary: "The mobile-security-service admin console endpoint has been unavailable for more that 5 minutes. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServiceDatabaseDown
- expr: absent(kube_pod_container_status_running{namespace="{{ mobile_security_service_namespace }}",container="database"}==1)
- for: 5m
- labels:
- severity: critical
- annotations:
- description: "The mobile-security-service-db pod has been down for more than 5 minutes"
- summary: "The mobile-security-service-db is down. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServicePodCPUHigh
- expr: "(rate(process_cpu_seconds_total{job='mobile-security-service-application'}[1m])) > (((kube_pod_container_resource_limits_cpu_cores{namespace='{{ mobile_security_service_namespace }}',container='application'})/100)*90)"
- for: 5m
- labels:
- severity: warning
- annotations:
- description: "The mobile-security-service pod has been at 90% CPU usage for more than 5 minutes"
- summary: "The mobile-security-service is reporting high cpu usage for more that 5 minutes. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServicePodMemoryHigh
- expr: "(process_resident_memory_bytes{job='mobile-security-service-application'}) > (((kube_pod_container_resource_limits_memory_bytes{namespace='{{ mobile_security_service_namespace }}',container='application'})/100)*90)"
- for: 5m
- labels:
- severity: warning
- annotations:
- description: "The mobile-security-service pod has been at 90% memory usage for more than 5 minutes"
- summary: "The mobile-security-service is reporting high memory usage for more that 5 minutes. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServiceApiHighRequestDuration
- expr: "api_requests_duration_seconds{job='mobile-security-service-application', quantile='0.5'} > 30"
- for: 5m
- labels:
- severity: warning
- annotations:
- description: "The mobile-security-service api has had http requests latency longer that 30 seconds for more than 5 minutes"
- summary: "The mobile-security-service is reporting high request latency for more that 5 minutes. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServiceApiHighConcurrentRequests
- expr: "api_requests_in_flight{job='mobile-security-service-application'} > 50"
- for: 5m
- labels:
- severity: warning
- annotations:
- description: "The mobile-security-service api has had 50 concurrent requests for more than 5 minutes"
- summary: "The mobile-security-service is reporting high request concurrency for more that 5 minutes. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
- - alert: MobileSecurityServiceApiHighRequestFailure
- expr: "rate(api_requests_failure_total{job='mobile-security-service-application'}[1h])>10"
- for: 1h
- labels:
- severity: warning
- annotations:
- description: "The mobile-security-service api has reported more that 10 request failures in an hour"
- summary: "The mobile-security-service is reporting a high request failure over an hour. For more information see on the MSS at https://github.com/aerogear/mobile-security-service"
- sop_url: "https://github.com/aerogear/mobile-security-service-operator/blob/{{ mobile_security_service_operator_release_tag }}/SOP/SOP-mss.adoc"
\ No newline at end of file
diff --git a/roles/mobile_security_service/templates/operator.yml.j2 b/roles/mobile_security_service/templates/operator.yml.j2
deleted file mode 100644
index ff72078b..00000000
--- a/roles/mobile_security_service/templates/operator.yml.j2
+++ /dev/null
@@ -1,33 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
- name: mobile-security-service-operator
-spec:
- replicas: 1
- selector:
- matchLabels:
- name: mobile-security-service-operator
- template:
- metadata:
- labels:
- name: mobile-security-service-operator
- spec:
- serviceAccountName: mobile-security-service-operator
- containers:
- - name: mobile-security-service-operator
- # Replace this with the built image name
- image: {{ mobile_security_service_operator_image }}
- command:
- - mobile-security-service-operator
- imagePullPolicy: Always
- env:
- - name: WATCH_NAMESPACE
- value: ""
- - name: APP_NAMESPACES
- value: "{{ mdc_namespace }}"
- - name: POD_NAME
- valueFrom:
- fieldRef:
- fieldPath: metadata.name
- - name: OPERATOR_NAME
- value: "mobile-security-service-operator"
\ No newline at end of file
diff --git a/roles/msbroker/defaults/main.yml b/roles/msbroker/defaults/main.yml
index b7c5396e..73fab7fe 100644
--- a/roles/msbroker/defaults/main.yml
+++ b/roles/msbroker/defaults/main.yml
@@ -6,7 +6,7 @@ msbroker_clusterrolebinding: default-cluster-account-managed-service
msbroker_deployment_name: msb
msbroker_servicebroker_name: managed-service-broker
monitoring_key: brokered-fuse-online
-msbroker_fuse_operator_resources_url: "{{ fuse_online_operator_resources }}"
+msbroker_fuse_operator_resources_url: "{{ fuse_online_binary_resources_base }}"
msbroker_required_crds:
- "{{ fuse_online_crd_resources }}"
diff --git a/roles/msbroker/tasks/apply_msbroker_template.yml b/roles/msbroker/tasks/apply_msbroker_template.yml
index cad4f59d..154dd7a4 100644
--- a/roles/msbroker/tasks/apply_msbroker_template.yml
+++ b/roles/msbroker/tasks/apply_msbroker_template.yml
@@ -10,10 +10,10 @@
- name: Get Che dashboard url
shell: oc get route/codeready -o jsonpath='{.spec.host}' -n {{ eval_che_namespace }}
register: che_host_cmd
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- set_fact:
che_dashboard_url: "https://{{ che_host_cmd.stdout}}"
- when: (che | bool) and (launcher | bool)
+ when: che and launcher
- name: Include threescale route vars
include_vars: ../../3scale/defaults/main.yml
@@ -73,10 +73,6 @@
product_ns_pull_secret_name: "{{ fuse_pull_secret_name }}"
when: fuse_online
-- name: Create CRDs
- shell: "oc apply -f {{ item }}"
- with_items: "{{ msbroker_required_crds }}"
-
- set_fact:
fuse_online_enabled: "true"
when: fuse_online
@@ -89,15 +85,14 @@
shell: oc process -f {{ msbroker_template }} \
-p 'NAMESPACE={{ msbroker_namespace }}' \
-p 'ROUTE_SUFFIX={{ route_suffix }}' \
- -p 'LAUNCHER_DASHBOARD_URL={{ launcher_dashboard_url | default('#') }}' \
- -p 'CHE_DASHBOARD_URL={{ che_dashboard_url | default('#') }}' \
- -p 'THREESCALE_DASHBOARD_URL={{ threescale_dashboard_url | default('#') }}' \
- -p 'APICURIO_DASHBOARD_URL={{ apicurito_dashboard_url | default('#') }}' \
- -p 'SHARED_FUSE_DASHBOARD_URL={{ fuse_dashboard_url | default('#') }}' \
+ -p 'LAUNCHER_DASHBOARD_URL={{ launcher_dashboard_url }}' \
+ -p 'CHE_DASHBOARD_URL={{ che_dashboard_url }}' \
+ -p 'THREESCALE_DASHBOARD_URL={{ threescale_dashboard_url }}' \
+ -p 'APICURIO_DASHBOARD_URL={{ apicurito_dashboard_url }}' \
+ -p 'SHARED_FUSE_DASHBOARD_URL={{ fuse_dashboard_url }}' \
-p 'SSO_URL=https://{{ sso_route.stdout }}/auth/admin/{{ sso_realm }}/console' \
-p 'USER_SSO_URL=https://{{ user_sso_route.stdout }}' \
-p 'FUSE_OPERATOR_RESOURCES_URL={{ msbroker_fuse_operator_resources_url }}' \
- -p 'MDC_DASHBOARD_URL={{ msb_mdc_url | default('#') }}' \
-p 'IMAGE_ORG={{ msbroker_image_org }}' \
-p 'IMAGE_TAG={{ msbroker_release_tag }}' \
-p 'MONITORING_KEY={{ monitoring_key }}' | oc apply -n "{{ msbroker_namespace }}" -f -
diff --git a/roles/msbroker/tasks/upgrade_images.yml b/roles/msbroker/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/msbroker/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/namespace/defaults/main.yml b/roles/namespace/defaults/main.yml
index 9472758b..4324059c 100644
--- a/roles/namespace/defaults/main.yml
+++ b/roles/namespace/defaults/main.yml
@@ -1 +1,4 @@
-namespace_file: namespace.yml
\ No newline at end of file
+namespace_file: namespace.yml
+
+rhmi_label: rhmi
+rhmi_label_value: "true"
\ No newline at end of file
diff --git a/roles/namespace/tasks/upgrade.yml b/roles/namespace/tasks/upgrade.yml
new file mode 100644
index 00000000..4a892f05
--- /dev/null
+++ b/roles/namespace/tasks/upgrade.yml
@@ -0,0 +1,17 @@
+# Get Console Config Namespace
+- name: Expose console config vars
+ include_vars: "../../customise-web-console/defaults/main.yml"
+
+- name: Get integreatly middleware services namespaces
+ shell: "oc get namespace -l integreatly-middleware-service=true | tail -n +2 | awk '{print $1}'"
+ register: rhmi_namespaces
+
+# Ensures all namespaces created by the Integreatly installer has the label rhmi=true
+- name: "Add rhmi label to Integreatly namespaces"
+ shell: "oc label namespace {{ item }} {{ rhmi_label }}={{ rhmi_label_value }} --overwrite=true"
+ register: output
+ failed_when: output.stderr != '' and 'NotFound' not in output.stderr
+ changed_when: output.rc == 0
+ with_items:
+ - "{{ rhmi_namespaces.stdout_lines }}"
+ - "{{ customise_web_console.namespace}}"
\ No newline at end of file
diff --git a/roles/namespace/templates/namespace.yml.j2 b/roles/namespace/templates/namespace.yml.j2
index 22e28752..d9a4e764 100644
--- a/roles/namespace/templates/namespace.yml.j2
+++ b/roles/namespace/templates/namespace.yml.j2
@@ -3,6 +3,7 @@ apiVersion: v1
metadata:
name: {{ name }}
labels:
+ {{ rhmi_label }}: {{ rhmi_label_value}}
{% if monitor is defined %}
{{ monitoring_label_name }}: {{ monitoring_label_value }}
{% endif %}
diff --git a/roles/resource_limits/defaults/main.yml b/roles/resource_limits/defaults/main.yml
index e1956034..e95b2533 100644
--- a/roles/resource_limits/defaults/main.yml
+++ b/roles/resource_limits/defaults/main.yml
@@ -7,3 +7,10 @@ resource_limits_horizontal_scaling: false
# vertical scaling enables management of resource requests/limits
resource_limits_vertical_scaling: true
+# wait for all rollouts for a given namespace to complete before moving to next
+# if disabled, failed rollouts will not be detected
+# if enabled, if any rollout failed or timed out, script will halt
+resource_limits_wait_for_rollouts_per_ns: true
+
+# if enabled, script will halt if an override is specified for a resource that does not exist
+resource_limits_fail_on_missing_resource: false
\ No newline at end of file
diff --git a/roles/resource_limits/tasks/main.yml b/roles/resource_limits/tasks/main.yml
index 1c988085..1f18c191 100644
--- a/roles/resource_limits/tasks/main.yml
+++ b/roles/resource_limits/tasks/main.yml
@@ -1,19 +1,53 @@
---
+- set_fact:
+ valid_resource_patches: '{{ resources | selectattr("kind","defined") | selectattr("name","defined") | list }}'
+
+- name: Warn about invalid patches that will be skipped
+ pause:
+ prompt: |
+
+ ***************************************************************************
+
+ WARNING: one or more patch items are invalid for namespace for ns={{ ns }}
+
+ ***************************************************************************
+
+ The following resource patches will be processed:
+
+ {% for i in (resources | intersect(valid_resource_patches)) %}
+ {{ i }}
+ {% endfor %}
+
+ The following resource patches will NOT be processed:
+
+ {% for i in (resources | difference(valid_resource_patches)) %}
+ {{ i }}
+ {% endfor %}
+
+ {{ (resources | difference(valid_resource_patches)) }}
+ {{ (resources | difference(valid_resource_patches)) | length }}
+
+ Press enter to continue or Ctrl+C to abort?
+ when: (resources | difference(valid_resource_patches)) | length > 0
+
-
include_tasks: patch_resource.yml
with_items:
- "{{ resources }}"
+ "{{ valid_resource_patches }}"
loop_control:
loop_var: resource_patch
- name: Set up watch for rollout status of resources with overrides
- shell: oc rollout status {{ item.kind }}/{{ item.name }} -n {{ ns }} -w
+ shell: oc rollout status {{ item.kind }}/{{ item.name }} -n {{ ns }} -w
async: 7200
poll: 0
register: rollout_status
+ changed_when: rollout_status.stderr is defined and rollout_status.stderr != "" and (rollout_status.stderr is regex("not found|server doesn't have a resource type"))
with_items:
- "{{ resources }}"
+ '{{ valid_resource_patches }}'
+ when:
+ - resource_limits_wait_for_rollouts_per_ns | default(true) | bool
- name: Wait for all rollouts to complete for ns={{ ns }}
async_status: jid={{ item.ansible_job_id }}
@@ -22,3 +56,9 @@
retries: 300
with_items:
"{{ rollout_status.results }}"
+ when: resource_limits_wait_for_rollouts_per_ns | default(true) | bool
+ changed_when: patch_jobs.stderr is defined and patch_jobs.stderr != "" and (patch_jobs.stderr is regex("not found|server doesn't have a resource type"))
+ failed_when:
+ - patch_jobs.rc is defined
+ - patch_jobs.rc != 0
+ - not (patch_jobs.stderr is regex("not found|server doesn't have a resource type") and not (resource_limits_fail_on_missing_resource | default(false) | bool))
diff --git a/roles/resource_limits/tasks/patch_resource.yml b/roles/resource_limits/tasks/patch_resource.yml
index 89c0e630..e5aa3a26 100644
--- a/roles/resource_limits/tasks/patch_resource.yml
+++ b/roles/resource_limits/tasks/patch_resource.yml
@@ -1,45 +1,99 @@
---
-- name: Pause auto-rollout for {{ resource_patch.kind }}/{{ resource_patch.name }}
- shell: oc rollout pause {{ resource_patch.kind }}/{{ resource_patch.name }} -n {{ ns }}
- register: rollout_pause_cmd
- changed_when: rollout_pause_cmd.rc == 0
- failed_when: rollout_pause_cmd.rc != 0 and ("is already paused" not in rollout_pause_cmd.stderr)
- when:
- - resource_limits_vertical_scaling | default(true) | bool
- - resource_limits_horizontal_scaling | default(false) | bool
-
-# set resources does not properly handle disabling a limit by setting it to 0 when directly
-# modifying the config. We can bypass the incorrect validation by outputting the new config
-# with dry-run and applying it with replace instead.
-#
-# A request/limit with a value of 0 has the same behavior as if the setting wasn't there in
-# the first place; however, people aren't used to seeing this. Since we are already piping the
-# output anyway, we go ahead and remove the confusing values=0 with a simple pass thru jq
-- name: Update resource requests/limits for {{ resource_patch.kind }}/{{ resource_patch.name }}
- shell: |
- oc set resources {{ resource_patch.kind }} {{ resource_patch.name }} -n {{ ns }}
- {%- for k,v in resource_patch.resources.items() %}
- --{{ k }}={% for k2,v2 in v.items() %}{{ k2 }}={{ v2 }}{% if not loop.last %},{% endif %}{% endfor %}
- {%- endfor %}
- --dry-run -o json | jq 'del(.spec.template.spec.containers[].resources[][]|select(.=="0"))' | oc replace -f -
- register: set_resources_cmd
- changed_when: ('not changed' not in set_resources_cmd.stderr)
- failed_when: set_resources_cmd.stderr != '' and ("not changed" not in set_resources_cmd.stderr)
- when:
- - resource_limits_vertical_scaling | default(true) | bool
- - resource_patch.resources is defined
+- name: Check resources patch inputs are valid/complete
+ set_fact:
+ needs_horizontal_scaling: "{{ ((resource_patch.replicas | d(-1, true)) > 0) and (resource_limits_horizontal_scaling | default(false) | bool) }}"
+ needs_vertical_scaling: "{{ (resource_patch.resources | d({}) | length > 0) and (resource_limits_vertical_scaling | default(true) | bool) }}"
+
+- name: Check if resource exists
+ shell: oc get {{ resource_patch.kind }} {{ resource_patch.name }} -n {{ ns }}
+ changed_when: false
+ register: target_resource_exists
+ failed_when:
+ - target_resource_exists.rc != 0
+ - not (target_resource_exists.stderr is regex("not found|server doesn't have a resource type") and not (resource_limits_fail_on_missing_resource | default(false) | bool))
-- name: Update number of replicas for {{ resource_patch.kind }}/{{ resource_patch.name }}
- shell: oc scale {{ resource_patch.kind }} {{ resource_patch.name }} --replicas={{ resource_patch.replicas }} -n {{ ns }}
- register: scale_resources_cmd
+- name: Skip attempting updates that cannot succeed due to invalid target/inputs
when:
- - resource_limits_horizontal_scaling | default(false) | bool
- - resource_patch.replicas is defined
-
-- name: Resume auto-rollout for {{ resource_patch.kind }}/{{ resource_patch.name }} if it was paused by this script
- shell: oc rollout resume {{ resource_patch.kind }}/{{ resource_patch.name }} -n {{ ns }}
- register: rollout_resume_cmd
- changed_when: rollout_resume_cmd.rc == 0
- failed_when: rollout_resume_cmd.rc != 0 and "is not paused" not in rollout_resume_cmd.stderr
- when: rollout_pause_cmd is changed
+ - target_resource_exists is succeeded
+ - (needs_horizontal_scaling or needs_vertical_scaling)
+ block:
+ - name: Pause auto-rollout for {{ resource_patch.kind }}/{{ resource_patch.name }}
+ shell: oc rollout pause {{ resource_patch.kind }}/{{ resource_patch.name }} -n {{ ns }}
+ register: rollout_pause_cmd
+ changed_when: rollout_pause_cmd.rc == 0
+ failed_when:
+ - rollout_pause_cmd.rc != 0
+ - rollout_pause_cmd.stderr is not regex("is already paused|is not supported")
+
+ # set resources does not properly handle disabling a limit by setting it to 0 when directly
+ # modifying the config. We can bypass the incorrect validation by outputting the new config
+ # with dry-run and applying it with replace instead.
+ #
+ # A request/limit with a value of 0 has the same behavior as if the setting wasn't there in
+ # the first place; however, people aren't used to seeing this. Since we are already piping the
+ # output anyway, we go ahead and remove the confusing values=0 with a simple pass thru jq
+ - name: Update resource requests/limits for {{ resource_patch.kind }}/{{ resource_patch.name }}
+ vars:
+ containerResources: |
+ {% if resource_patch.resources is not defined -%}
+ []
+ {%- elif (resource_patch.resources | type_debug) == "list" -%}
+ {{ resource_patch.resources }}
+ {%- else -%}
+ {{ [resource_patch.resources] }}
+ {%- endif %}
+ shell: |
+ (set -o pipefail && oc set resources {{ resource_patch.kind }} {{ resource_patch.name }} -n {{ ns }}
+ {%- if item.container | d() | length > 0 %}
+ --containers={{ item.container }} \
+ {%- endif %}
+ {%- if item.requests | d({}) | length > 0 %}
+ --requests={% for k,v in item.requests.items() %}{{ k }}={{ v }}{% if not loop.last %},{% endif %}{% endfor %} \
+ {%- endif %}
+ {%- if item.limits | d({}) | length > 0 %}
+ --limits={% for k,v in item.limits.items() %}{{ k }}={{ v }}{% if not loop.last %},{% endif %}{% endfor %} \
+ {%- endif %}
+ --dry-run -o json | jq 'del(.spec.template.spec.containers[].resources[][]|select(.=="0"))' | oc replace -f -)
+ register: set_resources_cmd
+ changed_when: set_resources_cmd.rc == 0 and ('not changed' not in set_resources_cmd.stderr)
+ failed_when:
+ - set_resources_cmd.rc != 0
+ - set_resources_cmd.stderr != ""
+ - ("not changed" not in set_resources_cmd.stderr)
+ - not (set_resources_cmd.stderr is regex("not found|server doesn't have a resource type") and not (resource_limits_fail_on_missing_resource | default(false) | bool))
+ when:
+ - needs_vertical_scaling
+ - (item.requests is defined or item.limits is defined)
+ loop: "{{ containerResources }}"
+
+ - name: Update number of replicas for {{ resource_patch.kind }}/{{ resource_patch.name }}
+ shell: oc scale {{ resource_patch.kind }} {{ resource_patch.name }} --replicas={{ resource_patch.replicas }} -n {{ ns }}
+ register: scale_resources_cmd
+ when: needs_horizontal_scaling
+
+ rescue:
+ #
+ # In theory, this rescue should be able to use ansible_failed_task to read the task name; however,
+ # this appears to be a bug in ansible that causes another failure about undefined variable
+ # see: https://github.com/ansible/ansible/issues/49942
+ #
+
+ - name: A failure occurred while processing an app resource in ns={{ ns }}
+ pause:
+ prompt: |
+ The patch that failed was (see previous task for details):
+
+ {{ resource_patch | to_yaml }}
+
+ You can continue to process the remaining changes by pressing enter or use Ctrl+C then "a" to abort.
+
+ NOTE: If a rollout was paused by this script, it will be resumed automatically regardless of
+ whether you continue or abort in order to leave the cluster in a consistent state.
+
+ always:
+ - name: Resume auto-rollout for {{ resource_patch.kind }}/{{ resource_patch.name }} if it was paused by this script
+ shell: oc rollout resume {{ resource_patch.kind }}/{{ resource_patch.name }} -n {{ ns }}
+ register: rollout_resume_cmd
+ failed_when: rollout_resume_cmd.rc != 0 and "is not paused" not in rollout_resume_cmd.stderr
+ when: rollout_pause_cmd is changed
diff --git a/roles/rhsso-user/defaults/main.yml b/roles/rhsso-user/defaults/main.yml
index a2db4ace..29d8a3ec 100644
--- a/roles/rhsso-user/defaults/main.yml
+++ b/roles/rhsso-user/defaults/main.yml
@@ -4,3 +4,15 @@ rhsso_user_namespace: "{{ eval_user_rhsso_namespace | default('user-sso') }}"
rhsso_user_mdc_namespace: "{{ eval_mdc_namespace | default('mobile-developer-console') }}"
rhsso_user_rbac_name: mdc-account-keycloak-operator
rhsso_user_client_id: user-sso
+
+rhsso_user_resources:
+ - name: sso
+ kind: dc
+ resources:
+ requests:
+ memory: 500Mi
+ - name: sso-postgresql
+ kind: dc
+ resources:
+ requests:
+ memory: 25Mi
diff --git a/roles/rhsso-user/tasks/new_limits.yml b/roles/rhsso-user/tasks/new_limits.yml
new file mode 100644
index 00000000..240d641a
--- /dev/null
+++ b/roles/rhsso-user/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for user rhsso
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ rhsso_user_namespace }}"
+ resources: "{{ rhsso_user_resources }}"
+ when: (rhsso_user_resources | d([], true) | length) > 0
diff --git a/roles/rhsso-user/tasks/upgrade.yaml b/roles/rhsso-user/tasks/upgrade.yaml
deleted file mode 100644
index 5df96b90..00000000
--- a/roles/rhsso-user/tasks/upgrade.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
----
-- name: "include rhsso vars"
- include_vars: ../../rhsso/defaults/main.yml
-
-- name: patch new operator version
- shell: "oc patch deployment keycloak-operator -n {{ rhsso_user_namespace }} --type json -p '[{\"op\": \"replace\", \"path\": \"/spec/template/spec/containers/0/image\", \"value\": \"{{ upgrade_sso_operator_image }}\"}]'"
- register: patch
- failed_when: patch.stderr != ''
diff --git a/roles/rhsso/defaults/main.yml b/roles/rhsso/defaults/main.yml
index 2f3c9078..6eafc5bb 100644
--- a/roles/rhsso/defaults/main.yml
+++ b/roles/rhsso/defaults/main.yml
@@ -58,4 +58,20 @@ rhsso_backups:
rhsso_threescale_route_creator_role: '3scale-route-creator'
rhsso_threescale_route_creator_role_filepath: '/tmp/3scale-route-creator.yml'
-upgrade_sso_operator_image: "quay.io/integreatly/keycloak-operator:{{rhsso_operator_release_tag}}"
+rhsso_operator_image: "quay.io/integreatly/keycloak-operator:{{rhsso_operator_release_tag}}"
+
+rhsso_resources:
+ - name: sso
+ kind: dc
+ resources:
+ requests:
+ memory: 500Mi
+ - name: sso-postgresql
+ kind: dc
+ resources:
+ requests:
+ memory: 25Mi
+
+rhsso_image_streams:
+ - "{{ rhsso_imagestream_name }}"
+ - postgresql:9.5
diff --git a/roles/rhsso/tasks/new_limits.yml b/roles/rhsso/tasks/new_limits.yml
new file mode 100644
index 00000000..1df4d8ef
--- /dev/null
+++ b/roles/rhsso/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for rhsso
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ rhsso_namespace }}"
+ resources: "{{ rhsso_resources }}"
+ when: (rhsso_resources | d([], true) | length) > 0
diff --git a/roles/rhsso/tasks/upgrade.yaml b/roles/rhsso/tasks/upgrade.yaml
index fb96788c..d2876fb3 100644
--- a/roles/rhsso/tasks/upgrade.yaml
+++ b/roles/rhsso/tasks/upgrade.yaml
@@ -1,5 +1,27 @@
---
-- name: patch new operator version
- shell: "oc patch deployment keycloak-operator -n {{ eval_rhsso_namespace }} --type json -p '[{\"op\": \"replace\", \"path\": \"/spec/template/spec/containers/0/image\", \"value\": \"{{ upgrade_sso_operator_image }}\"}]'"
+- name: Setup RH-SSO Imagestreams
+ include_tasks: imagestreams.yaml
+ vars:
+ sso_namespace: "{{ rhsso_namespace }}"
+
+- name: patch new keycloak operator version
+ shell: "oc patch deployment keycloak-operator -n {{ eval_rhsso_namespace }} --type json -p '[{\"op\": \"replace\", \"path\": \"/spec/template/spec/containers/0/image\", \"value\": \"{{ rhsso_operator_image }}\"}]'"
register: patch
failed_when: patch.stderr != ''
+
+- name: Wait for the new keycloak operator to be ready
+ shell: "oc rollout status deployment/keycloak-operator -n {{ eval_rhsso_namespace }}"
+ register: rollout_cmd
+ failed_when: rollout_cmd.rc != 0
+ changed_when: rollout_cmd.rc == 0
+
+- name: Delete existing prometheus rule
+ shell: "oc delete prometheusrule application-monitoring -n {{ eval_rhsso_namespace }}"
+ register: delete_cmd
+ failed_when: delete_cmd.stderr != '' and 'NotFound' not in delete_cmd.stderr
+ changed_when: delete_cmd.rc == 0
+
+- name: "patch the monitoringResourcesCreated boolean to false in the rhsso CR"
+ shell: "oc patch keycloak rhsso -n {{ eval_rhsso_namespace }} --type json -p '[{\"op\": \"replace\", \"path\": \"/status/monitoringResourcesCreated\", \"value\": false}]'"
+ register: cr_patch_keycloak_version
+ failed_when: cr_patch_keycloak_version.stderr != ''
diff --git a/roles/rhsso/tasks/upgrade_images.yml b/roles/rhsso/tasks/upgrade_images.yml
new file mode 100644
index 00000000..c09db3d1
--- /dev/null
+++ b/roles/rhsso/tasks/upgrade_images.yml
@@ -0,0 +1,12 @@
+---
+
+- name: Import Image Streams
+ shell: oc import-image {{ patch_image_stream_item }} -n openshift
+ with_items: "{{ rhsso_image_streams }}"
+ loop_control:
+ loop_var: patch_image_stream_item
+
+- name: Patch Keycloak Operator Deployment
+ shell: oc patch deployment keycloak-operator --patch='{"spec":{"template":{"spec":{"containers":[{"name":"keycloak-operator","image":"{{ rhsso_operator_image }}"}]}}}}' --namespace {{ rhsso_namespace }}
+ register: keycloak_operator_image_patch
+ failed_when: keycloak_operator_image_patch.stderr
diff --git a/roles/ups/defaults/main.yml b/roles/ups/defaults/main.yml
index 8f0e0eea..1b867d79 100644
--- a/roles/ups/defaults/main.yml
+++ b/roles/ups/defaults/main.yml
@@ -1,6 +1,6 @@
ups_namespace: "{{ eval_ups_namespace | default('mobile-unifiedpush') }}"
ups_app_namespaces: "{{ eval_mdc_namespace | default('mobile-developer-console') }}"
-ups_resources:
+ups_resources_list:
- "{{ ups_operator_resources }}/service_account.yaml"
- "{{ ups_operator_resources }}/role.yaml"
- "{{ ups_operator_resources }}/crds/push_v1alpha1_androidvariant_crd.yaml"
@@ -10,6 +10,7 @@ ups_resources:
ups_operator_deployment: "{{ ups_operator_resources }}/operator.yaml"
ups_template_dir: /tmp
ups_server_name: unifiedpush
+
#backup
ups_backup: "{{ backup_restore_install | default(false) }}"
ups_backup_name: ups-daily-at-midnight
@@ -18,8 +19,9 @@ ups_backup_secret: "s3-credentials"
ups_backup_secret_namespace: "{{ backup_namespace }}"
ups_encryption_secret: ''
ups_encryption_secret_namespace: "{{ backup_namespace }}"
+
#monitor
-ups_svc_monitor_resources:
+ups_svc_monitor_resource_list:
- "{{ ups_operator_resources }}/monitor/service_monitor.yaml"
- "{{ ups_operator_resources }}/monitor/push_service_monitor.yaml"
ups_svc_monitor_templates:
@@ -28,3 +30,6 @@ ups_svc_monitor_templates:
- operator_grafana_dashboard
- operator_prometheus_rule
ups_message_broker: false
+
+#scaling
+ups_resources: []
diff --git a/roles/ups/tasks/install-operator.yml b/roles/ups/tasks/install-operator.yml
index 8a9ed25b..d5520f25 100644
--- a/roles/ups/tasks/install-operator.yml
+++ b/roles/ups/tasks/install-operator.yml
@@ -11,7 +11,7 @@
- name: Install ups resources
shell: "oc apply -f {{ item }} -n {{ ups_namespace }}"
- with_items: "{{ ups_resources }}"
+ with_items: "{{ ups_resources_list }}"
- name: Copy role binding template
template:
diff --git a/roles/ups/tasks/monitoring.yml b/roles/ups/tasks/monitoring.yml
index 8ff391e7..d91fa333 100644
--- a/roles/ups/tasks/monitoring.yml
+++ b/roles/ups/tasks/monitoring.yml
@@ -1,7 +1,7 @@
---
- name: Create Service Monitor resource
shell: "oc apply -f {{ item }} -n {{ ups_namespace }}"
- with_items: "{{ ups_svc_monitor_resources }}"
+ with_items: "{{ ups_svc_monitor_resource_list }}"
register: output
failed_when: output.stderr != '' and 'already exists' not in output.stderr
diff --git a/roles/ups/tasks/new_limits.yml b/roles/ups/tasks/new_limits.yml
new file mode 100644
index 00000000..658fc267
--- /dev/null
+++ b/roles/ups/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for ups
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ ups_namespace }}"
+ resources: "{{ ups_resources }}"
+ when: (ups_resources | d([], true) | length) > 0
diff --git a/roles/ups/tasks/upgrade_images.yml b/roles/ups/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/ups/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/roles/ups/templates/operator.yml.j2 b/roles/ups/templates/operator.yml.j2
index dcbb6491..e3ae4a4d 100644
--- a/roles/ups/templates/operator.yml.j2
+++ b/roles/ups/templates/operator.yml.j2
@@ -36,7 +36,5 @@ spec:
value: "{{ ups_image }}"
- name: OAUTH_PROXY_IMAGE_STREAM_INITIAL_IMAGE
value: "{{ ups_proxy_image }}"
- - name: BACKUP_IMAGE
- value: {{ backup_image }}
- name: APP_NAMESPACES
value: "{{ ups_app_namespaces }}"
diff --git a/roles/ups/templates/operator_prometheus_rule.yml.j2 b/roles/ups/templates/operator_prometheus_rule.yml.j2
index de4eb569..a67b63a0 100644
--- a/roles/ups/templates/operator_prometheus_rule.yml.j2
+++ b/roles/ups/templates/operator_prometheus_rule.yml.j2
@@ -16,6 +16,6 @@ spec:
labels:
severity: critical
annotations:
- description: "The UnifiedPush Operator has been down for more than 5 minutes. "
- summary: "The UnifiedPush Operator is down. For more information see on the UnifiedPush Operator https://github.com/aerogear/unifiedpush-operator"
+ description: "The UnifiedPush Operator has been down for more than 5 minutes."
+ summary: "The UnifiedPush Operator is down."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-operator.adoc"
diff --git a/roles/ups/templates/prometheus_rule.yml.j2 b/roles/ups/templates/prometheus_rule.yml.j2
index b745859c..cf949155 100644
--- a/roles/ups/templates/prometheus_rule.yml.j2
+++ b/roles/ups/templates/prometheus_rule.yml.j2
@@ -16,8 +16,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The UnifiedPush Pod Server has been down for more than 5 minutes. "
- summary: "The UnifiedPush Pod Server is down. For more information see on the UnifiedPush at https://github.com/aerogear/aerogear-unifiedpush-server"
+ description: "The aerogear-unifiedpush-server Pod Server has been down for more than 5 minutes."
+ summary: "The aerogear-unifiedpush-server Pod Server is down."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushConsoleDown
expr: absent(kube_endpoint_address_available{endpoint="{{ ups_server_name }}-unifiedpush"} == 1)
@@ -25,8 +25,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The UnifiedPush admin console has been down for more than 5 minutes. "
- summary: "The UnifiedPush admin console endpoint has been unavailable for more that 5 minutes. For more information see on the UnifiedPush at https://github.com/aerogear/UnifiedPush "
+ description: "The aerogear-unifiedpush-server admin console has been down for more than 5 minutes. "
+ summary: "The aerogear-unifiedpush-server admin console endpoint has been unavailable for more that 5 minutes."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushDatabaseDown
expr: absent(kube_pod_container_status_running{namespace="{{ ups_namespace }}",container="postgresql"} == 1)
@@ -34,8 +34,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The UnifiedPush Database pod has been down for more than 5 minutes"
- summary: "The UnifiedPush Database is down. For more information see on the UnifiedPush at https://github.com/aerogear/aerogear-unifiedpush-serve"
+ description: "The aerogear-unifiedpush-server Database pod has been down for more than 5 minutes."
+ summary: "The aerogear-unifiedpush-server Database is down."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushJavaHeapThresholdExceeded
expr: |
@@ -46,8 +46,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The Heap Usage of the UnifiedPush Server exceeded 90% of usage"
- summary: "The UnifiedPush Server JVM Heap Threshold Exceeded 90% of usage. For more information see on the UnifiedPush Server at https://github.com/aerogear/UnifiedPush "
+ description: "The Heap Usage of the aerogear-unifiedpush-server Server exceeded 90% of usage."
+ summary: "The aerogear-unifiedpush-server Server JVM Heap Threshold Exceeded 90% of usage."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushJavaNonHeapThresholdExceeded
expr: |
@@ -58,8 +58,8 @@ spec:
labels:
severity: critical
annotations:
- description: "The nonheap usage of the UnifiedPush Server exceeded 90% of usage"
- summary: "The nonheap usage of the UnifiedPush Server exceeded 90% of usage .For more information see on the UnifiedPush Server at https://github.com/aerogear/UnifiedPush "
+ description: "The nonheap usage of the aerogear-unifiedpush-server Server exceeded 90% of usage."
+ summary: "The nonheap usage of the aerogear-unifiedpush-server Server exceeded 90% of usage."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushJavaGCTimePerMinuteScavenge
expr: |
@@ -68,8 +68,8 @@ spec:
labels:
severity: critical
annotations:
- description: "Amount of time per minute spent on garbage collection in the UnifiedPush Server pod exceeds 90%"
- summary: "Amount of time per minute spent on garbage collection in the UnifiedPush Server pod exceeds 90%. This could indicate that the available heap memory is insufficient..For more information see on the UnifiedPush Server at https://github.com/aerogear/UnifiedPush "
+ description: "Amount of time per minute spent on garbage collection in the aerogear-unifiedpush-server Server pod exceeds 90%."
+ summary: "Amount of time per minute spent on garbage collection in the aerogear-unifiedpush-server Server pod exceeds 90%. This could indicate that the available heap memory is insufficient."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushJavaDeadlockedThreads
expr: |
@@ -79,8 +79,8 @@ spec:
labels:
severity: warning
annotations:
- description: "Number of threads in deadlock state of the UnifiedPush Server > 0"
- summary: "Number of threads in deadlock state of the UnifiedPush Server > 0.For more information see on the UnifiedPush Server at https://github.com/aerogear/UnifiedPush "
+ description: "Number of threads in deadlock state of the aerogear-unifiedpush-server Server > 0."
+ summary: "Number of threads in deadlock state of the aerogear-unifiedpush-server Server > 0."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
- alert: UnifiedPushMessagesFailures
expr: >
@@ -90,6 +90,6 @@ spec:
labels:
severity: warning
annotations:
- description: "More than 50 failed requests attempts for UnifiedPush Server fails over the last 5 minutes."
- summary: "More than 50 failed messages attempts for UnifiedPush Server fails over the last 5 minutes. For more information see on the UnifiedPush Server at https://github.com/aerogear/UnifiedPush "
+ description: "More than 50 failed requests attempts for aerogear-unifiedpush-server Server fails over the last 5 minutes."
+ summary: "More than 50 failed messages attempts for aerogear-unifiedpush-server Server fails over the last 5 minutes."
sop_url: "https://github.com/aerogear/unifiedpush-operator/blob/{{ ups_operator_release_tag }}/SOP/SOP-push.adoc"
diff --git a/roles/walkthroughs/templates/crud_spboot_example.yml b/roles/walkthroughs/templates/crud_spboot_example.yml
index ec8d1525..57062348 100644
--- a/roles/walkthroughs/templates/crud_spboot_example.yml
+++ b/roles/walkthroughs/templates/crud_spboot_example.yml
@@ -1,269 +1,68 @@
-apiVersion: v1
+apiVersion: template.openshift.io/v1
kind: Template
metadata:
name: spring-boot-rest-http-crud
annotations:
- iconClass: icon-spring
- tags: spring-boot, rest, crud, java, microservice
- openshift.io/display-name: Spring Boot - REST HTTP and CRUD
- openshift.io/provider-display-name: "Red Hat, Inc."
- openshift.io/documentation-url: "https://appdev.prod-preview.openshift.io/docs/spring-boot-runtime.html#mission-crud-spring-boot-tomcat"
- description: >-
- The Relational Database Backend booster expands on the REST API Level 0 booster to provide a basic example of performing create, read, update and delete (CRUD) operations on a PostgreSQL database using a simple HTTP API.
- CRUD operations are the four basic functions of persistent storage, widely used when developing an HTTP API dealing with a database.
-parameters:
-- name: RUNTIME_VERSION
- displayName: OpenJDK 8 image version to use
- description: Specifies which version of the OpenShift OpenJDK 8 image to use
- value: {{ crud_spboot_runtime_version }}
- required: true
+ iconClass: icon-node
+ tags: nodejs, crud
+ openshift.io/display-name: Fruit CRUD Application
+ openshift.io/provider-display-name: Red Hat, Inc.
+ openshift.io/documentation-url: https://github.com/integr8ly/walkthrough-applications.git
+ description: Basic CRUD application for fruit
objects:
-- apiVersion: v1
- kind: ImageStream
+- kind: DeploymentConfig
+ apiVersion: apps.openshift.io/v1
metadata:
name: spring-boot-rest-http-crud
- spec: {}
-- apiVersion: v1
- kind: ImageStream
- metadata:
- name: runtime
- spec:
- tags:
- - name: "${RUNTIME_VERSION}"
- from:
- kind: DockerImage
- name: registry.access.redhat.com/redhat-openjdk-18/openjdk18-openshift:${RUNTIME_VERSION}
-
-- apiVersion: v1
- kind: Secret
- metadata:
- labels:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- version: "BOOSTER_VERSION"
- group: io.openshift.booster
- name: my-database-secret
- stringData:
- user: luke
- password: secret
-- apiVersion: v1
- kind: Service
- metadata:
- labels:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- version: "BOOSTER_VERSION"
- group: io.openshift.booster
- name: spring-boot-rest-http-crud
- spec:
- ports:
- - name: http
- port: 8080
- protocol: TCP
- targetPort: 8080
- selector:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- group: io.openshift.booster
-- apiVersion: v1
- kind: DeploymentConfig
- metadata:
labels:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- version: "BOOSTER_VERSION"
- group: io.openshift.booster
- name: spring-boot-rest-http-crud
+ app: crud-app
spec:
replicas: 1
- revisionHistoryLimit: 2
+ revisionHistoryLimit: 10
+ test: false
selector:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- group: io.openshift.booster
- strategy:
- rollingParams:
- timeoutSeconds: 3600
- type: Rolling
+ app: crud-app
template:
metadata:
labels:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- version: "BOOSTER_VERSION"
- group: io.openshift.booster
+ app: crud-app
spec:
containers:
- - env:
- - name: DB_USERNAME
- valueFrom:
- secretKeyRef:
- key: user
- name: my-database-secret
- - name: DB_PASSWORD
- valueFrom:
- secretKeyRef:
- key: password
- name: my-database-secret
- - name: JAVA_OPTIONS
- value: -Dspring.profiles.active=openshift
- - name: KUBERNETES_NAMESPACE
- valueFrom:
- fieldRef:
- fieldPath: metadata.namespace
- image: quay.io/integreatly/spring-boot-rest-http-crud:{{ crud_spboot_image_tag }}
- imagePullPolicy: Always
- livenessProbe:
- httpGet:
- path: /health
- port: 8080
- scheme: HTTP
- initialDelaySeconds: 180
- name: spring-boot
+ - name: crud-app
+ image: quay.io/integreatly/fruit-crud-app:1.0.1
ports:
- containerPort: 8080
- name: http
protocol: TCP
- - containerPort: 8778
- name: jolokia
- protocol: TCP
- readinessProbe:
- httpGet:
- path: /health
- port: 8080
- scheme: HTTP
- initialDelaySeconds: 10
- securityContext:
- privileged: false
- triggers:
- - imageChangeParams:
- automatic: true
- containerNames:
- - spring-boot
- from:
- kind: ImageStreamTag
- name: 'spring-boot-rest-http-crud:latest'
- type: ConfigChange
-- apiVersion: v1
- kind: Route
+ resources: {}
+ terminationMessagePath: "/dev/termination-log"
+ terminationMessagePolicy: File
+ imagePullPolicy: IfNotPresent
+ restartPolicy: Always
+ terminationGracePeriodSeconds: 30
+ dnsPolicy: ClusterFirst
+ securityContext: {}
+ schedulerName: default-scheduler
+- kind: Service
+ apiVersion: v1
metadata:
- labels:
- app: spring-boot-rest-http-crud
- provider: snowdrop
- version: "BOOSTER_VERSION"
- group: io.openshift.booster
- name: crud
+ name: spring-boot-rest-http-crud
spec:
- port:
+ ports:
+ - protocol: TCP
+ port: 8080
targetPort: 8080
- to:
- kind: Service
- name: spring-boot-rest-http-crud
-- apiVersion: v1
- kind: ImageStream
- metadata:
- annotations:
- openshift.io/generated-by: OpenShiftNewApp
- creationTimestamp: null
- labels:
- app: my-database
- name: my-database
- spec:
- lookupPolicy:
- local: false
- tags:
- - annotations:
- openshift.io/imported-from: openshift/postgresql-92-centos7
- from:
- kind: DockerImage
- name: openshift/postgresql-92-centos7
- generation: null
- importPolicy: {}
- name: latest
- referencePolicy:
- type: ""
- status:
- dockerImageRepository: ""
-- apiVersion: v1
- kind: DeploymentConfig
- metadata:
- annotations:
- openshift.io/generated-by: OpenShiftNewApp
- creationTimestamp: null
- labels:
- app: my-database
- name: my-database
- spec:
- replicas: 1
selector:
- app: my-database
- deploymentconfig: my-database
- strategy:
- resources: {}
- template:
- metadata:
- annotations:
- openshift.io/generated-by: OpenShiftNewApp
- creationTimestamp: null
- labels:
- app: my-database
- deploymentconfig: my-database
- spec:
- containers:
- - env:
- - name: POSTGRESQL_DATABASE
- value: my_data
- - name: POSTGRESQL_PASSWORD
- value: secret
- - name: POSTGRESQL_USER
- value: luke
- image: openshift/postgresql-92-centos7
- name: my-database
- ports:
- - containerPort: 5432
- protocol: TCP
- resources: {}
- volumeMounts:
- - mountPath: /var/lib/pgsql/data
- name: my-database-volume-1
- volumes:
- - emptyDir: {}
- name: my-database-volume-1
- test: false
- triggers:
- - type: ConfigChange
- - imageChangeParams:
- automatic: true
- containerNames:
- - my-database
- from:
- kind: ImageStreamTag
- name: my-database:latest
- type: ImageChange
- status:
- availableReplicas: 0
- latestVersion: 0
- observedGeneration: 0
- replicas: 0
- unavailableReplicas: 0
- updatedReplicas: 0
-- apiVersion: v1
- kind: Service
+ app: crud-app
+- kind: Route
+ apiVersion: route.openshift.io/v1
metadata:
- annotations:
- openshift.io/generated-by: OpenShiftNewApp
- creationTimestamp: null
- labels:
- app: my-database
- name: my-database
+ name: crud
spec:
- ports:
- - name: 5432-tcp
- port: 5432
- protocol: TCP
- targetPort: 5432
- selector:
- app: my-database
- deploymentconfig: my-database
- status:
- loadBalancer: {}
+ to:
+ kind: Service
+ name: spring-boot-rest-http-crud
+ port:
+ targetPort: 8080
+ tls:
+ termination: edge
+ wildcardPolicy: None
\ No newline at end of file
diff --git a/roles/webapp/defaults/main.yml b/roles/webapp/defaults/main.yml
index 84142739..6e05d0a9 100644
--- a/roles/webapp/defaults/main.yml
+++ b/roles/webapp/defaults/main.yml
@@ -15,3 +15,5 @@ webapp_walkthrough_locations:
- "https://github.com/integr8ly/tutorial-web-app-walkthroughs#v1.7.3"
webapp_provision_services: []
webapp_watch_services: []
+
+webapp_resources: []
\ No newline at end of file
diff --git a/roles/webapp/tasks/new_limits.yml b/roles/webapp/tasks/new_limits.yml
new file mode 100644
index 00000000..a831b5d8
--- /dev/null
+++ b/roles/webapp/tasks/new_limits.yml
@@ -0,0 +1,8 @@
+---
+- name: Apply resource overrides for webapp
+ include_role:
+ name: resource_limits
+ vars:
+ ns: "{{ webapp_namespace }}"
+ resources: "{{ webapp_resources }}"
+ when: (webapp_resources | d([], true) | length) > 0
diff --git a/roles/webapp/tasks/upgrade.yaml b/roles/webapp/tasks/upgrade.yaml
index 92c54514..30002a75 100644
--- a/roles/webapp/tasks/upgrade.yaml
+++ b/roles/webapp/tasks/upgrade.yaml
@@ -22,6 +22,15 @@
failed_when: rollout_cmd.rc != 0
changed_when: rollout_cmd.rc == 0
+- name: Add mobile walkthrough if mdc is installed
+ set_fact:
+ webapp_walkthrough_locations: "{{ webapp_walkthrough_locations + [mobile_walkthrough_location] }}"
+ when:
+ - mdc is defined
+ - mobile_walkthrough_location is defined
+ - mdc|bool == True
+ - mobile_walkthrough_location|length > 0
+
- name: Generate WebApp custom resource template
template:
src: "cr.yaml.j2"
diff --git a/roles/webapp/tasks/upgrade_images.yml b/roles/webapp/tasks/upgrade_images.yml
new file mode 100644
index 00000000..fbf48c27
--- /dev/null
+++ b/roles/webapp/tasks/upgrade_images.yml
@@ -0,0 +1,4 @@
+---
+#ToDo Implement CVE image update steps as described https://github.com/RHCloudServices/integreatly-help/blob/master/sops/cves/applying-cve-updates.md
+- debug:
+ msg: "TODO Implement me!!"
diff --git a/scripts/release.sh b/scripts/release.sh
index 479739bf..c0432d6e 100755
--- a/scripts/release.sh
+++ b/scripts/release.sh
@@ -7,8 +7,12 @@ currentBranch=$(git symbolic-ref --short HEAD)
baseBranch=""
releaseTag=""
+function reset_current_branch {
+ #checkout current branch
+ git checkout "$currentBranch"
+}
+
while getopts ":b:r:h" opt; do
- echo "opt $opt"
case ${opt} in
b)
baseBranch=${OPTARG}
@@ -46,9 +50,29 @@ echo "Flags:
exit 1
fi
-echo "cutting release ${releaseTag} from branch ${REMOTE}/${baseBranch}"
-#do a fetch
+if [[ $releaseTag =~ ^release-([0-9]+).([0-9]+).([0-9]+)-?(.*)?$ ]]; then
+ MAJOR_VERSION=${BASH_REMATCH[1]}
+ MINOR_VERSION=${BASH_REMATCH[2]}
+ PATCH_VERSION=${BASH_REMATCH[3]}
+ LABEL_VERSION=${BASH_REMATCH[4]}
+else
+ echo "Invalid release tag $releaseTag"
+ exit 1
+fi
+
+if [[ $PATCH_VERSION -gt 0 ]]; then
+ RELEASE_TYPE="patch"
+else
+ if [[ $MINOR_VERSION -gt 0 ]]; then
+ RELEASE_TYPE="minor"
+ else
+ RELEASE_TYPE="major"
+ fi
+fi
+
+echo "cutting ${RELEASE_TYPE} release ${releaseTag} from branch ${REMOTE}/${baseBranch}"
+#do a fetch
git fetch ${REMOTE}
#check we have no local changes
@@ -61,8 +85,9 @@ echo "
exit 1
fi
-# check if the specified from branch already exists if it does check it out otherwise create it
+trap reset_current_branch EXIT
+# check if the specified from branch already exists if it does check it out otherwise create it
git checkout -B ${baseBranch} ${REMOTE}/${baseBranch}
if [[ $? > 0 ]]
then
@@ -81,12 +106,20 @@ a release with that name already exists
exit 1
fi
-
sed -i.bak -E "s/^integreatly_version: .*$/integreatly_version: ${releaseTag}/g" ./inventories/group_vars/all/manifest.yaml && rm ./inventories/group_vars/all/manifest.yaml.bak
-#commit the change and push
+#commit the change, tag
git commit -am "release manifest version update for ${releaseTag}"
-git push ${REMOTE} ${baseBranch}
-#tag and push
git tag ${releaseTag}
+
+#reset upgrade playbook and variables if this is the final release
+if [[ -z $LABEL_VERSION ]]; then
+ echo "resetting upgrade playbook and variables after final release $releaseTag"
+ cp scripts/upgrade.template.yml playbooks/upgrade.yml
+ sed "s,UPGRADE_FROM_VERSION,$releaseTag,g" scripts/upgrade_vars.template.yml > playbooks/group_vars/all/upgrade.yml
+ git commit -am "Reset upgrade variables after final release ${releaseTag}"
+fi
+
+#push branch
+git push ${REMOTE} ${baseBranch}
git push ${REMOTE} ${releaseTag}
diff --git a/scripts/upgrade.template.yml b/scripts/upgrade.template.yml
new file mode 100644
index 00000000..32d79228
--- /dev/null
+++ b/scripts/upgrade.template.yml
@@ -0,0 +1,45 @@
+# This file is re-generated after each release!
+# If you need to add tasks that should run on every upgrade, update the template here ../../scripts/upgrade.template.yml
+---
+- hosts: master
+ gather_facts: no
+ tasks:
+ - include_role:
+ name: openshift
+ tasks_from: set_master_vars
+ when: run_master_tasks | default(true) | bool
+
+# Required for Ansible Tower installs that need to login via oc as a prerequisite
+- import_playbook: "./openshift.yml"
+
+- hosts: localhost
+ gather_facts: yes
+ tasks:
+ - include_role:
+ name: prerequisites
+ tasks_from: upgrade
+ vars:
+ from_versions:
+ - "{{ upgrade_from_version }}"
+
+ - name: Set Upgrade Facts
+ set_fact: upgrade_{{ item }}=true
+ with_items: "{{ upgrade_product_roles }}"
+
+ - name: Upgrade product images
+ include_role:
+ name: "{{ item }}"
+ tasks_from: "upgrade_images"
+ with_items: "{{ upgrade_product_roles }}"
+
+# Add product specific upgrade tasks here, make sure to use the "when: upgrade_<product>|bool" condition on any new tasks added!!
+#
+# - name: Some Special webapp only upgrade thing
+# include_role:
+# name: webapp
+# tasks_from: upgrade_patch
+# when: upgrade_webapp|bool
+
+#Update product version (should always be last)
+- import_playbook: "./generate-customisation-inventory.yml"
+- import_playbook: "./mobile_generate_manifest.yml"
diff --git a/scripts/upgrade_vars.template.yml b/scripts/upgrade_vars.template.yml
new file mode 100644
index 00000000..75ce52ec
--- /dev/null
+++ b/scripts/upgrade_vars.template.yml
@@ -0,0 +1,3 @@
+# DO NOT EDIT This file is auto generated!!
+upgrade_from_version: UPGRADE_FROM_VERSION
+upgrade_product_roles: []
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment