Authors: Christophe Laprun, Charles Moulliard & Georgios Andrianakis
Reviewed: March, 13 - 2019
Links: Doc describing the whole release process
- How To
- Glossary
- Overview
- Dummy release
- Release with Jenkins
- BOM versions to productization build Milestones
- Extras
- Create a product
- Create a version
- Extended sources (deprecated since we no longer use Tomcat 8.5.x which was the source of this "problem")
- Release the milestone in PNC (deprecated since this now done automatically by PFG)
- Update Share Content Request (this is no longer done and the team(s) handling the our productized BOM haven't requested it)
- Generate NVR list (this is no longer done and the team(s) handling the our productized BOM haven't requested it)
- Creating a product and version
- Projects and document references
See instructions: https://docs.engineering.redhat.com/display/JP/How+To%3A+Install+PNC+CLI
Next, create a pnc config file using the following parameters under the path $HOME/.config/pnc-cli/pnc-cli.conf and add your kerberos identifier
[PNC]
#Upshift-stage
#pncurl = http://orch-stage.psi.redhat.com/
#keycloakurl = https://secure-sso-newcastle-devel.cloud.paas.upshift.redhat.com
# Prod
pncurl = http://orch.psi.redhat.com/
keycloakurl = https://secure-sso-newcastle.cloud.paas.upshift.redhat.com
keycloakrealm = pncredhat
keycloakclientid = pncdirect
username = KERBEROS_ID
Red Hat's internal build system, a productized version of koji.
Although the plan is to replace it with PNC (see below) for Middleware builds, it is and will be still used as a storage for build results. That is why we need to have brew tags for each product.
Alongside Maven coordinates (Group id, Artifact id, Version or GAV), Brew has its own identifier for an artifact, called NVR. When releasing a product version (e.g. RHOAR Beta), each team has to provide a list of NVRs for the maven repository zip contents.
Instructions on how to install the brew client can be found in mojo
The new build system for Middleware projects. It runs maven builds in an isolated environment.
Its maven repository manager (Indy) stores all the dependencies and files downloaded from the internet during a build to make it possible to re-run the build in the future.
For information on how to use it, check out PNC User Guide
A product represents a top-level entity in PNC and it should already be present before to start the process to build.
To productize the Spring Boot project for different versions (1.0, 1.4, 1.5, 2.1, ...), then the following Product has been created Spring Boot product.
If, for some reason, it needs to be re-created, you can either use the local instructions from this section
or the official PNC doc to create it.
If you do, be sure to use spring-boot as a short name.
In PNC, a version of a product stands for a number declared using the <major>.<minor> semantic (e.g. 7.0.).
Each version can have multiple milestones where a Milestone is a set of builds and
its name uses a <micro>.<classifier> template, where classifier is by example DR1, ER3, CR1.
Here is by example for the product Spring Boot, version 2.1, the milestones defined in PNC
The Milestones can be released.
Releasing a milestone means publishing it into brew.
You can view the existing versions either in the PNC UI (using the product page linked above)
or via pnc cli using the command pnc list-versions-for-product -n spring-boot, assuming you named the project spring-boot.
This command should result in a collection of JSON objects (unfortunately, not an array, so not a valid JSON document), each representing a version.
Look for the version top-level property of these objects. You can approximate this operation by performing:
pnc list-versions-for-product -n spring-boot | grep '"version":' and only keep the results with top-level indentation, e.g., with the following result:
"version": "1.0.0.DR0"
"version": "1.0"
"version": "1.4"
Only 1.0 and 1.4 are proper versions.
The productization (java) application (which currently is this but will soon be replaced by PNC 2 CLI) creates both of these if they don't exist, but they can also be created in the PNC UI prior to running the application.
To do it in the PNC UI, see instructions to create a project and next instructions to create a build configuration
Here is a build example created to release Spring Boot 2.1.3
and a list of projects related to the Product Spring Boot - 2.1
This document explains how to productize the upstream Spring Boot BOM and assumes basic knowledge of the Project Newcastle - PNC.
As a quick summary, a product supported by Red Hat consists of one or more upstream projects that are productized by Red Hat. A product is associated with a version number. See the glossary for more details.
Here is an overview of the required productization steps:
- Determine which product, product version and milestone to use
- If they don't already exist, then such product metadata needs to be created, see the relevant section
- PNC builds all the projects that are a part of the product. All the PNC builds should be done in a single
PNC milestone. Each PNC build consist of the following:
- dependency-alignment - changing the community versions of the artifacts to productized version,
- maven build in an isolated environment
- Gather a maven repository zip - zip file containing all the maven dependencies, that combined with public maven repositories, allowing users to use the project
- Gather licenses for the artifacts in the maven repository zip
- Gather the sources of the PNC builds
- Upload the generated files to staging area
- Release the produced milestone
- Promote the contents to the candidates area
Tasks 2-6 are performed by this jenkins job which is described in more details later on.
Tasks 2-5 are performed by the product-files-generator (also known as PFG or PiG) application, details of which can be found here
An example output of a productization run (for version 1.5.15.CR1) can be seen here
The product-files-generator (PFG from now on), in addition to the files required for a product, generates a set of helper
files that are put in the extras subdirectory. For a full list of additional files,
take a look at the product-files-generator README
A production build needs to be performed by a user with a Kerberos account, part of the LDAP jboss-prod group.
You can verify that by ssh'ing to the rcm-guest machine when you are connect to the Red Hat VPN and check which groups you belongs to:
ssh <kerberos-login>@rcm-guest.app.eng.bos.redhat.com
groups
NOTE
The best way to run the PNC and brew commands mentioned in this document is to use a docker image
from the https://gitlab.cee.redhat.com/middleware/productization-docker repository.
To build it and run commands in it use these instructions
NOTE Due to a problem in PNC, creating two or more successful builds of the same build configuration will cause uploading invalid artifacts to Brew and creating invalid sources repository. + If a build for a given release (e.g. ER3) was successful, but the binaries produced by it are invalid, please perform the next build in a new release (e.g. ER4).
The workflow to release a community or upstream project as Red Hat Final can be summarised as such.
Eng -> QE -> Signoff -> QE Doc Signoff -> RCM -> Released as Final
Eng: Engineering team
QE: Quality Engineering team
RCM: Release Engineering team
Different actors are involved and will be contacted by email or using a JIRA ticket in order to trigger the work
they will deliver for the release like: perform QE tests, review the Doc, move the GAVs productized between the candiadate and/or staging environment
and the final place which is the Red Hat Offical Maven Repository or MRRC
The PNC platform will be used by the Eng team to create (aka to convert the existing GAVs into Red Hat equivalent GAVs, ... as described within the previous section) a Red Hat release.
If we start from a new project such as Spring Boot that we will git clone from its current location https://github.com/spring-projects/spring-boot, then here are the steps required
-
Open the PNC Prod UI console and log on using your
KerberosIdandPassword
-
Create a
Spring Boot Productby clicking onCreatebutton from theProductmenu -http://orch.psi.redhat.com/pnc-web/#/product/. Fill the fieldsName,DescriptionandShort nameusing the following values.
-
Add now a new
Product versionby clicking on the buttonCreatewhich is displayed within the screen of theProductcreated. The version to be defined depends on the version of themajor.minorversion of the project
-
As a release to be productized can contain several git repos, then we are gonna to create a new project. Select then the
Projects itemfrom the menu and click on theCreatebutton. Use asProject namethe name of the github upstream project.
IMPORTANT The steps 1-2 are only needed when a newProductand/orversionmust be productized and step 4 will be used for every new project to be added -
WARNING We can now create a new
Build Configfor the project in order to specify the location of the upstream git repo that PNC will sync with a private internal git repository (that will be created incode.engineering.redhat.com). Later, it will be used by theJava Prod applicationto initiate aBuild. From theSpring Bootproject UI screen, click on the buttonCreatein order to use theWizardto create theBuild Conf. Fill the fields as showed within the next screenshots


-
End of the
WizardUI, you will be able to click on the buttoncreateto tell to PNC that it can git clone the project internally and setup the build
-
When the process is accomplished, then you can visualize the summary as depicted hereafter
IMPORTANT: The step 5 does not need to be repeated for every new build but only for the very first time !! The fields such as Build scriptmust be communicated by theProductionengineer -
Now that we have created all the needed resources under
PNC, we can create abuild configurationwithin the following git repository. According to the version to be productized, select or create a new folder (e.g spring-boot-1.5, spring-boot-2.1, ...) and next edit thebuild-config.yamlfile. Add a newbuildsection within the yaml file with the information of the newly created project.product: name: spring-boot abbreviation: spring-boot stage: GA issueTrackerUrl: http://issues.jboss.org/browse/SB # the version should match the upstream version we plan to build version: 2.1.3 # the milestone indicates the maturity of the build. In escalating maturity we have: DR (developer release), ER (engineering release), CR (candidate release) milestone: DR1 group: spring-boot-2.1-all defaultBuildParameters: environmentId: 1 buildScript: mvn clean deploy -DskipTests builds: - name: spring-boot-2.1 project: spring-boot buildScript: mvn clean deploy -DskipTests scmUrl: git+ssh://code.engineering.redhat.com/spring-projects/spring-boot.git # The sceRevision is the sha/branch/tag we plan to build in PNC scmRevision: 2.1.3.RELEASEThis configuration will be used by the Java prod application to drive PNC to build the project. Remark In order to make building projects easier, the Java prod application supports "variables" in
build-config.yaml(see this for example). The idea is that these variables get populated by the process that launches the Java prod application, usually a Jenkins job (see this for example). -
If the project to be build has some dependencies with other projects to be build, then add within your build config the following section
dependencies: - opentracing-java-spring-jaeger-spring-boot-2 - narayana-spring-boot-2 - resteasy-spring-boot-2 -
If the project to be build must include some specific GAVs version, then add the following section
customPmeParameters: - '-DdependencyOverride.org.apache.tomcat:*@*=9.0.7.redhat-16' - '-DdependencyOverride.io.undertow:*@*=2.0.15.Final-redhat-00001' - '-DdependencyOverride.org.hibernate.validator:*@*=6.0.14.Final-redhat-00001' - '-DdependencyOverride.org.hibernate:*@*=5.3.7.Final-redhat-00001' -
Commit the file modified.
-
Build the project.
Launching the java productization tool (usually as part of a Jenkins job similar to this one) Remark: This step is triggered automatically for the Snowdrop Spring Boot BOM
-
When the job is finished (= green status that you can see within the Jenkins Server), then the GAVs have been created under the
Indymaven repo and the tool has imported them within theRCM Guestmaven repo. They will appear asER1version within the repo. Remark you can find more info about the version's workflow here The tool will then create a new folder under this path - http://download-node-02.eng.bos.redhat.com/rcm-guest/staging/rhoar/spring-boot with theproduct namefollowed by theversionand.ER1(e.g http://download-node-02.eng.bos.redhat.com/rcm-guest/staging/rhoar/spring-boot/spring-boot-2.1.3.ER1/) -
If no major issues/problems are discovered, then the QE team will continue the process and will test the deliverable. The result of this work will be recorded in a signoff document. The upstream project will be released as
Final, a new Prod job triggered and the result of the process weill be to deliver/create GAVs released with the suffizCR1. This release will then be used by the RCM team, when approved to move at the launch or release date the content on theMRRCofficial repo -
If errors are reported, then QE will reject this release, the Engineering team will be notified and will resolve the problem(s). The result could be that the
build-config.yamlmust be changed to exclude some dependencies or add new, to fix a wrong upstream GAV or the code itself. Then, a new version of the project will be released asBeta2and the job doing the productization triggered again manually.
The main input that PFG uses in order to control the build process, is a build configuration file. For Spring Boot, that files can be found here. Essentially the file configures the following:
- Which repositories are part of the productization build - this repositories will be built in PNC in the order specified by the file
- What strategies will be used in order to produce the deliverables of the productization release (this is needed because PFG is used by various teams that have to produce different deliverables)
- Dependency Alignment properties. These are very important to ensure that the dependencies of the productizated BOM are what we expect them to be regardless of what the upstream BOM contains. More information about controlling the dependency aligment can be found here
Before every major release the team needs to decide what to do with the dependency alignment
An important feature of PFG that is used in conjunction with Jenkins in order to remove the need to constantly update the build configuration when performing successive builds, is the runtime variable substitution that is performed on the build configuration.
Essentially any expression like {{someVar}} will be substituted by PFG with the respective parameter passed to it.
The run-pfg Jenkins job uses this feature here.
So before a productization run, we need to check the build configuration to see if the dependency alignment needs to be updated or if the project dependencies of the BOM need to be updated (usually we either need to update the release of these dependencies or add a whole new dependency - see the git history of the spring-boot/build-config.yaml to see how examples of this).
If changes need to be made, simply perform and commit them.
The following tasks should be done only one time for setting up the permissions and credentials in order to be able to run the Jenkins job.
1 - Add Jenkins permissions: you need to be a Jenkins admin, so ask to some team member with admin preferences to add you to this group here For example, this is the profile of a person able to run the productization:
Jenkins User Id: claprun
Groups: devel
jboss-prod
ROLE_CLAPRUN
ROLE_DEVEL
ROLE_JBOSS-PROD
2 - Add credentials to Openshift: you need to go here, login using admin as user and admin as password and add a secret with username: KERBEROS_USER and password being your kerberos password.
3 - Add same credentials to Jenkins itself: you can do it here
Go to release job, insert your kerberos user into KERBEROS_USER and trigger the build.
The next steps all assume that the release job finished successfully. If something went wrong, you can look at the logs for the build by going to the
project page in PNC, find the version you just tried to perform a release for in the list of build configs and select it. You will then see the details for that particular build configuration and, in particular, the build history. Look for the build number that interest you (usually, the last one), select it to see details (in particular, logs). You can also see the status of the latest build for each build config going to the
product page and then selecting the appropriate version (e.g. 1.5).
Most of the time the issues involve figuring out the proper dependency alignment, which can be a rather tedious process since it's more or less a trial and error one.
Go to the staging repository for the product and select the newly created version to check the generated artifacts.
Once the deliverables are uploaded to the staging area, they can be promoted to candidates.
To do it run, trigger the upload to candidates job: https://spring-boot-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/prod-release/job/prod-upload-to-candidates/
Alternatively, it can be done locally:
$ ssh <keberos-login>@rcm-guest.app.eng.bos.redhat.com
$ kinit
# you'll be asked for password here, enter your kerberos password
$ /mnt/redhat/scripts/rel-eng/utility/bus-clients/stage-mw-release spring-boot-<version>.<release>
----
E.g. for 1.4.1.ER2:
----
$ /mnt/redhat/scripts/rel-eng/utility/bus-clients/stage-mw-release spring-boot-1.4.1.ER2
[NOTE] The name used must match versions in http://rcm-guest.app.eng.bos.redhat.com/rcm-guest/staging/rhoar/spring-boot.
This script idempotently copies the contents of the release from /mnt/rcm-guest/staging/rhoar/spring-boot/ to http://download-ipv4.eng.brq.redhat.com/devel/candidates/rhoar/spring-boot/.
If for some reason the release does not end up in the URL, you can also try http://download-node-02.eng.bos.redhat.com/devel/candidates/rhoar/spring-boot/
The following information ties together the various versions we use for the upstream BOM and the milestones in the PNC builds. Note that this may not be applicable for other products (or when more complete productization of Spring Framework + Spring Boot is going to be needed).
Currently when we want to release a first version of our BOM (say a version corresponding to the Pivotal Spring Boot 2.1.3.RELEASE BOM) we release 2.1.3.Beta1 of our BOM upstream. That version gets built in PNC using Milestone ER1, meaning that the version QE will test will be 2.1.3.ER1.
Note The use of this scheme for the product Milestones is enforced by this Jenkins job.
If QE finds bug or during testing we figure out that we should have made more changes to our BOM, we need to release 2.1.3.Beta2 and do another productization build using for example 2.1.3.ER2.
Once QE has cleared our 2.1.3.ER* release, we proceed to release our BOM upstream as 2.1.3.Final. That in turns triggers the productization build of 2.1.3.CR1.
When QE tests and verifies 2.1.3.CR1 then they will provide a signoff and the release process can continue.
Remark We currently don't use DR* builds except when there are changes we know will break things (like from Spring Boot 1.5 to 2.1) and in that case we only do it so we can get early feedback from QE.
Important The fact that we tag the releases as Beta is a convention that we use/follow internally but is not adopted at all by all the upstream projects (e.g Spring uses as convention M1, M2, ..., RELEASE as suffix). By sconsequence, some modifications
will then be required to adapt the existing tools when the release is not flagged as Beta.
In some cases it is needed to patch the BOM release after it the productized release has been completed and is available to customers. In this case we would do an upstream 2.1.3.SP1 release and use that to create the 2.1.3.CR2 productized build. Releasing further patches would follow the same logic.
The product, version and project can be created using either the Web User interface or the python pnc-client if installed on your machine
Here is the command to be executed to create by example the spring-boot product.
pnc create-product -p "Snowdrop Spring Boot BOM" spring-boot spring-boot
The pnc create-product command returns a minimal JSON containing
the ID of the created product.
As a project, product will change over the time, it will be required to create a new version under brew and PNC
To create a new version using PNC, run this command:
pnc create-product-version <PNC_ID_OF_THE_PRODUCT> <PRODUCT_VERSION>
E.g. to create the version 1.4 for the product previously created, then execute this command pnc create-product-version 4 1.4
Moreover, for each <major>.<minor> version (like 1.4 or 1.5), you have to create a dedicated brew tag.
For Spring Boot, the tag name should follow the following convention pnc-jb-<product-short-name>-<major>.<minor>, e.g. pnc-jb-spring-boot-1.4.
pnc-jb is a standard tag prefix for products built in PNC.
<product-short-name> is the Short name selected upon product creation.
You can also check the brew tag prefix on the version page in the PNC UI.
To request the tag, follow these instructions
Additionally, the spring boot tag has to be a part of the rhoar tag. A request similar to this should be created for it or incorporated into the ticket requesting the tag creation.
To check if a brew tag is already created, you can grep the list of all available tags:
brew list-tags
To check if your tag is under the rhoar-1.0 tag, use:
brew taginfo rhoar-1.0-maven-build
For a list of available brew commands, run brew -h
Extended sources (deprecated since we no longer use Tomcat 8.5.x which was the source of this "problem")
It is also important that the -src.zip contains both the spring-boot-bom sources (just the pom) and the sources of any unreleased artifacts.
That is why the Spring Boot build configuration uses
sourcesGeneration:
strategy: GENERATE_EXTENDEDThe GENERATE_EXTENDED strategy instructs the application to include all unreleased sources from dependencies of the built artifacts.
In the case of Spring Boot, it has been agreed (by QE and RCM?) that we don't need to include all unreleased sources, only those belonging to
some to the most important artifacts, thus the full build configuration for the source is:
sourcesGeneration:
strategy: GENERATE_EXTENDED
whitelistedArtifacts:
- 'tomcat'
- 'ecj'
- 'cxf'
- 'commons-daemon'Assuming the artifacts in the previous step are OK, go to the [product page in PNC]( http://orch.psi.redhat.com/pnc-web/#/product/4/, select the appropriate version (e.g. 1.5), find the milestone you want to close (usually, the highlighted one) and [close the milestone by clicking on the padlock icon](https://docs.engineering.redhat.com/pages/viewpage.action?pageId=44534467. Use the appropriate rcm-guest (full) link for the newly released version for the download URL.
Update Share Content Request (this is no longer done and the team(s) handling the our productized BOM haven't requested it)
Productized dependencies are the result of several project teams across Red Hat. When a project becomes productized, it is useful to track down which productized dependencies are used and which product(s) delivered these dependencies. The release process above generates a shared-content-request.csv file that you can find in the
staging repository under the extras sub-directory for the selected version.
This file gathers information about all productized versions of the dependencies defined by our BOM file and allows us to
update the [Shared Content Request document](https://docs.google.com/spreadsheets/d/103R3SKMD237W4blH2MRrGyGqnCRO2h-7aOPXq7FvehQ/edit#gid=681347717 to track which productized dependencies we use.
[NOTE] Only productized dependencies (i.e. the ones that are gathered in the CSV file) need to be tracked in the Shared Content Request document.
For jars delivered by EAP, we have the information gathered automatically.
For the rest of the artifacts, the CSV provides information
about the author of the build that delivered the dependency,
the brew tags that are applied to the build
and whether the dependency is delivered (released to maven.repository.redhat.com) or not.
The spreadsheet should be divided per product. For products other than EAP, it might be necessary to provide a version of the product that brings the dependency.
Steps to update the shared content request input spreadsheet:
- [optionally] back up the current spread sheet, e.g. in a new tab
- remove the artifacts that are not in the CSV (it means they once were in your product, but are no longer there)
- add any new artifacts from the CSV to the spreadsheet
- if an artifact comes from a released version of EAP (e.g. not ER1), its GAV (Group id / Artifact id / Version) needs to be added to the EAP section of the spreadsheet.
- if an artifact is released, but does not come from EAP:
- check if it's not already in the spreadsheet - if it is keep it in the same product :)
- if the brew tags listed in the CSV document uniquely determine the product, or if you know the product that delivered the artifact, put the artifact in the section dedicated to the product (create it if it does not exist)
- if you have no clue, contact the build author
- if you have trouble you can try asking Petr Sakar (psakar) from the EAP team, who maintains a database called Project Knowledge Base and has information about a lot of products
E.g.
antlr:antlr:pom:2.7.7.redhat-7 EAP,EAP,EAP,EAP,EAP,EAP,EAP,EAP 7.0.1,7.0.2,7.0.3,7.0.4,7.0.5,7.1.0.ER1,7.0.6,7.0.7 false null null null
antlr:antlr:jar:2.7.7.redhat-7 EAP,EAP,EAP,EAP,EAP,EAP,EAP,EAP 7.0.1,7.0.2,7.0.3,7.0.4,7.0.5,7.1.0.ER1,7.0.6,7.0.7 false null null null
org.keycloak:keycloak-adapter-spi:jar:2.5.14.Final-redhat-1 null null true 603228 mkanis sso-7.1-candidate sso-7.1-candidate,rh-sso-7.1.3-rhel-6-released,rh-sso-7.1.3-rhel-7-released
org.keycloak:keycloak-authz-client:pom:2.5.14.Final-redhat-1 null null true 603228 mkanis sso-7.1-candidate sso-7.1-candidate,rh-sso-7.1.3-rhel-6-released,rh-sso-7.1.3-rhel-7-released
Translates to:
EAP
antlr:antlr:pom:2.7.7.redhat-7 EAP,EAP,EAP,EAP,EAP,EAP,EAP,EAP 7.0.1,7.0.2,7.0.3,7.0.4,7.0.5,7.1.0.ER1,7.0.6,7.0.7
antlr:antlr:jar:2.7.7.redhat-7 EAP,EAP,EAP,EAP,EAP,EAP,EAP,EAP 7.0.1,7.0.2,7.0.3,7.0.4,7.0.5,7.1.0.ER1,7.0.6,7.0.7
RHSSO 7.1
org.keycloak:keycloak-adapter-spi:jar:2.5.14.Final-redhat-1 603228 mkanis sso-7.1-candidate sso-7.1-candidate,rh-sso-7.1.3-rhel-6-released,rh-sso-7.1.3-rhel-7-released
org.keycloak:keycloak-authz-client:pom:2.5.14.Final-redhat-1 603228 mkanis sso-7.1-candidate sso-7.1-candidate,rh-sso-7.1.3-rhel-6-released,rh-sso-7.1.3-rhel-7-released
Note, that columns on the right from the GAV of the artifact are optional.
Generate NVR list (this is no longer done and the team(s) handling the our productized BOM haven't requested it)
Unfortunately, the process regarding this list is not quite clear but PM might request this list.
This section describes the instructions to be executed in order to create a product.
If you don't have a product and version, then you have to:
- Create a product in PNC
- Create a version
- create a version entity in PNC
- create a tag
This section lists the projects and documents which are used to productize the Spring projects such as Spring Boot:
-
Document describing the whole
releasingprocess for the Spring Boot BOM and where productization fits in Google Doc -
Graphical depiction of the steps described in the aforementioned document Google Drawing
-
PNC Portals:
- Staging:
http://orch-stage.psi.redhat.com/pnc-web/ - Production:
http://orch.psi.redhat.com/pnc-web/
- Staging:
-
Indy Repository:
- Staging:
http://indy-stage.psi.redhat.com/ - Production:
http://indy.psi.redhat.com/
- Staging:
-
Brew maven repository :
http://brewweb.devel.redhat.com/brewroot/repos/rhoar-1.0-maven-build/latest/maven/ -
Maven Build repositories:
- Staging :
http://rcm-guest.app.eng.bos.redhat.com/rcm-guest/staging/rhoar/spring-boot/ - Candidates :
http://download.eng.brq.redhat.com/devel/candidates/rhoar/spring-boot/
- Staging :
-
Spring projects:
- Snowdrop BOM:
- Upstream - github repository:
[email protected]:snowdrop/spring-boot-bom.git - Downstream - git repository :
ssh://<kerberos login>@code.engineering.redhat.com:22/spring-boot/bom-parent
- Upstream - github repository:
- Snowdrop BOM: