Skip to content

Instantly share code, notes, and snippets.

@djvdorp

djvdorp/README.md

Forked from actionjack/README.md
Created August 30, 2016 08:03
Show Gist options
  • Save djvdorp/651c3a8b38fdf9f7776964fd55ab9136 to your computer and use it in GitHub Desktop.
Save djvdorp/651c3a8b38fdf9f7776964fd55ab9136 to your computer and use it in GitHub Desktop.
Download ZIP
How to Review Changes between Cloud Foundry Releases
Raw
README.md

How to Review Changes between Cloud Foundry Releases

This gist decribes a recommended way to review the differences between Cloud Foundry releases so they can be applied to our environments.

There are multiple ways listed to check the differences but typically you want to

  • Use git diff to see what has changed between release branches
  • Check the spec files to see the default options of the items that have been introduced or changed
  • Generate two Cloud Foundry manifests from each release and compare the two using spiff diff
  • Deploy the change to a test environment (I recommend you add all new properties even if we are not using them yet)

Reviewing the Cloud Foundry Release notes

To get a quick overview of the changes what are going to be introduced into a release, it is a good idea to review the Cloud Foundry Release Notes.

Create a method to diff between two manifests

The purpose section is to describe a method of generating a diff between two upstream Cloud Foundry manifest files

Using Git to diff between two Cloud Foundry releases

$ cd ~/workspace
$ git clone https://github.com/cloudfoundry/cf-release.git
$ cd cf-release
$ scripts/update #[...] have a coffee
$ git diff --exit-code -U5 --patience v224...v225 templates
diff --git a/templates/cf-jobs.yml b/templates/cf-jobs.yml
index ff4daf7..ebc624d 100644
--- a/templates/cf-jobs.yml
+++ b/templates/cf-jobs.yml
@@ -652,10 +652,12 @@ properties:
       password: (( .properties.nats.password ))
 
   dea_next:
     memory_overcommit_factor: null
     disk_overcommit_factor: null
+    instance_bandwidth_limit: (( merge || nil ))
+    staging_bandwidth_limit: (( merge || nil ))
     memory_mb: (( merge || nil ))
     disk_mb: (( merge || nil ))
     staging_disk_inode_limit: 200000
     instance_disk_inode_limit: 200000
     deny_networks: (( merge || [] ))
$ echo $?
1
  • 2 new properties instance_bandwidth_limit and staging_bandwidth_limit in the cf-jobs.yml file. Searching in the cf-jobs.yml file shows this is in the properties.dea_next section
  • The default value is nil so we could deploy CF version 225 with our current manifests without adding in. BUT we should add these variables to our manifests anyway, even if set to nil because they may be used in later releases.
  • the option --exit-code makes the command return 0 when there is no difference and 1 when there is a difference. This can help to automate the process.

For more details on the new variables check the spec file in jobs/dea_next/spec:

  dea_next.staging_bandwidth_limit.rate:
    description: "Network bandwidth limit for staging tasks in bytes per second"
  dea_next.staging_bandwidth_limit.burst:
    description: "Network bandwidth burst limit for staging tasks in bytes"
  dea_next.instance_bandwidth_limit.rate:
    description: "Network bandwidth limit for running instances in bytes per second"
  dea_next.instance_bandwidth_limit.burst:
    description: "Network bandwidth burst limit for running instances in bytes"

We can see that instance_bandwidth_limit and staging_bandwidth_limit are both hashes.

They are used in jobs/dea_next/templates/dea.yml.erb:

staging:
  enabled: true
  max_staging_duration: <%= p("dea_next.max_staging_duration") %>
  memory_limit_mb: <%= p("dea_next.staging_memory_limit_mb") %>
  disk_limit_mb: <%= p("dea_next.staging_disk_limit_mb") %>
  cpu_limit_shares: <%= p("dea_next.staging_cpu_limit_shares") %>
  disk_inode_limit: <%= p("dea_next.staging_disk_inode_limit") %>
<% if_p("dea_next.staging_bandwidth_limit.rate", "dea_next.staging_bandwidth_limit.burst") do |rate, burst| %>
  bandwidth_limit:
    rate: <%= rate %>
    burst: <%= burst %>
<% end %>

instance:
  memory_to_cpu_share_ratio: <%= p("dea_next.instance_memory_to_cpu_share_ratio") %>
  min_cpu_share_limit: <%= p("dea_next.instance_min_cpu_share_limit") %>
  max_cpu_share_limit: <%= p("dea_next.instance_max_cpu_share_limit") %>
  disk_inode_limit: <%= p("dea_next.instance_disk_inode_limit") %>
<% if_p("dea_next.instance_bandwidth_limit.rate", "dea_next.instance_bandwidth_limit.burst") do |rate, burst| %>
  bandwidth_limit:
    rate: <%= rate %>
    burst: <%= burst %>
<% end %>

Generate two Cloud Foundry manifests from each release and compare the two

From Hector Proof of Concept gist

$ scripts/generate_manifest.sh ~/workspace/cf-release v224
HEAD is now at 55e40e2... Final release 224
2015/11/19 11:56:30 error generating manifest: unresolved nodes:
    (( merge )) in ~/workspace/cf-release/templates/cf-properties.yml properties.uaa.clients.cc_routing.secret    (properties.uaa.clients.cc_routing.secret)

This shows we need to add properties.uaa.clients.cc_routing.secret property to our stub:

properties:
 ...
  uaa:
 ...
    clients:
 ...
      gorouter:
        secret: (( secrets.uaa_clients_gorouter_secret ))
+     cc_routing:
+       secret: (( secrets.uaa_clients_cc_routing_secret ))

We also need to add secrets.uaa_clients_cc_routing_secret since we have just defined it:

secrets:
 ...
  uaa_clients_firehose_password: 123abc_uaa_clients_firehose_password
+ uaa_clients_cc_routing_secret: 123abc_uaa_clients_cc_routing_secret
 ...

Try it again to generate both manifests:

$ scripts/generate_manifest.sh ~/workspace/cf-release v224
$ scripts/generate_manifest.sh ~/workspace/cf-release v225

Using the spiff command on the manifests to see that the changes are what are expected

$ spiff diff v224.yml v225.yml
Difference in properties.dea_next.instance_bandwidth_limit
  v225.yml has:
    null
    
Difference in properties.dea_next.staging_bandwidth_limit
  v225.yml has:
    null
    
Difference in compilation.cloud_properties.instance_type
  v224.yml has:
    c3.large
    
Difference in compilation.cloud_properties.availability_zone
  v224.yml has:
    __stub__eu-west-1a
    
  v225.yml has:
    __stub__eu-west-1b
    
Difference in compilation.network
  v225.yml has:
    cf1
    
Difference in jobs.consul_z1.properties.consul.agent
  v224.yml has:
    mode: server
    
  v225.yml has:
    null
    
Difference in jobs.consul_z1.networks.cf1.static_ips.[2]
  v224.yml has:
    10.0.10.39
    
Difference in jobs.consul_z1.networks.cf1.static_ips.[3]
  v224.yml has:
    10.0.10.40
    
Difference in jobs.consul_z1.networks.cf1.static_ips.[4]
  v224.yml has:
    10.0.10.41

Using the Diff command on the manifests to see that the changes are what are expected

$ diff -U 5  scripts/v224.yml scripts/v225.yml
--- scripts/v224.yml    2015-11-19 12:07:19.000000000 +0000
+++ scripts/v225.yml    2015-11-19 12:17:06.000000000 +0000
@@ -1014,17 +1014,19 @@
     directory_server_protocol: https
     disk_mb: 10240
     disk_overcommit_factor: null
     evacuation_bail_out_time_in_seconds: 600
     heartbeat_interval_in_seconds: 10
+    instance_bandwidth_limit: null
     instance_disk_inode_limit: 200000
     kernel_network_tuning_enabled: true
     logging_level: debug
     memory_mb: 4096
     memory_overcommit_factor: null
     mtu: null
     rlimit_core: 0
+    staging_bandwidth_limit: null
     staging_disk_inode_limit: 200000
     staging_disk_limit_mb: 6144
     staging_memory_limit_mb: 1024
   description: null
   disk_quota_enabled: true

Diff Example Code using Ruby

We have a some ruby example code showing how 2 Cloud Foundry manifests can be compared here this gist shows the path of the properties rather than just a raw diff.

Deploying the change

This is where it is really important to have already checked the spec file. If the default is set to nil it will inherit the default settings from the spec file. The spec file's default properties that may make changes to the bosh deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment