Some notes on the problems I've had upgrading Chef on jenkins.ed and cronalicious.ed.

Why upgrade?

First...why do we need to upgrade?

You might not need to for other cookbooks, but for Apollo there's a problem with the mongo cookbook when using Chef < 10.26.0, which looks like this:

================================================================================
Error executing action `add` on resource 'apt_repository[10gen]'
================================================================================

RuntimeError
------------
The repository file to create is nil, cannot continue.

.
.
.

There's nothing wrong with the repo...it just seems to be a Chef bug, and upgrading to 10.26.0 fixes it.

However, upgrading beyond 10.26.0 introduces other problems when deploying, so after some experimentation I'd suggest that we stick with this version for now.

How to upgrade?

The easiest way to do it is to use the knife bootstrap command. If you ensure that you're in the kitchen directory of the deploy repo then the configuration options are set properly to the right version, and the necessary keys and user name are all set, too. The command to set jenkins.ed to the correct Chef version would be:

knife bootstrap --sudo jenkins.ed

(The 'use_sudo' option is also set in the configuration options but it doesn't seem to be taking effect, so set it manually here.)

However...watch out!

If you are using Chef 11 locally then you'll end up installing a bunch of options that don't work with the 10.26 version when it is installed on the server. (I have no idea why the bootstrap command is provisioning partly from a remote repo, and partly from your local installation, but there you are.) So to avoid any problems make sure you have 10.26 locally before running the command. You can check the version with:

knife --version

If the version you're seeing is not 10.26, but you do have it installed then you can remove other versions with the following command:

sudo gem remove chef

This will give you a list of versions you have, and allow you to pick versions to remove.

And if you haven't got 10.26 installed then you can install it like this:

sudo gem install chef -v 10.26.0 --no-ri --no-rdoc

Problems

I had a variety of problems during the course of doing all of this, and although they may not show up again if you use knife bootstrap and get your versions aligned, I'll still list them here in case they pop up again.

ruby-dev is needed

Do this before the gem install chef step above:

sudo apt-get install ruby-dev

gem is not found

If gem is not present you'll need to do:

sudo apt-get install rubygems

Error message that refers to 'net/http' problems

I was sometimes getting this error:

/usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- net/https (LoadError)
        from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from /var/lib/gems/1.8/gems/chef-10.26.0/bin/../lib/chef/rest.rb:24
        .
        .
        .

The solution was:

sudo apt-get install libopenssl-ruby

bootstrap works but then 'chef-client' is not found

At one point, even though the installation had worked fine I was still completely unable to run chef-client or knife from the command line. In the end I had to create the symlinks manually:

sudo ln -s /var/lib/gems/1.8/gems/chef-10.26.0/bin/knife /usr/local/bin
sudo ln -s /var/lib/gems/1.8/gems/chef-10.26.0/bin/chef-client /usr/local/bin
sudo ln -s /var/lib/gems/1.8/gems/chef-10.26.0/bin/chef-solo /usr/local/bin

markbirbeck/gist:5635086