Communicating to private github repo in a chef recipe.

This gist file is basic documentation that I used to communicate with github private repositories.

Connecting to a private git repository from a chef recipe. Create a directory called tmp-ssh and in it generate a passphraseless ssh key.

  mkdir tmp-ssh

  cd tmp-ssh

Creates a 4096 bit rsa key in the file called "id_rsa" with no passphrase.

  ssh-keygen -t rsa -b 4096 -C "[email protected]" -P "" -f ./id_rsa

Remove the carriage returns in the private key and convert them to the literal characters "\n" When we process this file later on the "\n" characters will again be turned into new lines.

  perl -pi -e 's/\n/\\n/g' id_rsa

Now we create a file called "deploy_key.json" it's contents are as so. Where XXXXXXXXX is the concatinated string found in the private ssh key. It'll be much longer than the XXXXXXX string I have used below.

{
  "id": "deploy_key",
  "key_val": "XXXXXXXXX"
}

This is the file that will represent our encrypted data bag.

Next we want to use knife vault to encrypt the databag "deploy_key" is the name of the ID in the json file above, so it should be changed if. you are adding something new. It will be put in the secret-ssh-private-keys databag, so if you want to use another databag please use a different definition.

  knife vault create secret-ssh-private-keys deploy_key --json ./pathto/deploy_key.json --search '*:*'

knife will create the secret-ssh-private-keys in the data_bags directory inside will be 2 files. The first "deploy_key.json" contains the encrypted version of our deploy_key.json file the second contains keys used to help encrypt and decrypt using the clients private keys. Both now need to be pushed to your data bag on the chef server.

  knife data bag from file secret-ssh-private-keys data_bags/secret-ssh-private-keys/deploy_key.json
  knife data bag from file secret-ssh-private-keys data_bags/secret-ssh-private-keys/deploy_key_keys.json

Now that the chef vault managed encrypted data bags are on the chef server it's time to work on the cookbook that will pull the repository

  chef create cookbook cookbooks/my-app-repo

Create templates for the known_hosts file and the ssh key's public key

  chef generate template cookbooks/my-app-repo known_hosts
  chef generate template cookbooks/my-app-repo id_rsa.pub

copy and paste id_rsa.pub files contents into the id_rsa.erb file located in the cookbook's template file copy the below contents into the known_hosts.erb file

  github.com,192.30.255.112 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==

The above allows you to ssh to [email protected] without having to be asked for the fingerprint from withing your provisioned server.

Below is the recipe used to pull it all together

# Make sure the chef-vault gem is installed at compile time so we can use it.
chef_gem 'chef-vault' do
  compile_time true if respond_to?(:compile_time)
end

# require the chef-vault gem.
require 'chef-vault'

# Get the deploy_key object from the "secret-ssh-private-keys" data gag and decrypt it with the client's private key.
deploy_key = ChefVault::Item.load('secret-ssh-private-keys','deploy_key')

# Make sure the .ssh directory exists. (and make sure it is owned by the appropriate user in this case "vagrant")
directory '/home/vagrant/.ssh' do
  owner 'vagrant'
  group 'vagrant'
  mode '700'
  action :create
end

# create the known_hosts file from the template file.
template '/home/vagrant/.ssh/known_hosts' do
  source 'known_hosts.erb'
  owner 'vagrant'
  group 'vagrant'
  mode '0644'
end

# create the ssh public key file from the template file.
template '/home/vagrant/.ssh/id_rsa.pub' do
  source 'id_rsa.pub.erb'
  owner 'vagrant'
  group 'vagrant'
  mode '0644'
end

# create the ssh private key file from the databag
file '/home/vagrant/.ssh/id_rsa' do
  content deploy_key['key_value']
  owner 'vagrant'
  group 'vagrant'
  mode '0600'
end

# Now using git pull down the repository
git '/tmp/yourchefrepo' do
  repository '[email protected]:yourgitusername/yourchefrepo.git'
  revision 'master'
  action :sync
  user 'vagrant'
  group 'vagrant'
end

Lastly take the contents of id_rsa.pub and add it as a deploy key on the repository you specified in your recipe. Now whenever chef converges it'll check to see if the master repository has been updated and if so it will resync it to production and updating your website code.

thenetimp/chef-github.md