Manually trigger code-manager and file-sync

manual_code_deploy.sh

#!/bin/bash
# GIST_URL: https://gist.github.com/natemccurdy/797fa9128b7eef1f07be
# This script can be run to manually trigger Code Manager to deploy code from your control-repo. This sort of
# thing is neccesary when, for example:
#   - You've turned on Code Manager but have not yet made an RBAC token.
#   - You want to pull down the latest version of a Puppetfile module without pushing to your GMS.
#   - Something has broken the post-receive hook on your GMS that would've triggered Code Manager.
#   - Syntax errors in your Puppetfile prevent you from retrieving those fixes to that Puppetfile.
#   - Puppetserver has crashed due to file-sync issues between code and code-staging.
#   - Code Manager can't deploy your code for various reasons that are hard to track down.

[[ $EUID -eq 0 ]] || { echo "${0##*/} must be run as root or with sudo" >&2; exit 1; }

[[ -f /opt/puppetlabs/server/data/code-manager/r10k.yaml ]] || { echo "Looks like Code Manager isn't even enabled. Why are you running this?" >&2; exit 1; }

echo "==> Disabling the Puppet agent"
/opt/puppetlabs/puppet/bin/puppet agent --disable "Disabled while waiting for code deploy at $(date)"

echo "==> Stopping pe-puppetserver"
/opt/puppetlabs/puppet/bin/puppet resource service pe-puppetserver ensure=stopped

echo "==> Fixing permissions on the code-staging directory"
chown -c -R pe-puppet:pe-puppet /etc/puppetlabs/code-staging

echo "==> Removing Code Manager worker caches"
rm -rf /opt/puppetlabs/server/data/code-manager/worker-caches/deploy-pool-*

echo "==> Removing Code Manager git caches"
rm -rf /opt/puppetlabs/server/data/code-manager/git/*

echo "==> Running r10k manually as pe-puppet to fetch new code"
sudo -u pe-puppet -H bash -c '/opt/puppetlabs/puppet/bin/r10k deploy environment -c /opt/puppetlabs/server/data/code-manager/r10k.yaml -p -v debug'

deploy_result=$?
[[ $deploy_result -eq 0 ]] || { echo -e "\nR10k failed to deploy your code. Check the scroll-back for errors.\n" >&2; exit 1; }

echo "==> Reset the file-sync cache"
rm -rf /opt/puppetlabs/server/data/puppetserver/filesync/storage
rm -rf /opt/puppetlabs/server/data/puppetserver/filesync/client

echo "==> Reset the orchestration services code and data dirs"
rm -rf /opt/puppetlabs/server/data/orchestration-services/data-dir
rm -rf /opt/puppetlabs/server/data/orchestration-services/code

echo "==> Delete environments in the code-dir so file-sync can do its thing"
rm -rf /etc/puppetlabs/code/*

configured_environment="$(/opt/puppetlabs/puppet/bin/puppet config print environment --section master)"
echo "==> Recreating the ${configured_environment:-production} environment directory so puppetserver can start"
mkdir -v -p "/etc/puppetlabs/code/environments/${configured_environment:-production}"

echo "==> Fixing permissions on the code directory"
chown -c -R pe-puppet:pe-puppet /etc/puppetlabs/code

echo "==> Starting pe-puppetserver"
/opt/puppetlabs/puppet/bin/puppet resource service pe-puppetserver ensure=running

# Determine paths to certs.
certname="$(/opt/puppetlabs/puppet/bin/puppet config print --section agent certname)"
certdir="$(/opt/puppetlabs/puppet/bin/puppet config print --section agent certdir)"

# Set variables for the curl.
cert="${certdir}/${certname}.pem"
key="$(/opt/puppetlabs/puppet/bin/puppet config print --section agent privatekeydir)/${certname}.pem"
cacert="${certdir}/ca.pem"

echo "==> Hitting the file-sync commit endpoint at https://$(hostname -f):8140/file-sync/v1/commit"
/opt/puppetlabs/puppet/bin/curl -v -s --request POST --header "Content-Type: application/json" --data '{"commit-all": true}' \
                                --cert "$cert" \
                                --key "$key" \
                                --cacert "$cacert" \
                                "https://$(hostname -f):8140/file-sync/v1/commit" && echo

echo "==> Enabling the Puppet agent"
/opt/puppetlabs/puppet/bin/puppet agent --enable

echo "Done!"

I think -R is missing on line 43, at least I had to run it to fix CM

I think -R is missing on line 43, at least I had to run it to fix CM

Thanks @vchepkov

Was getting a "boost" error when using the /opt/puppetlabs/puppet/bin/puppet agent --configprint certdir commands.

Ended up using: puppet config print --section agent certdir and puppet config print --section agent certname to perform the same functionality

Ended up using: puppet config print --section agent certdir and puppet config print --section agent certname to perform the same functionality

Thanks @logicminds, I've updated the script with those commands.

Need to stop puppet agent as well, since it will bring puppetserver service up

The "Running r10k manually as pe-puppet to fetch new code" step was failing for me with errors like unable to determine current branches for git source 'puppet' until I removed the contents of /opt/puppetlabs/server/data/code-manager/git. Then this script was able to succeed.

Thanks for the info @vchepkov and @kenyon, I've updated the script to:

• Stop the puppet agent then start it after the deploy
• Remove the Code Manager git caches at /opt/puppetlabs/server/data/code-manager/git

@natemccurdy , I wonder if any additional steps needs to be done on replica, if DR configuration is enabled?

FYI, the code needs to be updated to support 'Lockless deployment'