With git you can have anything versioned. You're used to version your code, which is a bunch of files. Your server configuration (on Linux) is also just a bunch of files, so it can be versioned as well.
The idea is simple: create a git repository in /etc/ and commit everytime you change any configuration of your server. Written in code:
cd /etc
git init
git add .
git commit -m 'fresh server installation'
apt-get install postfix
git add .
git commit -m 'installed postfix'This gets very useful, because now you not only know what has been changed, but also when it happened, who did it and why -- it's written in the commit message.
You don't need to set a remote for this repository, nor push it anywhere else. It's all on the single server, and it is enough. Remember that with git you don't need an external server with the repository.
Don't forget, that everytime you change something, you have to commit it. And all your collegues need to do it as well. And because we know our collegues, it's always good to check /etc for any non-commited changes before you start changing anything. Just in case they forgot :)
The reason, why something has been changed, is often very important. Thus it is very important to write proper commit messages. You write proper commit messages in your normal project, so it shouldn't be a problem, right?
Let's consider this example: we have one account for the whole company to log into to check email sent from the server, and we need to change it's password. Many people I know would commit it with message password. But when someone else did this and you're reading such commit message, do you have any idea what change was made? Nope.
Let's try to make it better: changed password. Still not right.
Describe what was really changed, like you were telling it to someone else: changed password of user abc. Better, but still not right.
Write down also the reason why the change was made, or attach any details someone may ask you:
changed password of user abc
generated new password, because it was given to a customer for testing
the new password is stored in Password Manager under Server X -> mailserver
That's a proper commit message. It's clear what happened, why it happened and where to find more details. Everything's clear and nobody will waste their time in the future trying to understand your changes.
Tip: If you were following an installation guide on the web, you may like to find it again when you need it. It's good to write the link into the commit message.
When installing new programs, you should make two commits: first just after you installed the program, and the second after you've configured it.
The first commit usually containts lots of files with the default configuration brought by the new program. It is not really important - it has the default parameters, documentation comments etc. And it's always the same, you can get it for free the next time you apt-get install.
The most important part of new installed programs is how you configure them compared to defaults. That's the know-how you want to preserve.
After some months, when you'll be installing another server, you'll have newer versions of all programs, with slightly different default configuration. And you'll need to know the set of changes you did before. Therefore the second commits becomes handy.
A small notice: you don't need to do only two commits. Make more small commits if it's possible, e.g.:
- commit 1 - installed postfix
- commit 2 - configured postfix for local users
- commit 3 - configured forwarding of special email addresses
- commit 4 - configured imap & pop3 access
- commit 5 - enabled & configured ssl/tls
Again, your future self will be happy, because he'll have a detailed manual how to configure the new future server.
Because /etc can contain sensitive information, like password hashes or private keys, it needs to be hidden from public. Git, by default, makes the repository readable by everyone (drwxr--r--). You shall make it accessible only by root:
chmod 0700 .gitBe also careful when you push this repository to another server, e.g. GitHub. Everyone will be able to read the sensitive information.
Not all useful configuration is held in /etc, you often have some config files under your HOME directory. Or, you can have programs installed in /usr/local and thus their config in /usr/local/apache2/conf. These config files don't get into git.
I have a special directory /etc/_other, where I copy all these bits of configuration and commit it together with /etc. You can prepare a bash script /etc/_other/update.sh, which will do all the copying for you. And possibly, set it as pre-commit hook, so that you don't forget it.
Example script can be found below this post.
Because your colleagues are not that familiar with versioning server configuration, they may forget to commit after they have changed something. Or they may forget to copy all those external bits of configuration.
Or, because you're a colleague of them, you may sometimes forget it as well (happened to me several times).
Good solution is to set up a cron, that will do git status every night and email you, if there are un-commited changes. If you get such email, you can find out what happened and who forgot to commit. And you can fix it early, before too many configuration changes happened.
There are some files in /etc which describe current state of the server, thus they change pretty often.
Also, there are some compiled files or backups.
You don't need to have these versioned. I recommend setting up a .gitignore file -- one can be found below this post.
Having the configuration versioned helped me many times. Mostly when I needed to install a new server, I exactly knew what to configure and where, and I had links to articles explaining the steps. Also, when something got broken, I was able to track down the changes which caused it and fix them.
I'm happy that all my /etc's are versioned. Yours can be too.
How do you keep your configuration tidy and organised? Do you have more tips? Please provide me with comments, or fork and update this post.