The list of actions listed below was taken mostly from Book Of Zeus with minor modifications and did the job well for Ubuntu version, which was available at that moment (May 2016). This gist was created for internal use and was never meant to be discovered by the web, although Google managed to find and index this page, which was a great surprise for me. Please check the original source for the updated information (links are provided in most of the sections), and read the comments below: they provide more details about the usage experience.
http://bookofzeus.com/harden-ubuntu/initial-setup/system-updates/
Keeping the system updated is vital before starting anything on your system. This will prevent people to use known vulnerabilities to enter in your system.