nginx101.md

Overview

How to setup DigitalOcean's Droplet with Ubuntu 16.04 + UFW + NGINX + SSL + F2B

Table of Contents

1. SSH
2. Ubuntu server setup
3. Nginx installation and server blocks
4. Issue SSL with acme.sh
5. F2B (Fail2Ban)
   5.1. Installation
   5.2. Configuration
   5.3. Activating your nginx jails
   5.4. Getting info about enabled jails
   5.5. Testing Fail2Ban Policies
6. Nginx-http-auth

1. SSH

References:

Secure Ubuntu server for non-root user using only SSH keys
ssh-keygen best practices

$ ssh-keygen -t ed25519 -a 100

OR

$ ssh-keygen -t rsa -b 4096 -o -a 100

2. Ubuntu server setup

References:

Initial Server Setup with Ubuntu 16.04

$ sudo ufw app list //List app from UFW

Available applications:
  OpenSSH
  
$ sudo ufw allow OpenSSH //Allow the app in UFW
$ sudo ufw enable //Enable the Firewall
$ sudo ufw status //Check UFW status

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)

3. Nginx installation and server blocks

References:

How To Install Nginx on Ubuntu 16.04
How To Set Up Nginx Server Blocks (Virtual Hosts) on Ubuntu 16.04

$ sudo apt-get update
$ sudo apt-get install nginx
$ sudo ufw allow 'Nginx Full'
$ sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
Nginx Full                 ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)             
Nginx Full (v6)            ALLOW       Anywhere (v6)

$ systemctl status nginx

Visit the server's IP address or domain. You should see the default Nginx landing page.

By default, Nginx on Ubuntu 16.04 has one server block enabled. It is configured to serve documents out of a directory at /var/www/html.
We need to create these directories for each of our sites. The -p flag tells mkdir to create any necessary parent directories along the way:

$ sudo mkdir -p /var/www/example.com/html
$ sudo mkdir -p /var/www/test.com/html

Now that we have our directories, we will reassign ownership of the web directories to our normal user account. This will let us write to them without sudo.

$ sudo chown -R $USER:$USER /var/www/example.com/html
$ sudo chown -R $USER:$USER /var/www/test.com/html

Now that we have our directory structure set up, let's create a default page for each of our sites so that we will have something to display.

$ nano /var/www/example.com/html/index.html
$ nano /var/www/test.com/html/index.html

Create server blocks

$ sudo cp /etc/nginx/sites-available/default /etc/nginx/sites-available/example.com
$ sudo nano /etc/nginx/sites-available/example.com

server {
        listen 80;
        listen [::]:80;

        root /var/www/example.com/html;
        index index.html index.htm index.nginx-debian.html;

        server_name example.com;

        location / {
                try_files $uri $uri/ =404;
        }
}

$ sudo nano /etc/nginx/sites-available/test.com

server {
        listen 80;
        listen [::]:80;

        root /var/www/test.com/html;
        index index.html index.htm index.nginx-debian.html;

        server_name test.com;

        location / {
                try_files $uri $uri/ =404;
        }
}

Create symlink

$ sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
$ sudo ln -s /etc/nginx/sites-available/test.com /etc/nginx/sites-enabled/

In order to avoid a possible hash bucket memory problem that can arise from adding additional server names, we will go ahead and adjust a single value within our /etc/nginx/nginx.conf file. Open the file now:

$ sudo nano /etc/nginx/nginx.conf

http {
    . . .

    server_names_hash_bucket_size 64;

    . . .
}

Next, test to make sure that there are no syntax errors in any of your Nginx files:

$ sudo nginx -t

If no problems were found, restart Nginx to enable your changes:

$ sudo systemctl restart nginx

4. Issue SSL with acme.sh

References:

acme.sh client for Let's Encrypt
acme.sh DNS Manual mode

1. Create an API token in DigitalOcean here
2. Export the API token

$ export DO_API_KEY="$mytoken"

3. Issue the cert. Single quotes '' are important for wildcard domains.

$ acme.sh --issue --dns dns_dgon \
    -d example.com -d '*.example.com'  \
    -d example2.com -d '*.example2.com'

OR issue them manually

$ acme.sh --issue -d example.com --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please

# Add the TXT to your DNS records

$ acme.sh --renew -d example.com --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please

4. Install the cert (preferred path is /etc/nginx/ssl/example.com/)

$ acme.sh --install-cert \
    -d example.com -d '*.example.com'  \
    -d example2.com -d '*.example2.com'
    --key-file /path/to/save/the/key/file/key.pem \
    --fullchain-file /path/to/save/the/fullchain/file/cert.pem

5. Create a ssl.conf file (preferred path is /etc/nginx/ssl/example.com/) with the following content

ssl_certificate /path/to/save/the/fullchain/file/cert.pem;
ssl_certificate_key /path/to/save/the/key/file/key.pem;

6. Edit server block file from

server {
  listen 80;
  listen [::]:80;
  
  . . .
  
}

to

server {
  listen 443 ssl;
  listen [::]:443 ssl;
  
  . . .
  
  include /path/to/ssl.conf;
}

5. F2B (Fail2Ban)

References:

Fail2ban - Community Help Wiki
How To Protect an Nginx Server with Fail2Ban on Ubuntu 14.04

5.1. Installation

$ sudo apt-get update
$ sudo apt-get install fail2ban

5.2. Configuration

Make a copy of jail.conf as jail.local and ONLY edit jail.local

$ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
$ sudo nano /etc/fail2ban/jail.local

5.2.1. Edit jail.local

Email notification

destemail = [email protected]
sendername = Fail2Ban
mta = sendmail
. . .
action = %(action_mwl)s

Jail configuration

[ssh]

enabled = true
port    = ssh
filter  = sshd
logpath  = /var/log/auth.log
maxretry = 3

To enable log monitoring for Nginx login attempts, we will enable the [nginx-http-auth] jail. Edit the enabled directive within this section so that it reads "true":

[nginx-http-auth]

enabled  = true
filter   = nginx-http-auth
port     = http,https
logpath  = /var/log/nginx/error.log

If you do not use PHP or any other language in conjunction with your web server, you can add this jail to ban those who request these types of resources

[nginx-noscript]

enabled  = true
port     = http,https
filter   = nginx-noscript
logpath  = /var/log/nginx/access.log
maxretry = 6

We can add a section called [nginx-badbots] to stop some known malicious bot request patterns:

[nginx-badbots]

enabled  = true
port     = http,https
filter   = nginx-badbots
logpath  = /var/log/nginx/access.log
maxretry = 2

If you do not use Nginx to provide access to web content within users' home directories, you can ban users who request these resources by adding an [nginx-nohome] jail:

[nginx-nohome]

enabled  = true
port     = http,https
filter   = nginx-nohome
logpath  = /var/log/nginx/access.log
maxretry = 2

We should ban clients attempting to use our Nginx server as an open proxy. We can add an [nginx-noproxy] jail to match these requests:

[nginx-noproxy]

enabled  = true
port     = http,https
filter   = nginx-noproxy
logpath  = /var/log/nginx/access.log
maxretry = 2

5.2.2. Adding the filters for additional nginx jails

cd into the filters directory

$ cd /etc/fail2ban/filter.d

---

We actually want to start by adjusting the pre-supplied [nginx-http-auth] filter to match an additional failed login log pattern. Open the file for editing:

$ sudo nano nginx-http-auth.conf

Below the failregex specification:

[Definition]

failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$

ignoreregex =

Add an additional pattern. This will match lines where the user has entered no username or password:

^ \[error\] \d+#\d+: \*\d+ no user/password was provided for basic authentication, client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$

---

Next, we can copy the apache-badbots.conf file to use with [nginx-badbots] jail. We can use this file as-is, but we will copy it to a new name for clarity. This matches how we referenced the filter within the jail configuration:

$ sudo cp apache-badbots.conf nginx-badbots.conf

---

Next, we'll create a filter for our [nginx-noscript] jail:

$ sudo nano nginx-noscript.conf

[Definition]

failregex = ^<HOST> -.*GET.*(\.php|\.asp|\.exe|\.pl|\.cgi|\.scgi)

ignoreregex =

---

Next, create a filter for the [nginx-nohome] jail:

$ sudo nano nginx-nohome.conf

[Definition]

failregex = ^<HOST> -.*GET .*/~.*

ignoreregex =

---

Finally, we can create the filter for the [nginx-noproxy] jail:

$ sudo nano nginx-noproxy.conf

[Definition]

failregex = ^<HOST> -.*GET http.*

ignoreregex =

5.3. Activating your nginx jails

$ sudo service fail2ban restart

5.4. Getting info about enabled jails

$ sudo fail2ban-client status

You should see a list of all of the jails you enabled:

Status
|- Number of jail:      6
`- Jail list:           nginx-noproxy, nginx-noscript, nginx-nohome, nginx-http-auth, nginx-badbots, ssh

You can look at iptables to see that fail2ban has modified your firewall rules to create a framework for banning clients. Even with no previous firewall rules, you would now have a framework enabled that allows fail2ban to selectively ban clients by adding them to purpose-built chains:

$ sudo iptables -S

If you want to see the details of the bans being enforced by any one jail, it is probably easier to use the fail2ban-client again:

$ sudo fail2ban-client status nginx-http-auth

Status for the jail: nginx-http-auth
|- filter
|  |- File list:        /var/log/nginx/error.log 
|  |- Currently failed: 0
|  `- Total failed:     0
`- action
   |- Currently banned: 0
   |  `- IP list:
   `- Total banned:     0

5.5. Testing Fail2Ban Policies

It is important to test your fail2ban policies to ensure they block traffic as expected. For instance, for the Nginx authentication prompt, you can give incorrect credentials a number of times. After you have surpassed the limit, you should be banned and unable to access the site. If you set up email notifications, you should see messages regarding the ban in the email account you provided.
If you look at the status with the fail2ban-client command, you will see your IP address being banned from the site:

$ sudo fail2ban-client status nginx-http-auth

Status for the jail: nginx-http-auth
|- filter
|  |- File list:        /var/log/nginx/error.log 
|  |- Currently failed: 0
|  `- Total failed:     12
`- action
   |- Currently banned: 1
   |  `- IP list:       111.111.111.111
   `- Total banned:     1

When you are satisfied that your rules are working, you can manually un-ban your IP address with the fail2ban-client by typing:

$ sudo fail2ban-client set nginx-http-auth unbanip 111.111.111.111

6. Nginx-http-auth

https://www.youtube.com/watch?v=S9XeKva3AuU

sudo apt install apache2-utils
sudo htpasswd -c /etc/nginx/.htpassw USERNAME

auth_basic "Restricted Content";
auth_basic_user_file /etc/nginx/.htpassw;

service nginx reload