Firewall setup for 0L
===

[ToC]

I am not a certified security expert or anything alike. Nevertheless in this document I want to share how I configured the firewall on my 0L validator.

The following is for Ubuntu 20.04 and ufw.

## General thoughts

Some weeks after having set up my 0L validaor and everything running smoothly I was a bit concerned from what I detected in the audit log (``/var/log/auth.log``): massive attempts to log in to my root account or other (non existing) users's accounts. I have to admit, I did not expect this, as neither my server's address was published anywhere nor did I have a domain name assigned to it. Seems like my server's IP address had just been found by trial and error and now my sshd is under fire.

That's how the result of ``grep 'Failed password for invalid user' /var/log/auth.log`` may look like:

Nov  2 02:15:33 vmd87116 sshd[4159438]: Failed password for invalid user ubnt from 91.240.118.172 port 8212 ssh2
Nov  2 02:15:37 vmd87116 sshd[4159448]: Failed password for invalid user ubnt from 91.240.118.172 port 12068 ssh2
Nov  2 02:15:41 vmd87116 sshd[4159450]: Failed password for invalid user ubnt from 91.240.118.172 port 15437 ssh2
Nov  2 02:19:08 vmd87116 sshd[4159592]: Failed password for invalid user redis from 185.187.169.111 port 49750 ssh2
Nov  2 02:26:41 vmd87116 sshd[4159890]: Failed password for invalid user redis from 185.187.169.111 port 34080 ssh2
Nov  2 02:30:45 vmd87116 sshd[4160365]: Failed password for invalid user redis from 185.187.169.111 port 40364 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160370]: Failed password for invalid user apc from 35.232.218.36 port 57254 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160374]: Failed password for invalid user eurek from 35.232.218.36 port 57270 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160378]: Failed password for invalid user fwupgrade from 35.232.218.36 port 57164 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160367]: Failed password for invalid user cloudera from 35.232.218.36 port 57232 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160371]: Failed password for invalid user leo from 35.232.218.36 port 57134 ssh2
Nov  2 02:31:13 vmd87116 sshd[4160377]: Failed password for invalid user netscreen from 35.232.218.36 port 57276 ssh2
Nov  2 02:31:15 vmd87116 sshd[4160373]: Failed password for invalid user device from 35.232.218.36 port 57268 ssh2

So it's time to think about how to protect the server against login attempts and what else might need protection.

## Constraints

As at home I get a changing IP each day, it is not trivial to configure the firewall in a way that I don't lock myself out. Which would be the worst case and thus it is priority 1 that I do not loose access to my server.

## What else to protect

In my opinion only standard software is prone to exploits whereas our 0L services are not (yet) too much known to expect explicit attacks there. Given that, on a plain Ubuntu server used as 0L validator or validator full node, sshd should be the only standard software running which needs special attention. Sure it should be avoided to leave more ports open as needed.

## Strategy

Given that, the setup should:
- allow access to 0L ports for everyone
- allow access from the graphana server
- block access to sshd from "bad address ranges"
- deny everything else

## ufw setup

Step 1: ==make a snapshot of your VM so in case you mess things up, you can easily restore to the state of before the ufw changes!==

Next steps:

### Cleaning previous setup (optional)

If you already entered some rules and want to redo it completely from scratch, you might clear all previously installed rules by the following commands:

ufw disable
ufw reset

### Allow access to 0L ports

First let's open 0L ports:

This is the variant for validators:

ufw allow 6178 # (normally only for full nodes, but useful if you want to run this node as non public full node temporarily)
ufw allow 6179
ufw allow 6180
ufw allow 3030

This is the variant for full nodes (VFN):

ufw allow 6178
ufw allow 6179
ufw allow 8080
ufw allow 3030  # in case you have a web monitor in the VFN

### Allow access for graphana

Probably you have (you should have!) the node exporter running. Those rules allow access for the graphana server to fetch the statistics:

ufw allow from 85.215.101.127 to any port 9100
ufw allow from 85.215.101.127 to any port 9101
ufw allow from 85.215.101.127 to any port 9102

### Protect sshd (at least a bit)

I recommend blocking some address ranges and also use the "limit ssh" feature of ufw, which makes subsequent login attempts from the same origin slower:

ufw allow 22
ufw limit ssh
ufw insert 1 deny from 218.92.0.0/16 to any port 22
ufw insert 1 deny from 222.181.0.0/16 to any port 22
ufw insert 1 deny from 222.186.0.0/16 to any port 22
ufw insert 1 deny from 222.187.0.0/16 to any port 22
ufw insert 1 deny from 222.131.0.0/16 to any port 22
ufw insert 1 deny from 221.181.0.0/16 to any port 22
ufw insert 1 deny from 221.186.0.0/16 to any port 22
ufw insert 1 deny from 221.187.0.0/16 to any port 22
ufw insert 1 deny from 221.131.0.0/16 to any port 22
ufw insert 1 deny from 176.111.0.0/16 to any port 22
ufw insert 1 deny from 115.236.0.0/16 to any port 22
ufw insert 1 deny from 201.166.0.0/16 to any port 22

And basically you should not allow password login for root and any other user you might have enabled. Please use only key based login. See for example here: https://serverpilot.io/docs/how-to-use-ssh-public-key-authentication/

### Enabling ufw

Now after all the rules are added, ufw can be enabled with the default setting to block anything else which had not been specified:

ufw default allow outgoing
ufw default deny incoming
ufw enable

Please check, if the rules are activated:

ufw status

The result should look like this:

Status: active

To                         Action      From
--                         ------      ----
22                         DENY        201.166.0.0/16            
22                         DENY        115.236.0.0/16            
22                         DENY        176.111.0.0/16            
22                         DENY        221.131.0.0/16            
22                         DENY        221.187.0.0/16            
22                         DENY        221.186.0.0/16            
22                         DENY        221.181.0.0/16            
22                         DENY        222.131.0.0/16            
22                         DENY        222.187.0.0/16            
22                         DENY        222.186.0.0/16            
22                         DENY        222.181.0.0/16            
22                         DENY        218.92.0.0/16             
6178                       ALLOW       Anywhere                  
6179                       ALLOW       Anywhere                  
6180                       ALLOW       Anywhere                  
3030                       ALLOW       Anywhere                  
9100                       ALLOW       85.215.101.127            
9101                       ALLOW       85.215.101.127            
9102                       ALLOW       85.215.101.127            
22                         ALLOW       Anywhere                  
22/tcp                     LIMIT       Anywhere                  
6178 (v6)                  ALLOW       Anywhere (v6)             
6179 (v6)                  ALLOW       Anywhere (v6)             
6180 (v6)                  ALLOW       Anywhere (v6)             
3030 (v6)                  ALLOW       Anywhere (v6)             
22 (v6)                    ALLOW       Anywhere (v6)             
22/tcp (v6)                LIMIT       Anywhere (v6)             

...and to be sure, everything will also work fine also after the next reboot, I would do a reboot now.

## Regular monitoring to find out more addresses to block

It seems like a good idea to check the logfile ``/var/log/auth.log`` on a regular basis to take additional measures i.e. to block more addresses. I am using the following commands to find the top-10 address ranges, from which many ssh attacks are originating from:

grep 'Invalid user' /var/log/auth.log |awk -Ffrom '{print $NF}'|awk '{print $1}'|awk -F. '{print $1"."$2".0.0"}'| sort |uniq -c|sort -nr|head -10

Example output:

   2417 15.235.0.0
    259 43.143.0.0
    223 185.187.0.0
     80 157.230.0.0
     73 43.153.0.0
     67 81.17.0.0
     65 206.189.0.0
     64 179.60.0.0
     62 165.22.0.0
     62 157.245.0.0

In this case, I would additionally block at least the first 3 address ranges. But before doing so, I recommend to quickly check those address ranges e.g. with https://ipinfo.io/ that if you block those, you would not lock out yourself! After having done this, they can be added to the firewall rules - remember to use "insert 1" to be sure those rules are processed before the general "allow 22" rule:

ufw insert 1 deny from 15.235.0.0/16 to any port 22
ufw insert 1 deny from 43.143.0.0/16 to any port 22
ufw insert 1 deny from 185.187.0.0/16 to any port 22