# Log all traffic in Ubuntu
###### tags: `NTUToolmenLab`

## Add logger in iptable
about iptable
https://www.hostinger.com/tutorials/iptables-tutorial
https://help.ubuntu.com/community/IptablesHowTo

show it

sudo iptables --line-numbers -L
sudo iptables --line-numbers -L OUTPUT

Add logger

iptables -A OUTPUT -j LOG --log-prefix 'iptable log: ' --log-level 7

or more formally

iptables -N LOGGER
iptables -A LOGGER -j LOG --log-prefix 'iptable log: ' --log-level 7
iptables -A OUTPUT -j LOGGER

You should add at the top of the chain to avoid skip it by another chain before.
`iptables -I OUTPUT -j LOGGER`
`iptables -I INPUT -j LOGGER`

If you want to monitor in `docker`
`iptables -I FORWARD -j LOGGER`

delete it if needed
`iptables -D OUTPUT 1`

Reference
* https://ubuntuforums.org/showthread.php?t=1158091

## Separate from syslog
The concept of syslog
https://www.the-art-of-web.com/system/rsyslog-config/

For ubuntu 1804
`cat /var/log/syslog` or `cat /var/log/kern.log` to see the log

### method 1
Filter the log of iptable logger to specific file

edit
`/etc/rsyslog.d/iptable_logger.conf`

:msg,contains,"iptable log: " /var/log/iptable.log

### method 2(Recommanded)
Don't want to output in `kern.log` and `syslog`

Add below code in the begining of `/etc/rsyslog.d/50-default.conf`

:msg,startswith," iptable log: " /var/log/iptable.log
:msg,startswith," iptable log: " stop

### Better format
`/etc/rsyslog.conf`

module(load="imklog" ParseKernelTimestamp="on" KeepKernelTimestamp="off")
# $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$template mycleanformat,"%TIMESTAMP:::date-rfc3339% %syslogtag% %syslogseverity-text% %msg%\n"
$ActionFileDefaultTemplate mycleanformat

### reload
and finally restart
`systemctl restart syslog`

Reference
* https://askubuntu.com/questions/348439/where-can-i-find-the-iptables-log-file-and-how-can-i-change-its-location
* https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
* https://www.rsyslog.com/doc/v8-stable/rainerscript/expressions.html
* https://unix.stackexchange.com/questions/302972/how-to-stop-rsyslog-output-timestamp
* https://linux.die.net/man/5/rsyslog.conf
* https://www.rsyslog.com/doc/v8-stable/configuration/modules/imklog.html
* https://stackoverflow.com/questions/22755005/rsyslog-property-based-filtering-not-working
* https://rsyslog-5-8-6-doc.neocities.org/rsyslog_conf_templates.html

## Make it smaller
Using logrotated
`/etc/logrotate.d/rsyslog`

/var/log/iptable.log
{
    rotate 99
    size 500M
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

Because logrotate run everyday not every second by default,
create a crontab to run it.
`/etc/cron.d/iptable-log-rotate`

*/10  *  *  *  *   root /usr/sbin/logrotate /etc/logrotate.conf

`sudo systemctl restart cron`

In my mechine, the 100MB log file can be compressed into about 5MB

And same as systemd-journal

edit `/etc/systemd/journald.conf` to set
`SystemMaxUse=1G`

`sudo systemctl restart systemd-journald`

or stop all of journal
`sudo systemctl stop systemd-journald*`

Reference:
* https://www.digitalocean.com/community/tutorials/how-to-manage-logfiles-with-logrotate-on-ubuntu-16-04
* https://linux.die.net/man/8/logrotate
* https://www.digitalocean.com/community/tutorials/how-to-manage-logfiles-with-logrotate-on-ubuntu-16-04
* https://unix.stackexchange.com/questions/139513/how-to-clear-journalctl

## With calico(Brute force)
Force to write it because calico checks its iptable chain every 60 seconds

while true; do
    if ! iptables -L cali-FORWARD | grep -q LOGGER; then
        iptables -I  cali-FORWARD -j LOGGER;
        echo `date` "ADD"
    fi
    sleep 0.1;
done

And set calico checking seconds more longger in calico.yml

- name: FELIX_IPTABLESPOSTWRITECHECKINTERVALSECS
  value: "10"

Reference
* https://docs.projectcalico.org/v3.7/reference/felix/configuration

## CODE
Overall code
{%gist linnil1/26e688644156da7898ba345fc43ec41d %}

## TODO
* Monitor in kubernete with calico-cni in more formal way.