111-2專題研究筆記 Week 13-14

Problem Specification

I tried to use a project from github a few days ago, but it seems like it's code is out of date and is too old to use. So I decided to build a userspace filter program my self. To do so, I need to study deep from libpcap and SO_ATTACH_BPF so that it can fullfill the central concepts of userspace filtering.

The sole concept of userspace filtering is fuzzy. It has the advantages that can attach rules that more complicated than rules in the XDP or kernel space, and they can be varies from one applications to another. But should a packet go through kernel space or not? Thanks to the libpcap project, we can extract the packet out from the kernel early, and send it straight to the user space, otherwise the whole process is kind of slow because it needs to be copied from kernel memory spaces to user memory spaces after the packet go through the whole network stack datapath and which will take a lot of time.

But with DPDK project, the packet will be send straight to user space when it's in driver level, so it didn't have to go into network stack, which saves time and avoid the memory copy process. So which one, with libpcap or with DPDK, should be called the finest way to fulfill userspace filter? I think it's maybe both.

Task Specification

I planned to build a bpf socket program first. With SO_ATTACH_BPF function and libpcap, a socket can dynamically attach or dettach a BPF or eBPF filter on it.

The c program cotains serveral parts.
First, we need to include the header files:

#include <pcap.h>
#include <stdio.h>

After that, we need to declare some variabes:

int main(int argc, char *argv[])
{
	pcap_t *handle;		/* Session handle */
	char *dev;			/* The device to sniff on */
	char errbuf[PCAP_ERRBUF_SIZE];	/* Error string */
	struct bpf_program fp;		/* The compiled filter */
	char filter_exp[] = "port 23";	/* The filter expression */
	bpf_u_int32 mask;		/* Our netmask */
	bpf_u_int32 net;		/* Our IP */
	struct pcap_pkthdr header;	/* The header that pcap gives us */
	const u_char *packet;		/* The actual packet */

The dev string will be the device we use to take packets, which can be assign from pcap_lookupdev function.

handle will be the main handler of the session, it can be initiated by pcap_open_live function.

The fp is the main filter conponent, it can be a BPF program, which will be the one who decide if a packet can be accept or drop.Along with the filter_exp[] string which specific the filter rules, libpcap will complie fp and choose the packet it want to sniff.

For example, if we say:

char filter_exp[] = "port 23 and not ip host 192.168.69.9";

The program will only sniff the packets that is sent to port 23 but not from ipv4 address 192.168.69.9.

Then, by using:

pcap_compile(handle, &fp, filter_exp, 0, net);
pcap_setfilter(handle, &fp);

We can compile the BPF filter along with the whole session and attach it onto the session.

By the following code:

packet = pcap_next(handle, &header);
struct timeval now;
gettimeofday(&now, NULL);
printf("%lu \n", now.tv_sec);

We can sniff the packet sent into dev, check if it match the rules and if so, shot the timestamp.

So what we gonna do next is using the XDPdump program we use couple weeks ago to add a timestamp at that time when the packet come into the NIC device, which will give us the latency of the whole datapath, so we can compare it with XDP filter and kernel filter.

The program will run on a linux ubuntu 18.04 virtual machine with single CPU core, 4096 MB memory size.

Result

We first try the filter with only no rules in it.

XDP timestamp	Userspace timestamp	Latency
1653233494.039634335	1653233495.000231	961ms
1653233597.791560969	1653233598.661747	870ms
1653233670.660385326	1653233671.935206	1275ms
1653233807.877058220	1653233808.684124	807ms
1653233885.859696460	1653233887.000152	1141ms

So the standard(no rules) latency would be 1011ms.

Then with 16, 32, 64, 128, 256, 512 rules, we have:

• 1675ms with 16 rules
• 1680ms with 32 rules

When I tried to insert more than 32 rules, the program crashed.

Warning: Kernel filter failed: Cannot allocate memory

I think it's because the filter rules is save in a char array filter_exp, I will find a way to solve it at the future.

Conclusion

1, libpcap is honestly not the best approach to fulfill userspace filter, although the filting is not done by network stack, but the BPF filting function is still in the kernel space. By the description of DPDK, I think using DPDK to extract the packet to userspace then doing the filter stuff is more likely to be "userspace filtering".

2, Adding the latency of libpcap to the graph of kernel space filtering and driver level filtering I gathered last time:

We can see that the letancy is quite long, it's almost 2 to 3 times than the XDP and iptables. But the latency is high when there's no rules attach to it, so I think it's maybe the problem of libpcap's actions of extract the packets out of network stack.

3, There's another possibility about the high latency and limitation of number of rules – this experiment is running on a virtual machine. I wonder if the results will change after I re-test with a physical machine.