# DPDK introduction and building note (earlier than DPDK ver.20)
###### tags: `DPDK`
## Brief introduction of DPDK
Point of DPDK: kernel-bypass networking.
Most of network latency is caused by application, so there is a radical way to prevent latency caused by application: bypass kernel.
With DPDK, application can send package to userspace TCP/IP stack directory.
NIC send and receive packages by userspace TCP/IP stack (usermode?) directory. (I am not sure)
## Platform
Mother Board: CB-1920
OS: Ubuntu 18.04.5 LTS, kernel 4.15.0
DPDK: 19.11.4 LTS
## Install DPDK
### Required tools
Before you start to build DPDK, please make sure you have installed these packages below:
1. gcc
2. libnuma-dev
3. pkg-config
4. python
```javascript=
# apt install gcc
# apt install libnuma-dev
# apt install pkg-config
# apt install python
```
### Download DPDK
source: https://core.dpdk.org/download/
### Unpacking the file
```javascript=
# tar xf dpdk-19.11.4.tar.xz
# cd dpdk-stable-19.11.4
```
In this example, I put DPDK source file at /home/user.
### Compile kernel module
```javascript=
# cd /home/user/dpdk-stable-19.11.4/usertools
$ sudo ./dpdk-setup.sh
```
After dpdk-setup.sh executes.
* first, choose: [41] x86_64-native-linux-gcc
* second, choose: [45] Insert IGB UIO module
* finally, choose: [62] Exit Script
### Compile l3fwd
```javascript=
# cd /home/user/dpdk-stable-19.11.4/examples/l3fwd
# export RTE_SDK=/home/user/dpdk-stable-19.11.4 //RTE_SDK => DPDK folder path
# export RTE_TARGET=x86_64-native-linux-gcc //RTE_TARGET => what you build at executing dpdk-setup.sh
$ sudo make clean
$ sudo make
```
## Prepare to run DPDK l3fwd
### Setting hugepages and isolcpus
```javascript=
$ sudo vim /etc/default/grub
```
Add these content into the end of GRUB_CMDLINE_LINUX_DEFAULT.
```javascript=
default_hugepagesz=1G hugepagesz=1G hugepages=8 isolcpus=16,17,18,19 //isolcpus means OS would not run process on those chosen cores
```
After that...
```javascript=
# update-grub
# reboot
```
After reboot, check hugepages has been set.
```javascript=
cat /proc/meminfo
```
Use hugepages after hugepages set.
```javascript=
$ sudo mkdir /mnt/huge
$ mount -t hugetlbfs nodev /mnt/huge
```
Check HPET has been set.
```javascript=
$ sudo cat /boot/config-4.15.0-117-generic | grep HPET
```
Disable firewall.
```javascript=
service iptables stop
```
Enable IP forward.
```javascript=
$ sudo sysctl net.ipv4.ip_forward=1
```
Install kernel module.
```javascript=
$ sudo modprobe uio
$ sudo insmod /home/user/dpdk-stable-19.11.4/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
```
Check lan card information and bind lan card with igb_uio module.
```javascript=
# cd /home/user/dpdk-stable-19.11.4/usertools
$ sudo ./dpdk-devbind.py --status //to know lan card status
$ sudo ./dpdk-devbind.py -b igb_uio 18:00.0 //to bind lan card in dpdk
```
## Using l3fwd to test.
```javascript=
# cd /home/user/dpdk-stable-19.11.4/examples/l3fwd/build
$ sudo ./l3fwd -c f0001 -n 4 -- -P -p 0xf --config="(0,0,16),(1,0,17),(2,0,18),(3,0,19)"
```
※-c f0001 => 1111 0000 0000 0000 0001, 1 means binded, 0 means unbinded, 0th core must open(000"1"), 16~19th core binded for l3fwd(1111), others closed("0000 0000 0000 000"1)
※-n 4 => 4 RAM channels
※-P => Enable promiscuous mode?
※-p port amount to configure, 0xf means 4 ports
※config=(0,0,16)... => (no. of lan port, queue, CPU core no.)
## Using testpmd to test (l2)
1. Install kernel module.
```javascript=
$ sudo modprobe uio
$ sudo insmod /home/user/dpdk-stable-19.11.4/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
```
2. Mount hugepages
```javascript=
$ sudo mkdir /mnt/huge
$ mount -t hugetlbfs nodev /mnt/huge
```
3. Check lan card information and bind lan card with igb_uio module.
```javascript=
# cd /home/user/dpdk-stable-19.11.4/usertools
$ sudo ./dpdk-devbind.py --status //to know lan card status
$ sudo ./dpdk-devbind.py -b igb_uio 18:00.0 //to bind lan card in dpdk
```
4. Run testpmd
```javascript=
# cd /home/user/dpdk-stable-19.11.4/x86_64-native-linux-gcc/app
$ sudo ./testpmd -c 1f -n 4 -- -i --portmask=0xf --nb-cores=4
```
※-c 1f => 0001 1111, 1 means binded, 0 means unbinded, 0th core must open(000"1"), 2~5th core binded, so result: 1f
※-n 4 => 4 RAM channels
※-portmask => port amount to configure, 0xf means 4 ports
※--nb-cores => how many CPU cores binded for testpmd
## Notice
Every single version of DPDK, path of tool, kernel module such as l3fwd, igb_uio.ko dpdk-setup.sh testpmd and others are different
Use find command in linux to find path of the tool you need
## About DPDK igb_uio driver
Now igb_uio driver source code path is at: http://git.dpdk.org/dpdk-kmods
Download it via git
## Source
https://hackmd.io/@sysprog/linux-dev-review
https://www.slideshare.net/garyachy/dpdk-44585840
https://www.net.in.tum.de/fileadmin/bibtex/publications/theses/2014-gallenmueller-high-speed-packet-processing.pdf
https://core.dpdk.org/download/
## Special thanks
Rick Xiao
Sign in with Wallet
Connect another wallet