# Packet Loss in OVS

###### tags: `ovs` `network` `vmware`

This is a case study about an issue of using OVS and Vmware ESXi with NIC teaming. The issue will cause packet loss when transmitted a broadcast packet (such as ARP broadcast) in the network.


A virtual machine (Scientific Linux, hostname center) is running in a VMware ESXi compute node (left host). The VMware ESXi node has two interface in teaming mode connect to a physical switch. Another physical node (Scientific Linux, right host) is also connected to the physical switch.

The TrustKP and vswitch1 are local ports (or say, internal interface) with an IP address configured on OVS. But the TrustKP port is created manually and the vswitch1 is created automatically when the vswitch is created. But they have few difference.

The VM (the Scientific Linux VM) can communicate to another external host via the ports (Because the physical interface eth0 is bound on the OVS).


The problem is the when ping between the TrustKP ports, some packet loss occur, but for the vswitch1 port, no any packet loss.

- lose some packets
10.20.20.1 ping to 10.20.20.2
10.20.20.2 ping to 10.20.20.1

- no any loss of packet
10.30.30.1 ping to 10.30.30.2
10.30.30.2 ping to 10.30.30.1

So we initially observe the mac table in the OVS. The MAC address of the TrustKP (10.20.20.1) port is map in port number 2 in the OVS.


But when packet loss occur, we observed that the TrustKP is map to port number 5 in the OVS. And the port number 5 is belongs to the interface eth0. The MAC address of the TrustKP is flapping between the port 2 and port 5.


We use tcpdump to capture the packet in the interface of eth0 and TrustKP interfaces. We captured duplicated ARP broadcasts when trying to send ARP broadcast by ping a non exist IP (10.20.20.5).


Again, ping to a broadcast IP (10.20.20.255) to generate broadcast traffic.
We also captured duplicated ICMP request packet on both eth0 and TrustKP port.


Finally, we can confirm that a loop occur in our environment.
When using the teaming in the vSwitch of Vmware ESXi, it can detect the packet that is transmitted out and back from another interfaces but only for the MAC address of interface that is associated in the vSwitch.

Because of the TrustKP is created internally in the Virtual Machine, the vSwitch of Vmware ESXi does not know about its MAC address. So the NIC teaming will allow the traffic transmit back from another interface and forward to eth0 of the VM.

The vswitch1 port is not affected because its a built-in interface, its port number is fixed, it always bind to port number 65535 and it will not be updated.