Using SR IOV with BESS

Tested with QEMU 2.5

Host Setup

The kernel should boot with intel_iommu=on. Also make sure IOMMU is enabled in BIOS (VT-d).

Suppose 0000:02:00.1 (enp2s0f1) is the PF network interface from which you want to create a VF.

./dpdk-devbind.py -b i40e 0000:02:00.1  # only if not already bound to the kernel driver
echo 1 > /sys/class/net/enp2s0f1/device/sriov_numvfs   # This will create a VF. Check with lspci
modprobe pci-stub
echo "8086 154c" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:02:0a.0 > /sys/bus/pci/devices/0000:02:0a.0/driver/unbind  # 02:0a.0 is the BDF of VF
echo 0000:02:0a.0 > /sys/bus/pci/drivers/pci-stub/bind

QEMU option

qemu-system-x86_64 ... -device pci-assign,host=0000:02:0a.0,bus=pci.0,addr=0x10 ...

This will pass the 02:0a.0 host device to the VM as a PCI device at 00:10.0. bus and addr are optional.

Old QEMU versions may show this error message, but fear not.

qemu-system-x86_64: -device pci-assign,host=03:02.0: pci-assign: Invalid ROM.

For optimal performance, make sure that you launch the VM on hugepages, to minimize the IOTLB miss cost.

Guest Setup

i40evf driver should be working without any additional configuration.

Use igb_uio to bind the VF device to DPDK (when I tried, uio_pci_generic and vfio-pci didn't work). Check the MAC address of the VM; only packets with the matching destination MAC address will be switched to the VF.