Wednesday, November 10, 2010

xen on centos on hp blades: slow networking

Had Xen running on CentOS on some HP 460c blades. CentOS by itself on the blades had great network performance, but the blades running with the Xen kernel had terrible performance, both in the domU and the dom0. It turns out that the issue is the particular networking driver that the CentOS/RHEL/Fedora family has for the blades' Broadcom Corporation NetXtreme II BCM57711E 10-Gigabit PCIe cards. You can see something similar on this RHEL bug.

The lazy workaround is to add "options bnx2x disable_tpa=1" to /etc/modprobe.conf on the dom0 and then restart the dom0. Like so:

# cat /etc/modprobe.conf
alias eth0 bnx2x
alias eth1 bnx2x
options bnx2x disable_tpa=1
alias scsi_hostadapter cciss
alias scsi_hostadapter1 usb-storage

HP advises that you can manually reload the driver, but this did not work for me.