Configuring FCoE in Linux (RHEL) and HP FlexFabric
Actually it’s easy. Very easy indeed, like going 1, 2, 3.
- Collect information about MAC addresses to distinguish pure Ethernet NICs and CNA that will pass FCoE traffic. The latter have both MAC and WWN addresses.
- Power on a server and update /etc/udev/rules.d/70-persistent-net.rules if required.
- Activate new dev rules:
# udevadm trigger
- Install fcoe-utils and lldpad packages:
# yum install fcoe-utils.x86_64
- Rename /etc/fcoe/cfg-ethx file using the name of you CNAs. For example, if eth5 is your CNA interface, then:
# cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth5
- Edit /etc/fcoe/cfg-ethX files and set DCB_REQUIRED=”yes” to DCB_REQUIRED=”no”
- Start FCoE and LLDPAD services and set adminStatus to disable for ALL Broadcom-based CNA interfaces as stated by HP. Please note, that
…In a FlexFabric environment, LLPAD must be disabled on all network adapters…
# chkconfig lldpad on # chkconfig fcoe on # service lldpad start # service fcoe start # for d in `ip link ls | grep mtu | awk -F \: '{print $2}'`; do lldptool set-lldp -i $d adminStatus=disabled; done # cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth5
- Create Ethernet configuration file for all CNA iterfaces to make sure they will be brought online after reboot:
DEVICE=eth5 ONBOOT=yes BOOTPROTO=none USERCTL=NO MTU=9000
- Run ifup to bring FCoE interfaces up. If everything is OK reboot the server as a final test and start enjoying FCoE.
# ifup eth5
- Why MTU=9000? Because FC payload is 2,112 bytes jumbo frames must be turned on to avoid unnecessary IP fragmentation.
on August 30, 2016 at 3:50 pm
· Permalink
Great tuto, but I think there is a small typo on the line 6. I think you instead wanted to edit cfg-eth5 (instead of cfg-ethX)…
6. Edit /etc/fcoe/cfg-ethX files and set DCB_REQUIRED=”yes” to DCB_REQUIRED=”no”
on August 31, 2016 at 12:36 am
· Permalink
Hi Steve,
Thank you for stopping by and leaving a helpful comment. Deeply appreciate that. To tell the truth, I left cfg-ethX on purpose with the intention to emphasise that there could be other cfg-ethX interfaces and not all of them are CNAs. Obviously my approach didn’t work out and I will update the post to avoid any ambiguity.
Cheers.
on February 6, 2017 at 10:49 pm
· Permalink
Do these settings apply to a Dell qlogic CNA?
on February 7, 2017 at 1:12 pm
· Permalink
Hi.
Wish I had a solid answer but unfortunately, due to the lack of Dell’s equipment in my lab, I don’t.
But based on the information I gleaned from Redhat Configuring a Fibre-Channel Over Ethernet Interface none of the settings are vendor specific and hopefully they could be applied to Dell branded Qlogic CNA as well. Just pain attention to the warning:
That was exactly the reason to set DCB_REQUIRED=”no” and to disable adminStatus with lldptool. Which is also mentioned in Installation Notes on Supermicro’s web site.
Hope that helps.