Configuring FCoE in Linux (RHEL) and HP FlexFabric

Actually it’s easy. Very easy indeed, like going 1, 2, 3.

1. Collect information about MAC addresses to distinguish pure Ethernet NICs and CNA that will pass FCoE traffic. The latter have both MAC and WWN addresses.
2. Power on a server and update /etc/udev/rules.d/70-persistent-net.rules if required.
3. Activate new dev rules:

   # udevadm trigger
   

4. Install fcoe-utils and lldpad packages:

   # yum install fcoe-utils.x86_64
   

5. Rename /etc/fcoe/cfg-ethx file using the name of you CNAs. For example, if eth5 is your CNA interface, then:

   # cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth5
   

6. Edit /etc/fcoe/cfg-ethX files and set DCB_REQUIRED=”yes” to DCB_REQUIRED=”no”
7. Start FCoE and LLDPAD services and set adminStatus to disable for ALL Broadcom-based CNA interfaces as stated by HP. Please note, that
   

   …In a FlexFabric environment, LLPAD must be disabled on all network adapters…

   # chkconfig lldpad on
   # chkconfig fcoe on
   # service lldpad start
   # service fcoe start
   # for d in `ip link ls | grep mtu | awk -F \: '{print $2}'`; do lldptool set-lldp -i $d adminStatus=disabled; done
   # cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-eth5
   

8. Create Ethernet configuration file for all CNA iterfaces to make sure they will be brought online after reboot:

   DEVICE=eth5
   ONBOOT=yes
   BOOTPROTO=none
   USERCTL=NO
   MTU=9000
   

9. Run ifup to bring FCoE interfaces up. If everything is OK reboot the server as a final test and start enjoying FCoE.

   # ifup eth5
   

10. Why MTU=9000? Because FC payload is 2,112 bytes jumbo frames must be turned on to avoid unnecessary IP fragmentation.