How to remove the last dead path in ESX

Feb  9 10:56:27 esx vmkernel: 47:18:55:03.308 cpu8:4225)WARNING: NMP: nmp_DeviceRetryCommand: Device "naa.60060e80056e030000006e030000004b": awaiting fast path state update for failover with I/O blocked. No prior reservation exists on the device.
Feb  9 10:56:27 esx vmkernel: 47:18:55:03.308 cpu8:4225)WARNING: NMP: nmp_DeviceStartLoop: NMP Device "naa.60060e80056e030000006e030000004b" is blocked. Not starting I/O from device.
Feb  9 10:56:28 esx vmkernel: 47:18:55:04.323 cpu8:4264)WARNING: NMP: nmpDeviceAttemptFailover: Retry world failover device "naa.60060e80056e030000006e030000004b" - issuing command 0x41027f458540
Feb  9 10:56:28 esx vmkernel: 47:18:55:04.323 cpu8:4264)WARNING: NMP: nmpDeviceAttemptFailover: Retry world failover device "naa.60060e80056e030000006e030000004b" - failed to issue command due to Not found (APD), try again...

If these log lines are familiar and you’ve been desperately trying to remove the last FC path to your storage array, which is by a awry coincedence is also dead, then probably the following command could help you in dealing with the issue:

# esxcli corestorage claiming unclaim -A vmhba1 -C 0 -T 3 -L 33 -d naa.60060e80056e030000006e030000004b -t location
# esxcfg-scsidevs -o naa.60060e80056e030000006e030000004b

Just don’t copy/paste it boldly but instead use your C:T:L and naa.* values appropriately.

Worked for me like a charm.

Linux solution for ARP flux

If for some reason you have to configure multiple network interfaces in the same subnet, e.g. eth0 – 192.168.1.1/24, eth1 – 192.168.1.2/24, eth2 – 192.168.1.3/24, etc., on Linux box you’d certainly face a so called ARP flux issue.
To fix it (I assume you have kernel > 2.6.2) just set the following sysctl options as listed below:

net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2

In my particular case that wasn’t a complete remedy as I had to amend an additional sysctl parameter:

net.ipv4.conf.ethX.rp_filter = 0

where ethX is your network adapter, e.g. eth0, eth1, eth3, etc.

Read more about arp_announce and arp_ignore options.

Hope that helps. Good luck.