An issue was encountered when using a Mellanox ConnectX-4 NIC in an ESXi 6.7 server connecting to VPSA Storage Nodes equipped with Mellanox ConnectX-3 Pro NICs - iSER would not connect to the VPSA. iSCSI connectivity worked OK.
The issue was solved after changing RoCE to version 1 on the ESXi initiator side.
On the ESXi server perform the following steps:
#esxcli system settings kernel set -s rdmaRoceIPBasedGidGeneration -v FALSE
Then, to confirm the setting is changed, perform the following steps:
1. #esxcli system settings kernel list | grep rdma
rdmaRoceIPBasedGidGeneration Bool FALSE TRUE
2. Reboot the server to implement the change.
3. #esxcli system settings kernel list | grep rdma
rdmaRoceIPBasedGidGeneration Bool FALSE FALSE TRUE
After rebooting the ESXi server, iSER connectivity could be established.