How to setup Mellanox Connect-IB InfiniBand Card ?

1) InfiniBand card

Mellanox Connect-IB FDR InfiniBand Card

• MCB193A-FCAT
• Single QSFP+ Port 56 Gb/s
• PCI 3.0 x16
• Hardware revision: A4
• Firmware version: 10.0.2300

2) OS Linux

Red Hat Enterprise Linux 6.5

3) Show InfiniBand cards

# lspci -v | grep Mellanox

4) Upgrade InfiniBand card firmware

Download and install _old_ Mellanox OFED for Linux (MLNX_OFED_LINUX v2.0-3.0.0)
MLNX_OFED_LINUX-2.0-3.0.0-rhel6.4-x86_64.iso

# mkdir /mnt/disk
# mount -o loop MLNX_OFED_LINUX-2.0-3.0.0-rhel6.4-x86_64.iso /mnt/disk
# /mnt/disk/mlnxofedinstall --fw-update-only

5) Make InfiniBand card driver

Download _latest_ Mellanox OFED for Linux (MLNX_OFED_LINUX v2.1-1.0.0)
MLNX_OFED_LINUX-2.1-1.0.0-rhel6.5-x86_64.iso

# umount /mnt/disk
# mount -o loop MLNX_OFED_LINUX-2.1-1.0.0-rhel6.5-x86_64.iso /mnt/disk
# /mnt/disk/mlnx_add_kernel_support.sh -m /mnt/disk/ --make-iso

6) Install InfiniBand card driver

# umount /mnt/disk
# mount -o loop /tmp/MLNX_OFED_LINUX-2.1-1.0.0-rhel6.5-x86_64-ext.iso /mnt/disk
# /mnt/disk/mlnxofedinstall

7) Check InfiniBand card

# hca_self_test.ofed

8) Show InfiniBand cluster members

# ibnodes

%%