There’s a Gluster 3.4 RDMA test day right around the corner, and I want to join in on the fun. The trouble is, I don’t have any RDMA-capable hardware in my lab right now. Undaunted, I hit the Web in search of a software-based solution, one that would at least allow me to run through the tests.
I found a pair of promising-looking options:
The information out on the web about these projects is a bit thinner than I’d like, but I found a blog post howto on installing Soft-iWARP on Ubuntu 10.10 and another for Debian 6 and figured I’d try it out on Fedora 18.
Here are the steps I followed:
I started with a VM running the x86_64 version of Fedora 18 with all updates applied.
I installed a bunch of dependencies:
yum install -y kernel-devel libtool autoconf rdma \ libibverbs-utils libibcommon libibcm-devel libibverbs \ git wget yum-utils librdmacm-utils yum-builddep -y kernel-devel
Next, grab the source for Soft-iWARP:
git clone http://git.gitorious.org/softiwarp/kernel.git git clone http://git.gitorious.org/softiwarp/userlib.git
Time to build & install:
cd kernel/softiwarp/ make sudo make install cd ../../userlib/libsiw-0.1/ ./autogen.sh ./autogen.sh (yes, you have to run it twice) ./configure prefix=/usr libdir=/usr/lib64 make sudo make install cp siw.driver /etc/libibverbs.d/
There’s a series of modules that need loading, I created a shell script, as suggested in one of the howtos, to load them:
cd ~ vi load-em-up.sh #!/bin/bash modprobe ib_addr modprobe ib_cm modprobe ib_core modprobe ib_mad modprobe ib_sa modprobe ib_ucm modprobe ib_umad modprobe ib_uverbs modprobe iw_cm modprobe rdma_cm modprobe rdma_ucm modprobe siw sh load-em-up.sh
From here, you can try the commands ibv_devinfo and ibv_devices to get info on your interface(s) — on my one-NIC VM, interfaces appeared for both my virtual NIC and my loopback interface.
You can run the command "rping -s" paired with "rping -c -a 127.0.0.1 -v" on another shell, to ping your loopback interface to see if everything’s working.
Swap out the "127.0.0.1" with the IP address of your test machine to try pinging the non-loopback interface. Use Ctrl-C to break the cycle-of-pingage.
Thank you very much for this page.
It's clear and quite complete.