Skip to content

Instantly share code, notes, and snippets.

@jlm
Last active June 6, 2017 21:40
Show Gist options
  • Save jlm/8fc97297d47584784032e28712e30dc9 to your computer and use it in GitHub Desktop.
Save jlm/8fc97297d47584784032e28712e30dc9 to your computer and use it in GitHub Desktop.
How to have Emoncms on Raspberry keep its root on an iSCSI target

Motivation

Fed up with SD cards wearing out and stopping working.

Method

Write the emonPi image to an 8G SC card (here using macOS)

    diskutil unmountDisk disk2
    sudo dd if=Downloads/emonSD-07Nov16.img of=/dev/rdisk2 bs=1m

Boot the image, update and install open-iscsi.

    sudo apt-get update
    sudo apt-get upgrade
    sudo apt-get install open-iscsi
    sudo shutdown -r now "Load iSCSI drivers"

The upgrade prompts you to choose whether to keep the existing php.ini or use the new one from the upgrade. In this case we keep the existing one, as it has been customised for the Emoncms application.

Open-iscsi already includes the necessary kernel modules, so there is no need to recompile the kernel. After the reboot, we can test whether the iSCSI initiator works. For this description, I am assuming you already have an iSCSI target set up and ready to use. I am using a Synology Diskstation for this purpose. The attached screenshot shows an iSCSI target that is suitable for using for Emoncms. This is configured using the Synology Storage Manager, on the iSCSI Target tab.

Now test whether the iSCSI initiator works and can see the target. Put the IP address of your iSCSI target in place of the XXX.XXX.XXX.XXX.

    sudo iscsiadm --mode discovery --type sendtargets --portal XXX.XXX.XXX.XXX

If all goes well, you will see one or two lines of output for each available iSCSI target. In my case I see two lines for each target, one with an IPv4 address and another with an IPv6 address.

To attach the target as a disk we can use:

    sudo iscsiadm --mode node --targetname <targetname-from-discovery-phase> --portal XXX.XXX.XXX.XXX --login

If all goes well, a message should be displayed indicating that Login to the target was successful.
In my testing I experienced inconsistent results when using CHAP authentication as configured in the screenshot. Typing dmesg should indicate which device has been attached, in my case, sda.

Now that the disk is available, we need to copy the root filesystem onto it. This process destroys anything that was already on the iSCSI target. Make sure you use the correct device - it might not be sda in your case!

    sudo mkfs.ext4 -m0 /dev/sda
    sudo mkdir /mnt/iscsi
    sudo mount /dev/sda /mnt/iscsi

Now we copy over the root filesystem from the SD card to the new disk, excluding stuff which is created by Linux dynamically:

    sudo rsync -avhP --exclude /boot --exclude /proc --exclude /sys --exclude /mnt --exclude /media --exclude /run / /mnt/iscsi/
    sudo mkdir /mnt/iscsi/{proc,sys,boot,initrd,mnt,media,run}

This will take a while.

We will need an initramfs to allow the iSCSI target to be mounted during the boot process. Luckily, the open-iscsi package contains the necessary cleverness to do this. To trigger it, we create /etc/iscsi/iscsi.initramfs in the current and new root filesystems. As documented here and probably in better places too, any special iSCSI parameters can be entered into that file or on the Linux command line (cmdline.txt). However its the existance of the file which is most important. Create /etc/iscsi/iscsi.initramfs with the following one-line content:

    # The iSCSI parameters are specified in cmdline.txt.  This file triggers iSCSI support in initramfs.

Then copy it over into the new filesystem.

    sudo cp /etc/iscsi/iscsi.initramfs /mnt/iscsi/etc/iscsi/

Next we generate the initramfs image. This is used by the kernel during the initial boot as a root filesystem. Its job is to make the real root filesystem available. In this case we are using it to load up the kernel modules for iSCSI, login to the iSCSI target and mount the root filesystem. The update-initramfs command handles all of these functions automatically once it's been informed of the need for iSCSI as above.

    sudo update-initramfs -c -k `uname -r`

This creates the initramfs image. The output filename contains the kernel version number. In my case it was /boot/initrd.img-4.9.24-v7+. We rename it to avoid having to keep updating the kernel config.txt file:

    sudo mv /boot/initrd.img-`uname -r` /boot/initrd.img

Now we're ready to change the kernel's configuration files to use the new boot setup. Once this has been done, the next reboot will try to use the new settings, and if they don't work, the system probably won't boot anymore. If this happens, the thing to do is to mount the SD card in a computer which can access the /boot partition and change back to the system default settings. We keep the original configuration file handy in case this happens:

    cd /boot
    cp config.txt config.sdcard
    cp cmdline.txt cmdline.sdcard
    cp config.txt config.iscsi
    cp cmdline.txt cmdline.iscsi

Now edit cmdline.iscsi so that it looks like this:

dwc_otg.lpm_enable=0 console=tty1 ip=192.168.0.6:192.168.0.102:192.168.0.1:255.255.255.0:rpi-data:eth0:off iscsi_target_ip=192.168.0.102 iscsi_initiator=iqn.2016-06-11.uk.org.yourname.boilerpi:openiscsi-initiator iscsi_target_name=iqn.2016-06-13.uk.org.yourtarget:DiskStation.Target-4.cc5b6ecccb rw root=UUID=dfe9386d-5460-4a9c-8f24-72a74a770ca9 rootfs=ext4 elevator=deadline rootwait panic=15

You have to customise a number of those fields:

  • The ip= field: The first IP address is the address of this Raspberry Pi. The second is the IP address of an NFS server (if used) - I set it to the same as the iSCSI target IP. The third is the IP address of the default gateway (which is typically your router). Then comes the netmask. Then, this Raspberry Pi's hostname. Then, the name of the network interface to use. Then, the auto-configuration options. Full documentation is available here.
  • The iscsi_target_ip= field: This is the IP address of the iSCSI target.
  • The iscsi_target_name= field: this is the target name we used in the iscsiadm --login command above.
  • The iscsi_initiator= field: This should be a unique string.
  • The root= field: This is the UUID phrase obtained from running the command sudo blkid /dev/sda, assuming that your iSCSI target became /dev/sda following the iscsiadm --login command. Don't include the double quotes.

Then, edit /boot/config.iscsi and change the bottom of the file by adding these lines:

# Boot from ISCSI using scripts built into initramfs-tools
initramfs initrd.img followkernel

# Disable Raspberry Pi 3 bluetooth
dtoverlay=pi3-disable-bt

Disabling Raspberry Pi 3 bluetooth is nothing to do with getting it to boot from iSCSI, and should probably only be done on a Pi 3. It's necessary if you want your Pi to be able to communicate with the RFM69 module.

One more change is needed in the new root filesystem: we need to change /etc/fstab to reflect the fact that the root filesystem resides on the iSCSI target. In my version of the EmonCMS software, the fstab file is a symbolic link to a file in /home/pi, so we have to change that too, using mv and cp:

    cd /mnt/iscsi/etc
    sudo mv fstab fstab.was
    sudo cp fstab.was fstab

Then, edit the /mnt/iscsi/etc/fstab file and find the line whose second fields is /. Mine looked like this:

    /dev/mmcblk0p2  /       ext4    defaults,ro,noatime,nodiratime,errors=remount-ro 0 1

Change it to this, using the UUID value that we put in cmdline.iscsi earlier:

    UUID=dfe9386d-5460-4a9c-8f24-72a74a770ca9    /   ext4  defaults,rw,noatime,nodiratime,errors=remount-ro 0 1

You may notice that I changed the ro to rw in this line. That's because with an iSCSI-mounted root filesystem, there's no real need for it to be mounted read-only. However, there's no reason you couldn't leave it as read-only if you want.

If there is a line whose second field is /home/pi/data, then delete that line, so that the data will be stored on the remote iSCSI target instead of the SD card. There should also be a line whose second field is /boot and that line should remain, because we are still booting from the SD card.

OK, so now is the moment of truth. Replace the current kernel parameters with the iSCSI ones and reboot. See you on the flip side.

    sudo cp /boot/cmdline.iscsi /boot/cmdline.txt
    sudo cp /boot/config.iscsi /boot/config.txt
    shutdown -r now

Remember - if it doesn't work, you can mount the SD card on another computer and copy the scripts for SD card booting back again.

Resources

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment