Skip to content

Instantly share code, notes, and snippets.

@brantfaircloth
Created April 30, 2011 21:30
Show Gist options
  • Save brantfaircloth/950010 to your computer and use it in GitHub Desktop.
Save brantfaircloth/950010 to your computer and use it in GitHub Desktop.
mpi and mrbayes on ec2
  • start an image:

      ec2-run-instances --key keypair ami-74f0061d \
          --instance-type=c1.xlarge \
          --block-device-mapping '/dev/sda2=ephemeral0' \
          --block-device-mapping '/dev/sda3=ephemeral1'
    
  • log in

      ssh -i ec2-keypair [email protected]
    
  • install compilers and stuff (including mpich2 and screen):

      yum groupinstall "Development Tools"
      yum install mpich2-devel.x86_64 mpich2.x86_64 screen readline-devel
    
  • mount ephemeral store

      sudo su
      mkdir /mnt/data
      mount /dev/sda2 /mnt/data
      chown ec2-user:ec2-user /mnt/data
      exit
    
  • get mrbayes and compile mpi version (as ec2-user is fine)

      mkdir ~/src/ ~/bin/
      cd src && wget http://downloads.sourceforge.net/project/mrbayes/mrbayes/3.1.2/mrbayes-3.1.2.tar.gz
      tar -xzvf mrbayes-3.1.2.tar.gz
      cd mrbayes-3.1.2
    
  • edit makefile and at very least

      MPI ?= yes
    
  • run make - ensure compiling w/ mpicc

      make
      mv mb ../../bin/
    
  • set mpi password, restrict access to pwd file, and start daemon

      cd ~
      echo "MPD_SECRETWORD=kittens" > .mpd.conf
      chmod 0600 /home/ec2-user/.mpd.conf
      mpd &
    
  • rsync data up to ec2 instance

      rsync -avz -e "ssh -i /path/to/my/ec2-keypair" \
          ./ [email protected]:/mnt/data/ 
    
  • run mrbayes with

      cd /mnt/data
      # start screen
      screen
      mpirun -np 8 /home/ec2-user/bin/mb
      # do mrbayes stuff...
      # detach from screen
      <ctrl-a>,<ctrl-d>
      # wait
    
  • log back in to instance, and re-attach (-r) to screen session

      screen -r
    
  • when done, pull data down with:

      rsync -avz -e "ssh -i /path/to/my/ec2-keypair" \
          [email protected]:/mnt/data/ ./
    
  • you can pull data down while a run is in progress, but you may need to edit the *.p and *.t files to make the number of iterations equal in the *.p files and the *.t files and add "End;" to the *.t files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment