With updates to the modules on OzStar, we have to be careful with the modules setup and the flags we pass to Meraxes...

First off - make a copy of you module list so you can return to it if things go wrong:

module -t list >& $HOME/old_modules.txt

then purge your modules and set the purged state to be your default:

module purge
module save

You can now log out and log back in again.

I've been told by HPC support that module restore is not a good way to maintain the list of modules that we want to use as the method is susceptible to causing new versions of system default modules to not be loaded. It also is not reproducable. What I am doing instead is adding a new file to my home directory called .modules.sh and filling that with my module load commands, then pasting the contents of that directly into my job submission scripts. I also have the following in my .bash_profile so that these modules are loaded automatically when I log in:

module load anaconda3/5.1.0
source activate default
source $HOME/.modules.sh

where default is the name of my default anaconda environment that I use (your name might be different depending on what you chose).

From this point, we have two options for building Meraxes. We can either use gcc or the intel compilers. The former is the easiest to get working.

GCC

Here is the new list of modules that you will need to load:

git/2.16.0
cmake/3.10.2
cuda/9.0.176
gcc/6.4.0
openmpi/3.0.0
hdf5/1.10.1
gsl/2.4

Note that these are very deliberately not the most recent versions available.

You will also need to update your local.cmake file to point to a newly compiled gcc version of fftw:

set(FFTW_ROOT "/fred/oz013/3rd_party/fftw-3.3.7-gcc")

Once you've done that, Meraxes should compile fine as usual. The GPU version should also work fine.

ICC

The intel compilers take a bit more faffing around, however, it is possible that they will produce faster binaries. I've not tested how much faster (if at all) they are in the case of Meraxes, but if someone want's to check that would be useful! :)

Here are the modules you'll need:

git/2.16.0
icc/2016.2.181-gcc-6.4.0
openmpi/3.0.0
cmake/3.10.2
cuda/9.0.176
gsl/2.4
hdf5/1.10.1

To compile Meraxes you will need to add extra commands to your local.cmake (or the equivalent on the command line when you call cmake):

set(CMAKE_C_COMPILER "icc")
set(CMAKE_CXX_COMPILER "icpc")
set(CXXFLAGS "-wd3802")

You will then need to make sure your version of Meraxes includes the most recent commits from smutch/master, or you'll need to fetch that branch and cherry pick commit a couple of commits:

git cherry-pick 26c16e8^..12f30c4

Meraxes should then compile and run fine (both CPU only and GPU versions).

smutch/meraxes-ozstar_notes.md

GCC

ICC