triton.md

Triton - Commands and Software install

Triton, TCC and TSCC, are clusters at the San Diego Supercomputer Center (SDSC): http://www.sdsc.edu/services/hpc/hpc_systems.html#tscc

Some commands

Account balance:

$ gbalance -u fpaolo
$ mybalance

Multiple serial jobs: http://tritonresource.sdsc.edu/bundling.php

Submit a batch file (with several jobs):

1. batch submission

   $ qsub jobs.sh

2. or to get an interactive node and launch it from there

   $ qsub -I $ sh jobs.sh

check PBS commands

q<Tab>

check on a job status (from the scheduler)

checkjob -vvv <jobid>
showq | grep <jobid>
showstart <jobid>

check the load in login node (high >= 5.0)

uptime

check feaseble procs (for particular queue)

showbf -f <queue>

Calculate time and cost:

NDS * 8 * Elap Time = SUs
TSK * Elap Time = SUs

Install Anaconda Python

TSCC is Red Hat Linux -> use Linux 64-Bit - Python 2.7

Download:

wget http://09c8d0b2229f813c1b93-c95ac804525aac4b6dba79b00b39d1d3.r79.cf1.rackcdn.com/Anaconda-2.0.1-Linux-x86_64.sh

Install:

sh Anaconda-2.0.1-Linux-x86.sh -b

Set path:

echo 'PATH=$PATH:$HOME/anaconda/bin; export PATH' >> .bash_profile
echo 'alias python=$HOME/anaconda/bin/python2.7' >> .bash_profile
source .bash_profile

Note 1: Anaconda already comes with mpi4py.

Note 2: Anaconda is installed in user folder.

Install mpi4py

Build and install

$ module purge
$ module load intel openmpi_mx
$ cd <path/to/mpi4py>
$ python setup.py build
$ python setup.py install --user

I think the problem has more to do with enthought python + netcdf was build with intel compilers (or at least I think, when I tried to run some tests earlier without the intel module loaded it came back with errors of not finding libimf.so - specific to intel). So trying to mix gcc+openmpi + intel+netcdf caused the crashes.

See: https://lists.sdsc.edu/pipermail/triton-discuss/2009-July/000184.html

To run code set the .bashrc:

# Portland is default on Triton
module unload pgi
module load intel openmpi_mx

And check:

$ source ~/.bashrc
$ module list
Currently Loaded Modulefiles:
  1) intel/11.1         2) openmpi_mx/1.4.1

General info and examples: http://engineering.ucsb.edu/~stefan/cs240-notes.html

Install GMT

Installation with Intel compiler.

Download the script install_gmt.sh from: http://gmt.soest.hawaii.edu/gmt/install_gmt.sh

And set path to netcdf/include:

export CPPFLAGS="-I/opt/netcdf/4.0.1/intel/include"

The path to netcdf/lib:

export LDFLAGS="-L/opt/netcdf/4.0.1/intel/lib"

The compiler to intel:

export CC=”icc”

And install (interactively create the GMT4param.txt):

$ sh install_gmt.sh