MARCH Login.md

MARCH Course Computation Resources

Login

Command: ssh -J <username>@<login node> <username>@<GPU node>

Login nodes

• 140.112.51.229 (z-gotham.math.ntu.edu.tw)
• 140.112.51.228 (z-greenwich.math.ntu.edu.tw)

GPU nodes

• 140.112.51.222 (brillante.math.ntu.edu.tw)
• 140.112.51.223 (cantabile.math.ntu.edu.tw)

Example: ssh -J [email protected] [email protected]

The default password is the same as your username. Change the default password with yppasswd

Use Tensorflow

Load necessary modules:

• module load anaconda3/5.2.0
• module load cuda/10.0
• module load cudnn-7.6/10.0
• module load tensorflow-gpu/1.14.0

Run you Tensroflow Python code!

Use Chainer

Note: It only works on CPU. Load necessary modules:

• module load anaconda3/5.2.0
• module load cupy
• module load chainer

Select GPU

• Check which GPU is not in use with nvidia-smi (Remember the ID!)
• Run export CUDA_VISIBLE_DEVICES=<ID> to use the specific GPU

If you encountered permission denied error on pip install, try to add --user.

where to get the demonstrated .py files: full_fisher.py, kfac.py, adam.py ... etc ?

PyTorch > Chainer translation

torch.nn > chainer.links

Conv2D > Convolution2D
Linear > Linear

torch.nn.functional > chainer.functions

MaxPool2d > max_pooling_2d
relu > relu
CrossEntropyLoss > softmax_cross_entropy
backward > backward

y=model.forward(x) > y=model(x)

torch.optim > chainer.optimizers

SGD > SGD
RMSprop > RMSprop
Adagrad > AdaGrad
Adadelta > AdaDelta
Adam > Adam
step > update
zero_grad > cleargrads

Use ChainerKFAC

• git clone https://github.com/tyohei/chainerkfac
• cd chainerkfac
• pip install -e .
• cd examples/mnist
• pip install mpi4py

Use TensorFlowKFAC

• use python2.7
• module load cuda/9.0 cudnn-7.1/9.0
• pip install tensorflow-gpu==1.12.0
• pip install 'kfac[tensorflow-gpu]'
• pip install tensorflow-probability==0.5.0
• pip install dm-sonnet==1.32.0
• Add a line tf.logging.set_verbosity(tf.logging.INFO) to line 334 in autoencoder_mnist.py.

Use ChainerKFAC with MPI

module load anaconda3/5.2.0
module load cupy/cuda100
module load chainer-gpu
module load mpi4py/3.0.0-mpi-3.1.4

mpirun -np 2 python train.py --distributed
Note: for some reasons, you need to go to dolcissimo (140.112.51.224).

Use PyTorchKFAC

module load cuda/10.0 cudnn-7.6/10.0 torch
https://github.com/alecwangcq/KFAC-Pytorch.git
pip install networkx
pip install tensorflow-gpu
pip install torch torchvision

[email protected]