Hackathon_steps

ICEBERG-hackathon.md

1. Login to the VM.

• From your terminal ssh to the following IP:

  ssh [email protected]

2. Generate XSEDE certificate.

• In order to be able to login to Bridges from your VM you need to generate XSEDE certificate using the following command:

  myproxy-logon -s myproxy.xsede.org -l user_name -t 72

3. Login to XSEDE Bridges HPC.

• gsissh -p 2222 bridges.psc.xsede.org

4. Create the use case VE and install it on the HPC.

• For the Seals use case: Create_VE
• For the Penguins use case: Create_VE
• Note that you need to install pandas using the following command: pip install pandas

5. Sign out of XSEDE Bridges.

• exit

6. Create the ICEBERG VE in your VM.

• Create a Python Virtual Environment and activate it using the following commands:

virtualenv -p python3 iceberg_env

7. Activate the VE in your VM.

source iceberg_env/bin/activate

8. Install ICEBERG-middleware in your VM.

• Install the Middle using the following commands:

    git clone https://github.com/iceberg-project/ICEBERG-middleware.git
    cd ICEBERG-middleware/
    git checkout devel
    mkdir model
    cd model/
    wget https://github.com/iceberg-project/Seals/raw/master/models/Heatmap-Cnt/UnetCntWRN/UnetCntWRN_ts-vanilla.tar
    cd ../
    pip install . --upgrade

9. Run the iceberg command.

• Setup MongoDB: export RADICAL_PILOT_DBURL=mongodb://iceberg:[email protected]:27017/iceberg
• For the Seals use case:

iceberg --resource xsede.bridges --queue GPU-small --cpus 32 --gpus 2 --input_path /pylon5/mc3bggp/bspitz/Scale/ --output_path ./ --walltime 20 --project mc3bggp --rmq_username iceberg --rmq_password 1Ce63Rg --rmq_endpoint 129.114.17.185 --rmq_port 5672 seals --scale_bands 299 --model UnetCntWRN_ts-vanilla.tar --model_path model/ --model_arch UnetCntWRN --hyperparameters A --ve_seals /pylon5/mc3bggp/paraskev/pytorchCuda

• For the Penguins use case:

iceberg --resource xsede.bridges --queue GPU-small --cpus 32 --gpus 2 --input_path /pylon5/mc3bggp/paraskev/penguins_data/ --output_path ./ --walltime 30 --project mc3bggp --rmq_username iceberg --rmq_password 1Ce63Rg --rmq_endpoint 129.114.17.185 --rmq_port 5672 penguins --epoch 300 --model v3weakly_unetr_bs96_main_model_ignore_bad --model_path /pylon5/mc3bggp/paraskev/checkpoints_dir/checkpoints_CVPR19W/ --ve_penguins /pylon5/mc3bggp/paraskev/iceberg_penguin