pytorch-workflow-testing.md

PyTorch workflow testing

In order to iterate faster on workflows it is necessary to setup an environment to run the test.

1. Launch a g5.4xlarge instance in aws with Deep Learning AMI GPU PyTorch 1.10.0 (Ubuntu 20.04) 20221003 ami-09b3fda578ad20138
2. Generate an ssh key pair and add it to your github account.
3. Clone pytorch/rl repo to /home/ec2-user/github/rl
4. Use the following script to start the container and get a shell...

#!/bin/bash

docker run -it --gpus all --rm -v /home/ubuntu/github/rl:/work pytorch/conda-builder:cuda11.6 /bin/bash

1. In the container shell, run the following...

export PYTHON_VERSION=3.8
export CUDA_ARCH_VERSION=11.6
export CU_VERSION="cu${CUDA_ARCH_VERSION:0:2}${CUDA_ARCH_VERSION:3:1}"

echo "PYTHON_VERSION: $PYTHON_VERSION"
echo "CU_VERSION: $CU_VERSION"
        
cd /work

## setup_env.sh
./.circleci/unittest/linux/scripts/setup_env.sh

## install.sh
./.circleci/unittest/linux/scripts/install.sh

## run_test.sh
./.circleci/unittest/linux/scripts/run_test.sh

## post_process.sh
./.circleci/unittest/linux/scripts/post_process.sh