PYTORCH_INSTALL_INSTRUCTIONS_CENTOS6.md

Step 1: Install Anaconda compatible with RHEL 6

The file I used was Anaconda3-2020.11-Linux-x86_64.sh from https://repo.anaconda.com/archive/

wget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh
bash Anaconda3-2020.11-Linux-x86_64.sh

Step 2: Create a new environment.

I tried lots of older and newer versions of python but 3.7.0 is the one that worked for me. The patchelf is necessary for the next step.

conda create --name py37
conda activate py37
conda install -c anaconda pip python=3.7.0 --yes
conda install patchelf -c conda-forge --yes
conda install pytorch==1.10.2 -c pytorch

At this point, python should work but importing torch should not.

Step 3: Manually download newer versions of glibc

This is solely derived from: https://gist.github.com/michaelchughes/85287f1c6f6440c060c3d86b4e7d764b

mkdir ~/my_libc_env
cd ~/my_libc_env

# Get libc files (URL verified by AS on 2022/02/24)
wget https://launchpadlibrarian.net/353523729/libc6_2.23-0ubuntu10_amd64.deb
wget http://launchpadlibrarian.net/353523714/libc6-dev_2.23-0ubuntu10_amd64.deb

# Unpack files into current directory (will create usr/ and lib/ and lib64/ folders)
ar p libc6_2.23-0ubuntu10_amd64.deb data.tar.xz | tar xvJ
ar p libc6-dev_2.23-0ubuntu10_amd64.deb data.tar.xz | tar xvJ

# Get libstdc++ (URL verified by AS on 2022/02/24)
wget https://yum.oracle.com/repo/OracleLinux/OL6/0/base/x86_64/getPackage/libstdc++-devel-4.4.4-13.el6.x86_64.rpm

# Unpack into current directory (will add content to lib/ and lib64/ folders)
rpm2cpio libstdc++6-4.8.2-3.2.mga4.x86_64.rpm | cpio -idmv

Step 4: Modify Mike's script (attached) and run it to patch the python

I've modified the original slightly to work. This broadly works in the following manner:

1. Create GLIBC_LD_PATH environment variable
2. Finds the current python path and stores it to python_exe (this expects you to be running it in a conda env)
3. Finds the lib folder in the conda environment and stores it to CONDA_ENV_LIB variable
4. Copies the original python to a file named 'python_backup' in the same folder
5. Creates the rpath variable (run-time path variable) which first searches in our custom GLIBC_PATH first
6. 'Patches' python file to hard-code these paths into the python binary

bash rewrite_python_exe_glibc_with_patchelf.sh

At this point, running python and importing torch still didn't work for me (if it did for you, GREAT! - you don't have to do the next step).

Step 5: Create an alias for this patched python (loaded along with the right glibc path)

You're better off adding this to your .bashrc file or re-create the alias at login (AFTER activating the conda environment)

conda activate py37
alias tpython="LD_LIBRARY_PATH=$HOME/my_libc_env/lib64  `which python`"

Step: Install rest of the libraries you need :

conda install numpy scipy pandas matplotlib --yes
conda install -c conda-forge scikit-learn bayesian-optimization --yes

run tpython instead of python

rewrite_python_exe_glibc_with_patchelf.sh

#!/usr/env bash

# Copied in large part from https://gist.github.com/michaelchughes/85287f1c6f6440c060c3d86b4e7d764b#file-rewrite_python_exe_glibc_with_patchelf-sh

export GLIBC_PATH=$HOME/my_libc_env
export GLIBC_LD_PATH=$GLIBC_PATH/lib/x86_64-linux-gnu/ld-2.23.so

if [[ ! -f $GLIBC_LD_PATH ]]; then
    echo "ERROR: Provided GLIBC_LD_PATH not valid"
    exit
fi

echo "OVERWRITING PYTHON EXECUTABLE:"
python_exe=`which python`
echo $python_exe

IS_CONDA_ENV=`/usr/bin/python -c "print('$python_exe'.count('/envs/') > 0)"`
echo "IS_CONDA_ENV: $IS_CONDA_ENV"

if [[ $IS_CONDA_ENV -ne 'True' ]]; then
    echo "ERROR: Current python executable not in conda env. Will not alter to avoid problems."
    exit
fi

CONDA_ENV_LIB=`/usr/bin/python -c "print('$python_exe'.replace('/bin/python', '/lib'))"`

echo "CREATING BACKUP PYTHON"
python_tmp_exe=`/usr/bin/python -c "print('$python_exe'.replace('python', 'python_backup'))"`
cp $python_exe $python_tmp_exe
echo "$python_tmp_exe"

rpath=$GLIBC_PATH/lib/x86_64-linux-gnu:$CONDA_ENV_LIB:/usr/lib64:/lib64:/lib


echo "$rpath"

echo "CALLING PATCHELF on 'python' binary"
patchelf --set-interpreter $GLIBC_LD_PATH --set-rpath $rpath $python_exe
echo "DONE! patchelf complete"