sudo apt update && sudo apt upgradesudo apt autoremove nvidia* --purgeubuntu-drivers devicesYou will install the NVIDIA driver whose version is tagged with recommended
sudo ubuntu-drivers autoinstallMy recommended version is 525, adapt to yours
sudo apt install nvidia-driver-525rebootafter restart verify that the following command works
nvidia-smisudo apt update && sudo apt upgradesudo apt install nvidia-cuda-toolkitnvcc --versionYou can download cuDNN file here. You will need an Nvidia account. Select the cuDNN version for the appropriate CUDA version, which is the version that appears when you run:
nvcc --versionsudo apt install ./<filename.deb>
sudo cp /var/cudnn-<something>.gpg /usr/share/keyrings/My cuDNN version is 8, adapt the following to your version:
sudo apt update
sudo apt install libcudnn8
sudo apt install libcudnn8-dev
sudo apt install libcudnn8-samplessudo apt-get install python3-pip
sudo pip3 install virtualenv
virtualenv -p py3.10 venv
source venv/bin/activatepip3 install torch torchvision torchaudioimport torch
print(torch.cuda.is_available()) # should be True
t = torch.rand(10, 10).cuda()
print(t.device) # should be CUDAGo to Nvidia webiste here. Select latest TensorRT version that matches your CUDA version and download the DEB file.
sudo apt install ./<filename.deb>
sudo apt update
sudo apt install tensorrtVerify that the trtexec utility is present.
whereis trtexec # should be trtexec: /usr/src/tensorrt/bin/trtexec
/usr/src/tensorrt/bin/trtexec
So following the instructions works just great - all installed no problem - but i realize that i cannot seem to get it to NOT install 11.5 - which is a very old version and not compatible with flashattn2 which requires 11.6 and up - ive been banging my head against the wall all day - downloaded .run files for older versions but then nvcc won't register cause its not the right cuda tools or something. Any help would be greatly appreciated.
-- solved ended up being a PATH issue. Also needed to fully purge ALL nvidia related package handlers AND do it all while GUI was turned off.