Installation procedure for CUDA & cuDNN

cuda_install.md

How to install CUDA & cuDNN on Ubuntu 22.04

Install NVIDIA drivers

Update & upgrade

sudo apt update && sudo apt upgrade

Remove previous NVIDIA installation

sudo apt autoremove nvidia* --purge

Check Ubuntu devices

ubuntu-drivers devices

You will install the NVIDIA driver whose version is tagged with recommended

Install NVIDIA drivers

My recommended version is 525, adapt to yours

sudo apt install nvidia-driver-535

Or install Ubuntu recommended drivers (instead of specific NVIDIA drivers above)

sudo ubuntu-drivers autoinstall

Reboot & Check

reboot

after restart verify that the following command works

nvidia-smi

Install CUDA drivers

Update & upgrade

sudo apt update && sudo apt upgrade

Install CUDA toolkit

sudo apt install nvidia-cuda-toolkit

Check CUDA install

nvcc --version

Install cuDNN

Download cuDNN .deb file

You can download cuDNN file here. You will need an Nvidia account. Select the cuDNN version for the appropriate CUDA version, which is the version that appears when you run:

nvcc --version

Install cuDNN

sudo apt install ./<filename.deb>
sudo cp /var/cudnn-<something>.gpg /usr/share/keyrings/

My cuDNN version is 8, adapt the following to your version:

sudo apt update
sudo apt install libcudnn8
sudo apt install libcudnn8-dev
sudo apt install libcudnn8-samples

Test CUDA on Pytorch

Create a mamba env and activate it

mamba create -n pytorch python=3.10
mamba activate pytorch

Install pytorch

mamba install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch-nightly -c nvidia

Open Python and execute a test

import torch
print(torch.cuda.is_available()) # should be True

t = torch.rand(10, 10).cuda()
print(t.device) # should be CUDA

NVIDIA Container Toolkit

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Requirements: Docker

Installing the NVIDIA Container Toolkit

1. Configure the production repository:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

2. Update the packages list from the repository:

sudo apt update

3. Install the NVIDIA Container Toolkit packages:

sudo apt install nvidia-container-toolkit

Configure the Docker daemon for the NVIDIA Container Runtime

Use the NVIDIA Container Toolkit CLI to configure Docker to use the NVIDIA libraries, then restart Docker:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Test Configuration

Run this command to check the Docker configuration for CUDA:

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Your output should resemble the nvidia-smi output.