Prepare custom focal image with NVIDIA CUDA

prepare_focal_nvidia_image.md

Download Ubuntu focal image

wget https://cloudhypervisorstorage.blob.core.windows.net/images/focal-server-cloudimg-amd64-custom-20210106-1.raw
mv focal-server-cloudimg-amd64-custom-20210106-1.raw focal-server-cloudimg-amd64-nvidia.raw

Extend the image

qemu-img resize focal-server-cloudimg-amd64-nvidia.raw 20G

Fix the GPT and resize the partition

sudo parted focal-server-cloudimg-amd64-nvidia.raw
(parted) print
Warning: Not all of the space available to /home/sboeuf/nvidia_vfio_image/focal-server-cloudimg-amd64-nvidia.raw appears to be
used, you can fix the GPT to use all of the space (an extra 37330944 blocks) or continue with the current setting?
Fix/Ignore? Fix
Model:  (file)
Disk /home/sboeuf/nvidia_vfio_image/focal-server-cloudimg-amd64-nvidia.raw: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
14      1049kB  5243kB  4194kB                     bios_grub
15      5243kB  116MB   111MB   fat32              boot, esp
 1      116MB   2361MB  2245MB  ext4

(parted) resizepart 1 21.5GB
(parted) print
Model:  (file)
Disk /home/sboeuf/nvidia_vfio_image/focal-server-cloudimg-amd64-nvidia.raw: 21.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
14      1049kB  5243kB  4194kB                     bios_grub
15      5243kB  116MB   111MB   fat32              boot, esp
 1      116MB   21.5GB  21.4GB  ext4

(parted) quit

Boot the image with a VM connected to the internet and install NVIDIA packages

./cloud-hypervisor \
	--kernel hypervisor-fw  \
	--disk path=focal-server-cloudimg-amd64-nvidia.raw path=/tmp/ubuntu-cloudinit.img \
	--cpus boot=16 \
	--memory size=16G \
	--net fd=3,mac=c2:67:4f:53:29:cb 3<>/dev/tap6

Setup and test connectivity

sudo dhclient
sudo apt update

NVIDIA CUDA installations

References from:

• https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html#ubuntu-lts)
• https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions
• https://developer.nvidia.com/dcgm

distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
wget https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/cuda-$distribution.pin
sudo mv cuda-$distribution.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/7fa2af80.pub
echo "deb http://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64 /" | sudo tee /etc/apt/sources.list.d/cuda.list
sudo apt update
sudo apt -y install cuda-drivers cuda datacenter-gpu-manager
export PATH=/usr/local/cuda-11.2/bin${PATH:+:${PATH}}
# Build CUDA samples
mkdir nvidia
cuda-install-samples-11.2.sh nvidia/
cd nvidia/NVIDIA_CUDA-11.2_Samples/
make -j16