According to this link, you need to install the latest CUDA to make the GPU accessible.
So follow the official instruction to install CUDA 12.8, and install pytorch-nightly later:
pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu128
I'm using Nvidia RTX 4060 mobile on M16R2 (Ubuntu 24.04 LTS).
I followed @aryoda step.
sudo apt install nvidia-driver-570-open
Then, I installed CUDA-11.8
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118