Skip to content

Instantly share code, notes, and snippets.

@g-k

g-k/0_pytorch_wsl2_install_notes.md

Last active February 16, 2024 00:44
Show Gist options
  • Save g-k/072180b537a405289e97dcd38fe34760 to your computer and use it in GitHub Desktop.
Save g-k/072180b537a405289e97dcd38fe34760 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
0_pytorch_wsl2_install_notes.md

Setting up CUDA and Pytorch on Windows Subsystem for Linux (WSL2)

2023-12-10

I recently set up CUDA and PyTorch to chat with LLMs and play with image recognition neural networks on my desktop. I used Windows Subsystem for Linux (WSL) to avoid running a full Linux alongside Windows 10. There are plenty of other guides for setting CUDA and WSL up, but this one is mine. Hopefully these notes will save someone else some time.

We roughly follow "Enable NVIDIA CUDA on WSL" here.

Note

To get started more quickly use Google Colaboratory. Or for production workloads pay for a Linux machine and GPU with more memory on cloud-hosted instances from runpod.io, fly.io, or another provider.

Tip

Nvidia's CUDA and PyTorch are the most popular options and easiest to set up, but AMD's consumer-grade cards offer more VRAM per $ and AMD's ROCm library is improving as an alternative to CUDA. Check the Radeon tab on the tables in the ROCm windows support and linux support docs to see if ROCm supports your AMD graphics card and this PyTorch page to see if PyTorch supports your ROCm version.

diagram of ML, CUDA, WSL, GPU driver, and Windows layers

from https://developer.nvidia.com/cuda/wsl

Requirements

  1. a computer running Windows 10 or 11
  2. a Nvidia GPU using the Pascal microarchitecture or newer i.e. 10XX or newer for official support. Use the "CUDA-Enabled GeForce and TITAN Products" table for consumer GPUs on this page for CUDA support and this page for CUDA on WSL support
  3. 50GB or more of free disk space
  4. Administrator access
  5. a fast and stable internet connection

Directions

  1. Run Windows Update to update Windows 10 to version 21H2 or newer. 1

  2. Update graphics card drivers.2

    steps
    1. Open https://www.nvidia.com/en-us/geforce/drivers/ and search for your graphics card. Game Ready and Studio drivers should both work. nvidia manual driver search form
    2. Download the a recent version of your driver nvidia manual driver search form
    3. Run the installer. I skip GeForce Experience to avoid automatic driver updates: nvidia manual driver installer wizard with only NVIVIDA graphics driver selected
    4. I also pick an express install: nvidia manual driver installer wizard with only NVIVIDA graphics driver selected
    5. After the install finishes check you still have graphics and restart your computer

  3. Install WSL2. 3

    steps
    1. Select PowerShell and "Run as administrator" start menu with search for PowerShell with "run as administrator" option visible
    2. Run wsl --install to install the Ubuntu Linux distribution
    3. Run wsl --update to update the Linux kernel version
    4. Restart your computer

Warning

These dependencies take up a decent amount of disk space. I installed WSL on my C:\ drive with ~25GB of space free before running into a dpkg error on the CUDA install step that moving the WSL install to a larger drive fixed (as recommended here).

  1. (Optional) Move WSL to a faster or larger drive. 4

    steps
    1. Select PowerShell and "Run as administrator"
    2. Create a directory on your larger drive. I ran mkdir E:\wsl-backup replace E:\ with your drive.
    3. Run wsl --export Ubuntu E:\wsl-backup\ubuntu.tar
    4. Then run wsl --unregister Ubuntu
    5. Run wsl --import Ubuntu E:\wsl\ E:\wsl-backup\ubuntu.tar to reimport it
    6. Run wsl --list --verbose to confirm that the move completed successfully: terminal showing a wsl list output including a stopped Ubuntu instance

  2. Install CUDA Support for WSL2. 5

    steps

    1. Note: Pascal / 10XX GPUs are optimized for CUDA 11.8.6 Use this link to install that version. cudnn optimization note

    2. Otherwise run the following commands from here to download and install CUDA Toolkit then configure the apt package manager to fetch and trust Nvidia's updates (the .deb file is ~3GB):

    wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.3.1/local_installers/cuda-repo-wsl-ubuntu-12-3-local_12.3.1-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-3-local_12.3.1-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-3-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-3

  3. Install the Anaconda Distribution. 7

    steps
    1. Run curl -O https://repo.anaconda.com/archive/Anaconda3-2023.09-0-Linux-x86_64.sh to download the install script. It is ~1GB.
    2. Run shasum -a 256 Anaconda3-2023.09-0-Linux-x86_64.sh to check the integrity of the installer. Its hash should be 6c8a4abb36fbb711dc055b7049a23bbfd61d356de9468b41c5140f8a11abd851.
    3. Run bash Anaconda3-2023.09-0-Linux-x86_64.sh and accept the prompts to install the distribution.

  4. Install PyTorch. 8

    steps
    1. Run conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia to install PyTorch with CUDA support (use conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia if you installed CUDA 11.8 for step 5.).

  5. Confirm everything works

    steps
    1. Run python -c "import torch;torch.cuda.is_available()" and python -c "import torch;torch.zeros(1).cuda()" to confirm that PyTorch and CUDA are working.

Footnotes

  1. https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl#install-windows-11-or-windows-10-version-21h2 ↩

  2. see also the Nvidia WSL User Guide ↩

  3. see also the Microsoft WSL install docs and Nvidia WSL install docs ↩

  4. https://superuser.com/a/1618643 ↩

  5. Option 1 from https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2 ↩

  6. https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html#abstract ↩

  7. Python dependency resolution can be messy. The Anaconda Distribution bundles a tested set of packages for data science. Check out Anaconda's miniconda distribution to save some disk space https://docs.anaconda.com/free/anaconda/getting-started/distro-or-miniconda/. Check https://repo.anaconda.com/archive/ to find newer versions and hashes. ↩

  8. https://pytorch.org/get-started/locally/#start-locally ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment