Skip to content

Instantly share code, notes, and snippets.

@Frajder

Frajder/installing-cuda-support-for-docker-on-kali-linux.md

Created December 12, 2024 20:29
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Frajder/711b79f878951aa35bc33ef4989474d0 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Frajder/711b79f878951aa35bc33ef4989474d0 to your computer and use it in GitHub Desktop.
Download ZIP
Things that come in handy for the next time.
Raw
installing-cuda-support-for-docker-on-kali-linux.md

Installing CUDA Support for Docker on Kali Linux

Table of Contents

Prerequisites

Add Correct NVIDIA Docker Repository

Install NVIDIA Container Toolkit

Configure Docker Daemon

Restart Docker Service

Verify Installation

Prerequisites

Ensure the following are already installed and properly configured on your system:

  • Docker: Confirmed by running docker --version.

    docker --version
# Example Output:
# Docker version 26.1.5+dfsg1, build a72d7cd

  • NVIDIA Drivers: Confirmed by running nvidia-smi.

    nvidia-smi
# Example Output:
# +-----------------------------------------------------------------------------+
# | NVIDIA-SMI 535.216.03 Driver Version: 535.216.03 CUDA Version: 12.2       |
# |-----------------------------------------------------------------------------|
# | ... (additional GPU information) ...                                      |
# +-----------------------------------------------------------------------------+

Add Correct NVIDIA Docker Repository

Kali Linux is based on Debian, so we will use NVIDIA's Debian-based repositories to ensure compatibility.

  1. Install Required Packages:

    Ensure curl, gnupg, and lsb-release are installed.

    sudo apt update
sudo apt install -y curl gnupg lsb-release

  2. Add NVIDIA's GPG Key:

    Import the GPG key and store it in the keyrings directory.

    curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

  3. Add the NVIDIA Docker Repository:

    Create a new repository list file with the correct Debian codename and architecture.

    DISTRO=$(lsb_release -cs)
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/deb/amd64 /" | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

    Note: Replace amd64 with your system's architecture if different. For most modern systems, amd64 is correct.

  4. Update Package Lists:

    Refresh the package index to include the new NVIDIA repository.

    sudo apt update

Install NVIDIA Container Toolkit

Install the NVIDIA Container Toolkit, which enables Docker to utilize NVIDIA GPUs.

sudo apt install -y nvidia-container-toolkit

Installation Summary:

  • Downloaded Packages:

    • libnvidia-container1
    • libnvidia-container-tools
    • nvidia-container-toolkit-base
    • nvidia-container-toolkit

  • Installation Output:

    • Packages are successfully installed, and necessary services are configured.

Configure Docker Daemon

Configure Docker to use the NVIDIA runtime by default.

  1. Backup Existing Docker Daemon Configuration:

    It's good practice to back up the current configuration before making changes.

    sudo cp /etc/docker/daemon.json /etc/docker/.daemon.json.bck

  2. Update Docker Daemon Configuration:

    Set the NVIDIA runtime in Docker's daemon.json.

    sudo tee /etc/docker/daemon.json > /dev/null <<EOF
{
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  },
  "default-runtime": "nvidia"
}
EOF

    Explanation:

    • runtimes.nvidia: Defines the NVIDIA runtime.
    • default-runtime: Sets NVIDIA as the default runtime for Docker containers.

  3. Verify daemon.json Configuration:

    Ensure the JSON syntax is correct.

    sudo cat /etc/docker/daemon.json

    It should display:

    {
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  },
  "default-runtime": "nvidia"
}

Restart Docker Service

Apply the new Docker daemon configuration by restarting the Docker service.

sudo systemctl restart docker

Note: If you encounter issues restarting Docker, ensure that the daemon.json file syntax is correct. You can validate the JSON format using tools like jq.

Verify Installation

Ensure that Docker can access your NVIDIA GPU by running a test container.

  1. Test with NVIDIA CUDA Image:

    Use a Correct Image Tag:

    The error you encountered was due to a typo and possibly incorrect image tags. Ensure you use the correct nvidia-smi command and valid CUDA image tags.

    docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu20.04 nvidia-smi

    Common Issues:

    • Image Not Found: Ensure you are using the correct image tag. Browse available tags on NVIDIA CUDA Docker Hub.
    • Typographical Errors: Ensure nvidia-smi is spelled correctly.

    Example Successful Output:

    +-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.216.03    Driver Version: 535.216.03    CUDA Version: 12.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce RTX 3060    Off  | 00000000:3B:00.0  On |                  N/A |
| 53%   51C    P2    39W / 170W   |    971MiB / 12288MiB |     18%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

  2. Alternative Verification Using Ubuntu Image:

    Since your ubuntu image with nvidia-smi worked, further verification shows that the setup is correct.

    docker run --rm --gpus all ubuntu nvidia-smi

    Successful Output:

    Thu Dec 12 19:40:43 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.216.03    Driver Version: 535.216.03    CUDA Version: 12.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce RTX 3060    Off  | 00000000:3B:00.0  On |                  N/A |
| 53%   51C    P2    39W / 170W   |    962MiB / 12288MiB |     21%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Summary

By following these corrected steps, you should have successfully enabled CUDA support for Docker on your Kali Linux system. Here's a quick recap in fife steps:

  1. Added the correct Debian-based NVIDIA Docker repository.
  2. Installed the NVIDIA Container Toolkit.
  3. Configured the Docker daemon to use the NVIDIA runtime by default.
  4. Restarted the Docker service to apply changes.
  5. Verified the installation using correct Docker image tags and commands.

Additional Resources

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment