Skip to content

Instantly share code, notes, and snippets.

@ngoc-minh-do

ngoc-minh-do/Nvidia-GPU-access-in-Docker.md

Last active August 3, 2025 06:55
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ngoc-minh-do/ffec1853bb298f4e5d444a6534f5aced to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ngoc-minh-do/ffec1853bb298f4e5d444a6534f5aced to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Nvidia-GPU-access-in-Docker.md

Accessing NVIDIA GPU in Docker Containers

1. Install NVIDIA Container Toolkit

To enable GPU access inside Docker containers, install the NVIDIA Container Toolkit on the Docker host:

📄 Official installation guide:
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

2. Verify GPU Access with Docker

After installation, test that the NVIDIA runtime works by running:

docker run   --runtime=nvidia   --gpus all   --rm -it   nvidia/cuda:12.3.1-base-ubuntu20.04 nvidia-smi

This should display GPU usage information from nvidia-smi.

3. Special Case: Docker Running Inside LXC

If your Docker host is itself an LXC container, you might encounter the following error:

stderr: Auto-detected mode as 'legacy' 
nvidia-container-cli: mount error: failed to add device rules: unable to find any existing device filters attached to the cgroup: bpf_prog_query(BPF_CGROUP_DEVICE) failed: operation not permitted: unknown.

✅ Solution:

Edit the config file:

sudo nano /etc/nvidia-container-runtime/config.toml

Set the following option:

no-cgroups = true

Then restart Docker:

sudo systemctl restart docker

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment