NVIDIA OpenGL Issue Note

nvidia_opengl_issue_note.md

Problem

Encountered slow and laggy performance in the Unreal Engine within my NVIDIA CUDA container following a driver update. The issue is suspected to be related to OpenGL not utilizing the GPU but instead relying on the CPU. This inference is drawn from the output of the glxheads command within the container, indicating the use of Mesa/X.org and llvmpipe for rendering:

GL_VERSION:  3.1 Mesa 21.2.6
GL_VENDOR:   Mesa/X.org
GL_RENDERER: llvmpipe (LLVM 12.0.0, 256 bits)

In contrast, running the same command on the host PC reveals NVIDIA usage:

GL_VERSION:  4.6.0 NVIDIA 545.29.06
GL_VENDOR:   NVIDIA Corporation
GL_RENDERER: NVIDIA GeForce RTX 3090/PCIe/SSE2

Additionally, the investigation extends to employing the ldd command on the glxheads executable, unveiling the libraries in use.

libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1
libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
...

Solution

After researching online and attempting various solutions, I found success in mounting host libraries to the container. To preserve the original container libraries essential for package building, a separate folder is created for hosting the additional libraries. The LD_LIBRARY_PATH is then set to this folder to ensure the correct library usage.

Here's the script I used for this configuration:

docker run -it --runtime=nvidia --gpus all -e DISPLAY -e XAUTHORITY -e SDL_VIDEODRIVER=x11 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /usr/lib/x86_64-linux-gnu:/usr/lib/docker_host:ro -e LD_LIBRARY_PATH='/usr/lib/docker_host:$LD_LIBRARY_PATH' -v /media/ssd/simulation_server/:/home/carla/ --net=host --name [CONTAINER_NAME] [IMAGE_NAME]

This script ensures the necessary environment variables, GPU runtime, and library paths are configured appropriately for optimal Unreal Engine performance within the NVIDIA CUDA container.

Additional Note

In case we need to update nvcc (Nvidia Cuda Compiler), we can just copy the correct version of cuda into /usr/local/cuda. The nvcc will be located in /usr/local/cuda/bin/nvcc. We can set CUDA_HOME or CUDA_PATH environment variables to that cuda folder.