There are two options:
- With Docker
- With Mamba/Conda
In either case it is utmost important to find version compatibility between GPU driver version, CUDA compute capability, Pytorch version.
Docker provides an isolated workspace which is ideal to replicate development environment. This works well if I do not want to change my default installations just to test out some crazy Github repos out there. This is learnt from past experiences of sweat and blood. The best place to pick prebuilt docker images which are well tested against Nvidia software layers, including CUDA toolkits, CuDNN, and versions of either Tensorflow or Pytorch.
NVIDIA NGC is the one stop shop that I always rely on: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch/tags.
If I want to use ComfyUI with A100 Nvidia gpu cards, its compute capability is 7.0, quite outdated. There are recent versions of Pytorch that drops support for SM_70. As a result I must check the latest Pytorch version that still supports this compute. By selecting tags and checking their layers in order to know version informations of Python, CUDA toolkit, NVidia driver, and compute capability, among others.
The docker image nvcr.io/nvidia/pytorch:24.06-py3 turns out to be fit for my purpose, with Python=3.10, Pytorch=2.4, TORCH_CUDA_ARCH_LIST=5.2 6.0 6.1 7.0 7.2 7.5 8.0 8.6 8.7 9.0+PTX, CUDA_VERSION=12.5.0.023 CUDA_DRIVER_VERSION=555.42.02 ...
The NVIDIA Driver looks good because it matches with the driver version on host machine:
phong@grasshopper:~/comfyui$ nvidia-smi
Sun Feb 9 16:56:23 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.02 Driver Version: 555.42.02 CUDA Version: 12.5 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
+-----------------------------------------+------------------------+----------------------+
And my old GPU card is also supported by PyTorch 2.4. What a relief!
A Dockerfile simple like this would work:
FROM nvcr.io/nvidia/pytorch:24.06-py3
RUN apt-get update && apt-get install -y sudo xterm && rm -rf /var/lib/apt/lists/*
ENV TERM=xterm-256color
USER ${USERNAME}
WORKDIR /home/${USERNAME}
RUN pip install --upgrade pip
RUN echo 'export PS1="\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ "' >> ~/.bashrc && echo 'export TERM=xterm-256color' >> ~/.bashrc
CMD ["/bin/bash"]
Building the docker image with this simple command:
docker build -t videogen/videocog/comfyui:latest .
Then launch a fresh container
docker run -it --rm --name comfyui --hostname docker --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus all -p 8188:8188 -v $(pwd):/home/workspace videogen/videocog/comfyui:latest
With this container, I map the local port 8188 to the container's port 8188. The idea is once ComfyUI starts, its web app listens on port 8188 and I hook the port from innside the container to the host. Adding an extra layer of complexity, if I am to run ComfyUI remotely, that means I must local port forwarding at the time of SSH-ing. Literally,
ssh -L 8188:<remote-ip-address>:8188 <my-user-name>@<remote-ip-address>
Let's go back to the bash inside the container. Navigate to the mounted space /home/workspace, clone the ComfyUI repository, installed required packages inside requirements.txt. You may want to skip some of the packages that have been installed in the container image, including torch and torchvision.
One last important detail to bear in mind is to set listening address for ComfyUI. To make it accessible outside the running container, I bind the ComfyUI to 0.0.0.0 which accepts incoming requests from any IP addresses.
python main.py --listen 0.0.0.0
This would make it.
Miniconda is famous for creating and managing Python virtual environments. Mamba is a faster version when it comes to resoluting package dependencies and installations. Alternatively one can use Python's own virtual environment. The reason of my preference of Conda/Mamba is that their environments are managed all in one place.
At the time of writting, ComfyUI works best with 3.12 >= Python >= 3.10. Ubuntu 24.04 may come with Python 3.13 but it is risky to use latest versions. To be on the safe side,
mamba create env -n comfyui python=3.12
mamba activate comfyui
As I have CUDA toolkit 12.8 installed, and Nvidia driver 560.35.03, which are quite new, and they are backward compatible, that means I can install "older" versions of PyTorch and other packages, ideally the stable versions.
These quick commands allow me to check the versions:
$nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jan_15_19:20:09_PST_2025
Cuda compilation tools, release 12.8, V12.8.61
Build cuda_12.8.r12.8/compiler.35404655_0
or
$ cat /usr/local/cuda/version.json
{
"cuda" : {
"name" : "CUDA SDK",
"version" : "12.8.20250116"
},
"cuda_cccl" : {
"name" : "CUDA C++ Core Compute Libraries",
"version" : "12.8.55"
},
"cuda_cudart" : {
"name" : "CUDA Runtime (cudart)",
"version" : "12.8.57"
},
This time I test Conda/Mamba on a different machine that has RTX 2080 Ti, with compute capability 7.5, slightly newer than V100.
$ nvidia-smi
Mon Feb 10 09:02:32 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03 Driver Version: 560.35.03 CUDA Version: 12.6 |
|-----------------------------------------+------------------------+----------------------+
The next step is to install Pytorch packages
pip install torch torchvision torchaudio
If is there any error using pip, then make sure which pip points to the Mamba's environment directory, and not using system's pip.
The remaining steps include cloning ComfyUI from Github https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file#manual-install-windows-linux, and any custom nodes, for example CogVideoX-Fun https://github.com/aigc-apps/CogVideoX-Fun/blob/main/comfyui/README.md.
One caveat is the dependencies of ComfyUI to xformers. If in trouble, try to manually compile xformers.