Custom obs-backgroundremoval build with support for NVIDIA cuda 12 on linux

README.md

The below CMakeUserPreset.json can be used to successfully build and install obs-backgroundremoval with a version of the ONNXRUNTIME that supports current NVIDIA CUDA 12.

It makes use of changes introduced in my fork with commit 2c658d8a. EDIT: The PR has now been merged into master

Due to changes upstream with ONNXRUNTIME, the plugin now depends on libcudnn for CUDA which is provided by nvidia-cudnn.

To make use of this:

1. Clone from my fork on the feature/custom-onnxruntime branch. Clone locaal-ai/obs-backgroundremoval
2. Place the below CMakeUserPresets.json file at the projects root
3. Run the following:

   rm -rf build_x86_64 release \
     && cmake --preset cuda12 \
     && cmake --build --preset cuda12 \
     && cmake --build build_x86_64 -t package \
     && sudo apt install ./release/obs-backgroundremoval-1.2.0-x86_64-linux-gnu.deb

4. Profit

Update 9/15/2025: Changed debian package dependencies for more accurate cuda selection

CMakeUserPresets.json

{
  "version": 3,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 28,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": "cuda12",
      "displayName": "Linux x86_64 w/ cuda12",
      "description": "Build for Linux x86_64 against CUDA 12",
      "inherits": ["template"],
      "binaryDir": "${sourceDir}/build_x86_64",
      "condition": {
        "type": "equals",
        "lhs": "${hostSystemName}",
        "rhs": "Linux"
      },
      "generator": "Ninja",
      "warnings": {"dev": true, "deprecated": true},
      "cacheVariables": {
        "QT_VERSION": "6",
        "CMAKE_INSTALL_PREFIX": "/usr",
        "CMAKE_BUILD_TYPE": "Release",
        "CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, libcuda1, nvidia-cuda-toolkit (>= 12~) | cuda-toolkit-12-9 | cuda-toolkit-12-8 | cuda-toolkit-12-6 | cuda-toolkit-12-5 | cuda-toolkit-12-4 | cuda-toolkit-12-3, nvidia-cuda-toolkit (<< 13~) | cuda-toolkit-12-9 | cuda-toolkit-12-8 | cuda-toolkit-12-6 | cuda-toolkit-12-5 | cuda-toolkit-12-4 | cuda-toolkit-12-3, nvidia-cudnn | cudnn",
        "CUSTOM_ONNXRUNTIME_VERSION": "1.21.0",
        "CUSTOM_ONNXRUNTIME_URL": "https://github.com/microsoft/onnxruntime/releases/download/v1.21.0/onnxruntime-linux-x64-gpu-1.21.0.tgz",
        "CUSTOM_ONNXRUNTIME_HASH": "SHA256=ef37a33ba75e457aebfd0d7b342ab20424aa6126bc5f565d247f1201b66996cf"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "cuda12",
      "configurePreset": "cuda12",
      "displayName": "Linux x86_64 w/ cuda12",
      "description": "Linux build for x86_64 against CUDA 12",
      "configuration": "Release"
    }
  ]
}

Managed to use this, but with small changes: "CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, nvidia-driver-580, cudnn9-cuda-12, nvidia-cuda-toolkit", on my Ubuntu 24.04 PC

Managed to use this, but with small changes: "CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, nvidia-driver-580, cudnn9-cuda-12, nvidia-cuda-toolkit", on my Ubuntu 24.04 PC

Hey @tomekjarosik thanks for that! Ubuntu packages their nvidia drivers in a non-debian way so that is a good catch.

Can you try building with:

"CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, libcuda1, (nvidia-cuda-toolkit (>= 12~) | cuda-toolkit-12), (nvidia-cuda-toolkit (<< 13~) | cuda-toolkit-12), (libcudnn9 | cudnn9-cuda-12)"

This in theory will gate the package to specifically require cuda 12 (which can be installed alongside 13). I had to reinstall 12 with the latest nvidia drivers from the developer repo to have the package work.

I believe switching the nvidia-driver requirement to libcuda1 should be platform agnostic

EDIT:

Actually I think cudnn can be based on cuda 13 and work just fine as long as cuda 12 is installed....

Try this:

"CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, libcuda1, (nvidia-cuda-toolkit (>= 12~) | cuda-toolkit-12), (nvidia-cuda-toolkit (<< 13~) | cuda-toolkit-12), nvidia-cudnn | cudnn"

Unfortunately, I got:

tomek@uextremepc:~/src/background-removal/obs-backgroundremoval$ sudo apt install ./release/obs-backgroundremoval-1.2.0-x86_64-linux-gnu.deb
Reading package lists... Error!
E: Problem parsing dependency 20 of obs-backgroundremoval:amd64=1.2.0
E: Error occurred while processing obs-backgroundremoval (NewVersion2)
E: Problem with MergeList /home/tomek/src/background-removal/obs-backgroundremoval/release/obs-backgroundremoval-1.2.0-x86_64-linux-gnu.deb
E: The package lists or status file could not be parsed or opened.

I tried some other variations and broke my plugin setup. Do you have any recommendation for best quality? So far I tried with GPU - CUDA + rvm_resnet50_fp32.onnx model + support with base 640x384 (https://github.com/tomekjarosik/obs-backgroundremoval/commits/dev/tjarosik/rvm50/). Not sure if I should aim for some other model + TensorRT

@tomekjarosik sorry about that, I had some extra parenthesis in there. I have now tested with the following which I tested on debian. The major change is that in order to specifically gate cuda to version 12 without forcing the removal of cuda 13 I also needed to specify the specific minor versions. What a mess!

"CPACK_DEBIAN_PACKAGE_DEPENDS": "obs-studio, libcuda1, nvidia-cuda-toolkit (>= 12~) | cuda-toolkit-12-9 | cuda-toolkit-12-8 | cuda-toolkit-12-6 | cuda-toolkit-12-5 | cuda-toolkit-12-4 | cuda-toolkit-12-3, nvidia-cuda-toolkit (<< 13~) | cuda-toolkit-12-9 | cuda-toolkit-12-8 | cuda-toolkit-12-6 | cuda-toolkit-12-5 | cuda-toolkit-12-4 | cuda-toolkit-12-3, nvidia-cudnn | cudnn"

I honestly haven't explored more onnx models beyond the v1.21.0 in the above config as it works solid for me!

I should note, I haven't tested if the nvidia-cuda-toolkit package will force a downgrade from cuda 13 but I don't think there is actually a cuda 13 version released for this package yet. So it should be a non-issue for now but might need to be adjusted in the future.