Fix Plex transcoding HDR videos very slowly when burning subtitles in at the same time

fix-plex-hdr-subtitles.md

This workaround should no longer be necessary with newer Plex versions: https://forums.plex.tv/t/subtitle-burn-in-improvement-test-build/884481

---

Old content left here for reference:

What?

For some reason, although Plex supports hardware-accelerated HDR tone mapping, Plex decides not to use it when subtitles need to be burned in. This results in extremely slow transcoding on hardware that should be able to keep up using hardware acceleration. I've noticed this issue on my server with Intel QuickSync, and this fix is made specifically for that case.

Before the fix, the SW tone mapping was only able to do 6-7 fps on a i5 7400T for an HDR 4K->1080p transcode. With the fix, that same transcode happens with HW tone mapping, and goes up to 82 fps. Even for HDR 4K->4K, this CPU is able to do 35 fps using QuickSync, thus keeping up with at least one video stream.

How do I fix this?

By wrapping "Plex Transcoder" with a script which replaces the software tone mapping with hardware tone mapping.

A filter configuration for software tone mapping looks like this: [0:0]scale=w=1920:h=1080:force_divisible_by=4[0];[0]format=p010,tonemap=mobius[1];[1]format=pix_fmts=nv12[2];[2]inlineass=font_scale=1.000000:font_path=/usr/lib/plexmediaserver/...[3];[3]hwupload[4]

A filter configuration for hardware tone mapping looks like this: [0:0]scale_vaapi=w=1920:h=1080:format=p010[0];[0]hwmap=derive_device=opencl,tonemap_opencl=tonemap=mobius:format=nv12:m=bt709:p=bt709:r=tv,hwdownload[1];[1]format=pix_fmts=nv12[2];[2]inlineass=font_scale=1.000000:font_path=/usr/lib/plexmediaserver/...[3];[3]hwupload[4]

For the linuxserver.io Plex docker image, I've attached a script which can be added into custom-cont-init.d (by using a bind mount) that fixes the issue described here. Use at your own risk, it may cause issues in other transcoding scenarios which I didn't test yet.

fix-plex-hdr-subtitles.sh

#!/bin/bash -ex
cd /usr/lib/plexmediaserver
if [ ! -f "Plex Transcoder.before-hdr-subtitles-fix" ]; then
  mv "Plex Transcoder" "Plex Transcoder.before-hdr-subtitles-fix"
fi
cat >"Plex Transcoder" <<"EOF"
#!/usr/bin/env -S perl -l
use warnings;
use strict;
my $found = 0;
for my $arg (@ARGV) {
  my $updated = $arg =~ s/(\[[\w:]+\])scale=(w=\d+:h=\d+)[^[]+(\[[\w:]+\]);\3format=p010,tonemap=(\w+)(\[[\w:]+\]);/$1scale_vaapi=$2:format=p010$3;$3hwmap=derive_device=opencl,tonemap_opencl=tonemap=$4:format=nv12:m=bt709:p=bt709:r=tv,hwdownload,format=pix_fmts=nv12$5;/r;
  if ($updated ne $arg) {
    print("Enabled HW tone mapping:");
    print("Original: $arg");
    print("Updated:  $updated");
    $arg = $updated;
    $found = 1;
    next;
  }
  $updated = $arg =~ s/(\[[\w:]+\])scale=(w=\d+:h=\d+)[^,[]+/$1hwupload,scale_vaapi=$2,hwdownload/r;
  if ($updated ne $arg) {
    print("Enabled HW scaling:");
    print("Original: $arg");
    print("Updated:  $updated");
    $arg = $updated;
    $found = 1;
    next;
  }
}
if ($found) {
  if (!defined $ENV{OCL_ICD_VENDORS}) {
    $ENV{OCL_ICD_VENDORS} = '/config/Library/Application Support/Plex Media Server/Cache/cl-icds-linux-x86_64';
    $ENV{cl_cache_dir} = '/config/Library/Application Support/Plex Media Server/Cache/Shaders/icr-97-linux-x86_64/';
  }
  unshift(@ARGV, "-hwaccel_output_format:0", "vaapi");
}
unshift(@ARGV, "/usr/lib/plexmediaserver/Plex Transcoder.before-hdr-subtitles-fix");
exec { $ARGV[0] } @ARGV;
EOF
chmod +x "Plex Transcoder"

I ran this script inside the official PMS docker container, but plex just dies as soon as it tries to transcode

This is for the Linux server Plex container. https://github.com/linuxserver/docker-plex

For the linuxserver.io Plex docker image, I've attached a script which can be added into custom-cont-init.d (by using a bind mount)

FYI, I have incorporated this functionality into the PMS test build found here, please come give it a try and let me know if everything works ok.

great thanks! could you add a docker image if possible?

…

On Tue, 6 Aug 2024 at 10:23 AM, chrisdecker08 ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ FYI, I have incorporated this functionality into the PMS test build found here <https://forums.plex.tv/t/subtitle-burn-in-improvement-test-build/884481/2>, please come give it a try and let me know if everything works ok. — Reply to this email directly, view it on GitHub <https://gist.github.com/bogsen/37cbad60f6c10cdaddf3b07a56e06147#gistcomment-5144831> or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEB3SF4NAKWLLAPK73ZZ5UDZQAJOZBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVEYTEMJRGE4TQOJZU52HE2LHM5SXFJTDOJSWC5DF> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

Unfortunately, we do not generate docker images for test builds. You'll have to wait until the work lands in beta.

FYI, I have incorporated this functionality into the PMS test build found here, please come give it a try and let me know if everything works ok.

I just installed it and gave a quick pre-test before and after of both actual real world performance issue, htop and intel_gpu_top, and after installing there is literally a 10 times performance boost. It's gone from not being able to do one 4k HDR -> 1080p SDR with PGS burnt to maybe doing 3-4?

Awesome. Thank YOU!!!

Thanks @chrisdecker08, nice to see this finally land! (I'm not currently using Plex so I can't test, but I've added a link to your forum post in the gist.)