DJI Osmo Action 4 D-Log M (10-bit) to HDR (HLG/BT.2020) batch encoder - outputs MP4s recognized as HDR on iPhone/QuickTime/TVs

encode_dji_oa4_dlogm_hdr_video.sh

#!/bin/bash
# Batch re-encode DJI Osmo Action 4 D-Log M (10-bit) footage to HEVC
# (libx265, crf 30, slow preset) as HDR .mp4 files that iPhones, QuickTime,
# Photos, Apple TV and HDR TVs recognize and play as HDR (HLG).
#
# Source format (Osmo Action 4, "10-bit D-Log M" color mode):
#   4K 29.97 fps HEVC ~90 Mbps, yuv420p10le, container/VUI tagged plain
#   BT.709 limited range, AAC-LC stereo audio, plus DJI extra tracks
#   (djmd/dbgi telemetry, tmcd timecode, embedded cover-art/preview JPEGs).
#
# Why this works — what D-Log M actually is:
#   DJI never published a D-Log M spec, but analysis of DJI's own official
#   "D-Log M to Rec.709" LUT shows its neutral axis is a near-identity curve
#   with only a mild S-contrast (a true log format would need a steep
#   expansion), and that it strongly mixes channels exactly like a
#   wide-gamut -> 709 matrix conversion. Together with the DaVinci Resolve
#   community consensus (interpreting D-Log M as "Rec.2020 / HLG" reads
#   naturally), D-Log M is in practice an HLG-style (ARIB STD-B67) signal in
#   Rec.2020-wide primaries that DJI mislabels as BT.709.
#
#   Retagging the signal verbatim as HLG plays as HDR but looks washed out:
#   DJI's intended rendering includes the mild S-curve their official LUT
#   applies (without it, shadows sit ~1.3 stops brighter/flatter than the
#   DJI look). So this script bakes that official neutral tone curve (the
#   gray axis of DJI's own "D-Log M to Rec.709" LUT, 2023 D-Log M family,
#   embedded below as a 33-point 1D LUT) into the signal. The curve is
#   ~identity above ~60% signal, so the HLG highlight headroom — the top
#   ~2 stops the camera captured — stays fully differentiated for HDR.
#   Chroma-wise, base saturation comes from the Rec.2020 interpretation,
#   which matches DJI's official rendering (verified on real frames from
#   this camera: SSIM vs the DJI-LUT reference 0.94 with the tone curve
#   vs 0.84 verbatim).
#
# Vivid look (default):
#   DJI ships no vivid LUT for D-Log M (the OA4 "vivid" download page
#   serves the same family-wide standard cube), so the vivid recipe is
#   ported from DJI's own Mavic 4 Pro D-Log standard/vivid LUT pair:
#   their vivid = extra S-contrast (gray axis of vivid o standard^-1)
#   plus ~1.44x saturation. Applied at full strength it crushes shadows
#   on non-aerial scenes, so the default here is 50% of DJI's vivid
#   contrast + 1.25x saturation (validated on real frames), with the
#   contrast blended back to the standard curve above ~60% signal so the
#   HDR highlight headroom stays fully differentiated. Knobs:
#     VIVID=0 ./encode_...  -> plain DJI-standard rendering, no extra sat
#     SAT=1.4 ./encode_...  -> override the saturation factor alone
#
# What it does per file:
#   - keeps only the main video + audio streams; audio is copied untouched
#     (drops DJI telemetry tracks and cover art, same as the SDR variant)
#   - applies DJI's neutral tone curve to R'G'B' (lut1d), then re-encodes
#     YCbCr with the bt2020nc matrix instead of the BT.709 matrix the
#     camera wrote
#   - tags the stream BT.2020 primaries / ARIB STD-B67 (HLG) transfer /
#     bt2020nc matrix in both the HEVC VUI and the mp4 colr atom ("9/18/9"),
#     the exact signaling iPhone HLG recordings use; HLG needs no HDR10
#     static metadata, and SDR-only players show HLG's built-in
#     SDR-compatible rendition (matching DJI's official 709 LUT look)
#   - verified via AVFoundation: containsHDRVideo=true (the API QuickTime
#     and Photos use for the HDR badge); 10-bit Main 10 + hvc1 throughout
#   - transfers container metadata with exiftool afterwards (capture dates,
#     camera name, firmware info, source modification time) — verified not
#     to disturb the HDR color atoms
#   - output is .mp4 (same filename, new extension)
#
# Already-encoded files are skipped, so the script can be re-run to resume.
# If an encode fails or is interrupted, the partial output is removed and
# the script stops instead of moving on to the next file.
#
# Encoder settings follow the SDR variant (crf 30 slow, no-sao): measured
# ~13 Mbps on 4K29.97 D-Log M test segments (~7x smaller than source).
# 10-bit is inherent here — never encode HDR 8-bit (banding + no HDR badge).
#
# Usage: encode_dji_oa4_dlogm_hdr_video.sh SRC_DIR DST_DIR
#   SRC_DIR  folder with the source .mp4/.MP4 files (D-Log M mode clips!
#            normal-mode SDR clips would be wrongly boosted — keep separate)
#   DST_DIR  output folder, created if missing (must differ from SRC_DIR)
#
# Recommended invocation for a long run on macOS (prevents idle/system
# sleep while on AC power; keep the lid open or use clamshell mode):
#   caffeinate -is ./encode_dji_oa4_dlogm_hdr_video.sh /path/to/src /path/to/out

# Treat unset variables as errors.
set -u

if [ $# -ne 2 ]; then
  echo "Usage: ${0##*/} SRC_DIR DST_DIR" >&2
  exit 1
fi
SRC="$1"
DST="$2"

if [ ! -d "$SRC" ]; then
  echo "Source folder does not exist: $SRC" >&2
  exit 1
fi
# Outputs keep the source basename, so encoding into the source folder would
# collide with the originals.
if [ "$(cd "$SRC" && pwd)" = "$(cd "$DST" 2>/dev/null && pwd)" ]; then
  echo "SRC_DIR and DST_DIR must be different folders" >&2
  exit 1
fi

# Check required tools are installed before starting a long run.
missing=""
for cmd in ffmpeg ffprobe exiftool; do
  command -v "$cmd" >/dev/null 2>&1 || missing="$missing $cmd"
done
if [ -n "$missing" ]; then
  echo "Missing required tools:$missing" >&2
  echo "Install with: brew install$missing" >&2
  exit 1
fi
# The tone curve needs ffmpeg's lut1d filter (present in normal builds;
# guard against unusual slim builds).
if ! ffmpeg -hide_banner -filters 2>/dev/null | grep -q ' lut1d '; then
  echo "This ffmpeg build lacks the 'lut1d' filter; install ffmpeg via brew" >&2
  exit 1
fi

mkdir -p "$DST"

# Tone curves, written out as 1D LUTs at runtime.
#
# Standard: the gray axis of DJI's official "D-Log M to Rec.709" 3D LUT
# (Mavic 3 Pro / Mini 4 Pro / Osmo Action 4 D-Log M family, 2023-03-24).
# Applied with linear interpolation — the same interpolation DJI's own
# 33-node cube gets — this reproduces DJI's intended neutral rendering
# exactly.
#
# Vivid: the standard curve + 50% of the extra S-contrast measured between
# DJI's Mavic 4 Pro D-Log standard and vivid LUTs, blended back to the
# standard curve above ~60% signal (highlight headroom preserved).
tone_dir=$(mktemp -d) || exit 1
trap 'rm -rf "$tone_dir"' EXIT
STD_LUT="$tone_dir/dlogm_dji_neutral.cube"
cat > "$STD_LUT" <<'EOF'
LUT_1D_SIZE 33
0.000000 0.000000 0.000000
0.002871 0.002871 0.002871
0.011483 0.011483 0.011483
0.027519 0.027519 0.027519
0.053470 0.053470 0.053470
0.084958 0.084958 0.084958
0.119194 0.119194 0.119194
0.155261 0.155261 0.155261
0.192511 0.192511 0.192511
0.230367 0.230367 0.230367
0.269187 0.269187 0.269187
0.308987 0.308987 0.308987
0.348870 0.348870 0.348870
0.388299 0.388299 0.388299
0.423923 0.423923 0.423923
0.456294 0.456294 0.456294
0.487482 0.487482 0.487482
0.519352 0.519352 0.519352
0.553866 0.553866 0.553866
0.590850 0.590850 0.590850
0.629943 0.629943 0.629943
0.668561 0.668561 0.668561
0.705709 0.705709 0.705709
0.740481 0.740481 0.740481
0.774365 0.774365 0.774365
0.807295 0.807295 0.807295
0.840201 0.840201 0.840201
0.872726 0.872726 0.872726
0.903984 0.903984 0.903984
0.933251 0.933251 0.933251
0.960446 0.960446 0.960446
0.983198 0.983198 0.983198
1.000000 1.000000 1.000000
EOF

VIVID_LUT="$tone_dir/dlogm_vivid.cube"
cat > "$VIVID_LUT" <<'EOF'
LUT_1D_SIZE 65
0.000000 0.000000 0.000000
0.001074 0.001074 0.001074
0.002277 0.002277 0.002277
0.005370 0.005370 0.005370
0.008814 0.008814 0.008814
0.013997 0.013997 0.013997
0.018695 0.018695 0.018695
0.025952 0.025952 0.025952
0.034072 0.034072 0.034072
0.045960 0.045960 0.045960
0.059691 0.059691 0.059691
0.075340 0.075340 0.075340
0.091972 0.091972 0.091972
0.109584 0.109584 0.109584
0.127367 0.127367 0.127367
0.145065 0.145065 0.145065
0.162683 0.162683 0.162683
0.180078 0.180078 0.180078
0.197321 0.197321 0.197321
0.215081 0.215081 0.215081
0.233316 0.233316 0.233316
0.251937 0.251937 0.251937
0.270964 0.270964 0.270964
0.290386 0.290386 0.290386
0.309811 0.309811 0.309811
0.329693 0.329693 0.329693
0.349917 0.349917 0.349917
0.368414 0.368414 0.368414
0.387218 0.387218 0.387218
0.405156 0.405156 0.405156
0.422911 0.422911 0.422911
0.440163 0.440163 0.440163
0.457981 0.457981 0.457981
0.476384 0.476384 0.476384
0.494878 0.494878 0.494878
0.514873 0.514873 0.514873
0.535608 0.535608 0.535608
0.557729 0.557729 0.557729
0.580170 0.580170 0.580170
0.605092 0.605092 0.605092
0.629686 0.629686 0.629686
0.653416 0.653416 0.653416
0.676360 0.676360 0.676360
0.697829 0.697829 0.697829
0.718295 0.718295 0.718295
0.736565 0.736565 0.736565
0.754108 0.754108 0.754108
0.770614 0.770614 0.770614
0.786245 0.786245 0.786245
0.801113 0.801113 0.801113
0.815812 0.815812 0.815812
0.830311 0.830311 0.830311
0.844872 0.844872 0.844872
0.859415 0.859415 0.859415
0.874141 0.874141 0.874141
0.888828 0.888828 0.888828
0.904009 0.904009 0.904009
0.918618 0.918618 0.918618
0.933189 0.933189 0.933189
0.946848 0.946848 0.946848
0.960268 0.960268 0.960268
0.971822 0.971822 0.971822
0.983020 0.983020 0.983020
0.991599 0.991599 0.991599
1.000000 1.000000 1.000000
EOF

# Look selection: VIVID=1 (default) uses the vivid curve + 1.25x saturation;
# VIVID=0 uses DJI's plain standard rendering. SAT overrides the saturation
# factor (applied around BT.2020 luma on nonlinear R'G'B' via
# colorchannelmixer; 1.0 = colorimetric, DJI's own vivid delta is ~1.44).
VIVID="${VIVID:-1}"
if [ "$VIVID" = "0" ]; then
  TONE_LUT="$STD_LUT"
  SAT="${SAT:-1.0}"
else
  TONE_LUT="$VIVID_LUT"
  SAT="${SAT:-1.25}"
fi
VF="lut1d=file=$TONE_LUT:interp=linear"
if [ "$SAT" != "1.0" ] && [ "$SAT" != "1" ]; then
  CCM=$(LC_ALL=C awk -v s="$SAT" 'BEGIN{
    wr=0.2627; wg=0.6780; wb=0.0593; k=1-s;
    printf "colorchannelmixer=rr=%.4f:rg=%.4f:rb=%.4f:gr=%.4f:gg=%.4f:gb=%.4f:br=%.4f:bg=%.4f:bb=%.4f",
      s+wr*k, wg*k, wb*k, wr*k, s+wg*k, wb*k, wr*k, wg*k, s+wb*k}')
  VF="$VF,$CCM"
fi
VF="$VF,scale=out_color_matrix=bt2020nc:out_range=tv,format=yuv420p10le"
VF="$VF,setparams=colorspace=bt2020nc:color_primaries=bt2020:color_trc=arib-std-b67:range=tv"

# Collect sources: *.mp4 in any case (DJI names files DJI_*.MP4), no error
# if none.
shopt -s nullglob nocaseglob
files=("$SRC"/*.mp4)
shopt -u nullglob nocaseglob

n=${#files[@]}
if [ "$n" -eq 0 ]; then
  echo "No .mp4/.MP4 files found in $SRC" >&2
  exit 1
fi
i=0

# Total length of the whole batch, shown alongside each file's own duration.
total_dur=$(for f in "${files[@]}"; do \
    ffprobe -v error -show_entries format=duration -of csv=p=0 "$f"; \
  done | awk '{s+=$1} END{printf "%d:%02d:%02d", s/3600, (s%3600)/60, s%60}')

for f in "${files[@]}"; do
  i=$((i + 1))
  # Output as .mp4 (same basename, extension normalized to lowercase .mp4).
  name="${f##*/}"
  base="${name%.*}"
  out="$DST/$base.mp4"

  # Skip files that already have an output — makes the script resumable.
  # A completed file is never touched; partial files are deleted below on
  # failure, so anything present here is a finished encode.
  if [ -e "$out" ]; then
    echo "[$i/$n] Skipping (already exists): $out"
    continue
  fi

  # Sanity check: D-Log M sources are tagged plain bt709. A clip already
  # tagged HLG/PQ/2020 is not D-Log M — converting it here would be wrong.
  src_trc=$(ffprobe -v error -select_streams v:0 \
    -show_entries stream=color_transfer -of csv=p=0 "$f")
  if [ "$src_trc" != "bt709" ] && [ "$src_trc" != "unknown" ] && [ -n "$src_trc" ]; then
    echo "[$i/$n] Skipping (transfer '$src_trc', not a D-Log M/BT.709 source): $name"
    continue
  fi

  dur=$(ffprobe -v error -pretty -show_entries format=duration -of csv=p=0 "$f")
  echo "[$i/$n] Encoding: $name ($(du -h "$f" | cut -f1), $dur of $total_dur total)"

  # -map 0:v:0 -map 0:a:0  keep only the main video and audio streams;
  #                        drops the DJI djmd/dbgi telemetry tracks and the
  #                        embedded cover-art JPEG (a timecode track is still
  #                        added automatically from the source's timecode
  #                        metadata)
  # lut1d                  tone curve on R'G'B' (decoded with the source's
  #                        bt709 matrix): DJI's neutral rendering, plus the
  #                        vivid contrast when VIVID=1 — without a tone
  #                        curve the HDR presentation looks washed out (see
  #                        header). ~identity toward the top, so HDR
  #                        highlight headroom is preserved
  # colorchannelmixer      the vivid saturation boost (skipped at SAT=1.0)
  # scale + format         re-encode R'G'B' to 10-bit YCbCr with the
  #                        bt2020nc matrix (instead of the camera's bt709)
  # setparams              stamps the true color properties (HLG =
  #                        arib-std-b67) on the frames; x265 builds its VUI
  #                        from these props
  # -c:v libx265           software HEVC encoder, Main 10 via yuv420p10le
  # -preset slow -crf 30   same size/quality tradeoff as the SDR variant
  # no-sao=1               disable the SAO in-loop filter (detail retention)
  # colorprim/transfer/colormatrix/range in -x265-params and the -color_*
  #                        output options: belt-and-suspenders BT.2020 + HLG
  #                        signaling in both the bitstream VUI and the mp4
  #                        colr atom — players key their HDR mode off these
  # -tag:v hvc1            Apple-compatible codec tag (default hev1 won't
  #                        play in QuickTime/Photos)
  # -c:a copy              pass the AAC audio through untouched
  # -movflags +faststart   moov atom up front for instant playback start
  ffmpeg -nostdin -loglevel error -stats -i "$f" -map 0:v:0 -map 0:a:0 \
    -vf "$VF" \
    -c:v libx265 -preset slow -crf 30 \
    -x265-params "no-sao=1:colorprim=bt2020:transfer=arib-std-b67:colormatrix=bt2020nc:range=limited" \
    -color_primaries bt2020 -color_trc arib-std-b67 -colorspace bt2020nc \
    -color_range tv \
    -tag:v hvc1 \
    -c:a copy \
    -movflags +faststart \
    "$out"

  # Stop on any failure, including Ctrl+C (ffmpeg exits nonzero on SIGINT).
  # Delete the partial output so the next run resumes at this file.
  status=$?
  if [ "$status" -ne 0 ]; then
    echo "Encode failed or interrupted (exit $status): $f" >&2
    rm -f "$out"
    exit "$status"
  fi

  # Assert the HDR signaling landed (guards against ffmpeg builds that drop
  # frame color properties) — without 9/18/9 the file would play SDR.
  # ffprobe prints these in its own fixed order: space, transfer, primaries.
  got=$(ffprobe -v error -select_streams v:0 \
    -show_entries stream=color_space,color_transfer,color_primaries \
    -of default=noprint_wrappers=1:nokey=1 "$out" | paste -sd, -)
  if [ "$got" != "bt2020nc,arib-std-b67,bt2020" ]; then
    echo "HDR color signaling verification failed ($got): $out" >&2
    rm -f "$out"
    exit 1
  fi

  # Transfer container metadata from the source: capture dates (movie, track
  # and media level — ffmpeg writes them as zeros), camera name (Encoder:
  # "DJI OsmoAction4"), original SD-card file path, DJI firmware Category
  # string, and the source file's modification time.
  # Excluded: embedded preview JPEGs (deliberately stripped in the encode) and
  # the old container's brand tags. The DJI telemetry track is a data stream,
  # not tags — it is dropped by the encode and cannot be transferred.
  # Verified: this does not touch the colr atom, the HDR tags survive.
  # exiftool preserves the faststart layout (moov stays at the front).
  echo "[$i/$n] Transferring metadata: $name"
  if ! exiftool -q -overwrite_original -api LargeFileSupport=1 \
    -TagsFromFile "$f" -All:All \
    --ItemList:CoverArt --ItemList:PreviewImage --ItemList:ThumbnailImage \
    --Keys:MajorBrand --Keys:MinorVersion --Keys:CompatibleBrands \
    "-FileModifyDate<FileModifyDate" "$out"; then
    echo "Metadata transfer failed: $out (encode kept)" >&2
    exit 1
  fi

  echo "[$i/$n] Done: $out ($(du -h "$out" | cut -f1))"
done

echo "All files encoded."