Extracting HDR Gain Map from iOS 14.1+ (iPhone 12+) photos

HDRGainMap.swift

import UIKit
import MobileCoreServices.UTCoreTypes

if #available(iOS 14.1, *) {
    let input = Bundle.main.url(forResource: "IMG_0037", withExtension: "HEIC")!
    let output = FileManager().temporaryDirectory.appendingPathComponent("IMG_0037.GAIN_MAP.BMP")
    let source = CGImageSourceCreateWithURL(input as CFURL, nil)!

    // urn:com:apple:photo:2020:aux:hdrgainmap
    let dataInfo = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeHDRGainMap)! as Dictionary
    let data = dataInfo[kCGImageAuxiliaryDataInfoData] as! Data
    let description = dataInfo[kCGImageAuxiliaryDataInfoDataDescription]! as! [String: Int]
    let size = CGSize(width: description["Width"]!, height: description["Height"]!)
    
    let ciImage = CIImage(bitmapData: data, bytesPerRow: description["BytesPerRow"]!, size: size, format: .L8, colorSpace: nil)
    let cgImage = CIContext().createCGImage(ciImage, from: CGRect(origin: CGPoint(x: 0, y: 0), size: size))!
    
    let destRef = CGImageDestinationCreateWithURL(output as CFURL, kUTTypeBMP, 1, nil)!
    CGImageDestinationAddImage(destRef, cgImage, [:] as CFDictionary)
    CGImageDestinationFinalize(destRef)

    print(output)
}

I don’t know that I’d quite say ISO HDR is the same as a video still frame, not in a literal sense anyhow. Video tends to define a 100 nits white point, photography is using 203. There are certainly many commonalities (sadly including the lack of a common standard for tone mapping).

The “ISO HDR” refers to encoding of an HDR image. There is another ISO draft for gain maps. These are complimentary, as an ISO gain map can use an ISO HDR as the base image. You can use one without the other, but they work best together (if the base image in a gain map is HDR).

Gain maps are the obvious way forward. They offer a much better experience for viewing on any display which lacks the full HDR headroom encoded in the image. It’s vastly better than tone mapping.

JPG gain map is the obvious choice right now. Browsers representing ~75% of viewers (once fully updated) support it on systems with HDR display. And backwards compatibility is 100% (if your browser / image software doesn’t support HDR or gain maps, you gracefully fall back to a nice SDR).

However, JPG requires large files for high quality. There is a draft AVIF gain map standard (Adobe has shared sample files and Chrome/Brave support it). This is clearly the way things “should” go.

I say “should” because best doesn’t always win. As of now, Microsoft Edge (and the file explorer) doesn’t support any AVIF, and that needs to change for AVIF to be viable. Assuming that happens (they have support under a dev flag), we still need an encoder and more browsers to add support. It’s part of the same draft ISO proposal for gain maps and I hope to see it happen.

Another theoretical candidate would be JXL (JPEG XL). As browser support is zero and Chrome pulled support, it doesn’t seem promising anytime soon.

If it were all up to me: we’d use JPG gain maps for now, transition to AVIF gain maps as soon as MS Edge supports AVIF (encoded with an SDR base image for compatibility), then finally get to the ideal AVIF encoded with an HDR base image when support for AVIF gain maps is widely available.

The other thing I’d like to see is encoder support to let the artist provide the SDR rendition as well. This has many benefits, which I outlined in this feature request for Photoshop (please upvote it if you agree): https://community.adobe.com/t5/photoshop-ecosystem-ideas/allow-full-user-control-of-the-sdr-rendition-in-an-hdr-gain-map-export/idi-p/14205440#M19464

Although I'm late to the party, I must say this thread is the most informative and inspiring resource on the entire internet regarding this specific topic. I've truly learned a lot from all the discussions above and want to sincerely thank every participant and contributor. Inspired by the discussion, it seems I've figured out how to use Apple's public APIs to generate a JPEG with a gain map that renders properly on both iPhone and Mac. It can also be inspected by Adobe's demo app and other third-party software like HoneyView. I've shared the code with detailed documentation, hoping it will be helpful to others: https://github.com/grapeot/AppleJPEGGainMap

Some more information from the apple's developer site: https://developer.apple.com/videos/play/wwdc2024/10177/