Function to scale any image to the pixel values of [-1, 1] for GAN input

scale_images.py

"""
scale_images.py
Function to scale any image to the pixel values of [-1, 1] for GAN input.

Author: liuhh02 https://machinelearningtutorials.weebly.com/
"""
from PIL import Image
import numpy as np
from os import listdir

def normalize(arr):
    ''' Function to scale an input array to [-1, 1] '''
    arr_min = arr.min()
    arr_max = arr.max()
    # Check the original min and max values
    print('Min: %.3f, Max: %.3f' % (arr_min, arr_max))
	arr_range = arr_max - arr_min
	scaled = np.array((arr-arr_min) / float(arr_range), dtype='f')
	arr_new = -1 + (scaled * 2)
    # Make sure min value is -1 and max value is 1
    print('Min: %.3f, Max: %.3f' % (arr_new.min(), arr_new.max()))
    return arr_new

# path to folder containing images
path = './directory/to/image/folder/'

# loop through all files in the directory
for filename in listdir(path):
	# load image
	image = Image.open(path + filename)
	# convert to numpy array
	image = np.array(image)
	# scale to [-1,1]
	image = normalize(image)

Hey @liuhh02 I have a few questions about normalisation of tifs if you have time to help me.
When I use this script my new min/max values are really low and does not come close to the edges of the -1,1 range.
Min: 1345.750, Max: 1439.853
Min: -0.034, Max: 0.034

When I re open my tifs in a viewing software(Qgis) it looks perfectly normalized but in your script notes it says to "Make sure ... max value is 1".
Would this narrow range be a problem for pix2pix to handle?

Best,
Johan

Hey @liuhh02 I have a few questions about normalisation of tifs if you have time to help me.
When I use this script my new min/max values are really low and does not come close to the edges of the -1,1 range.
Min: 1345.750, Max: 1439.853
Min: -0.034, Max: 0.034

When I re open my tifs in a viewing software(Qgis) it looks perfectly normalized but in your script notes it says to "Make sure ... max value is 1".
Would this narrow range be a problem for pix2pix to handle?

Best,
Johan

Hello Johan @JohanRaniseth, the reason why the values do not come close to the [-1, 1] range is because the assumption of the script is that the original minimum value would be 0. But no worries, I have changed the code so it can work with any minimum value. Try it out, it should work now!

Also, the narrow range would not be a problem, so don't worry about it.

Thank you so much for the help!

@liuhh02 is there any chance you would be willing to upload your pix2pix files to an open repository? I've had the same issue that you had here with ValueError: Too many dimensions: 3 > 2. but I'm struggling with finding out where the change should be made. I was thinking maybe in the util.py function tensor2im where it looks like the image array is being shaped but I didn't have much luck.

Best,
Johan

@liuhh02 is there any chance you would be willing to upload your pix2pix files to an open repository? I've had the same issue that you had here with ValueError: Too many dimensions: 3 > 2. but I'm struggling with finding out where the change should be made. I was thinking maybe in the util.py function tensor2im where it looks like the image array is being shaped but I didn't have much luck.

Best,
Johan

Are you working with greyscale tiff images?

I'm working with 1 band tiffs which I think are greyscale.
In a viewing software one of my tiffs looks like this where the black has the value 1311 and white has the value 1376.
I've used your scripts to align and normalize my tiffs with [1311, 1376] normalized to [-1,1]

On a sidenote, for the normalizing script I found out that I needed to use
scaled = np.array((arr-arr_min) / float(arr_range), dtype='f') instead of
scaled = np.array((arr-arr_min) / float(arr_range), dtype=float)
since it seems PILs Image.Open('file.tif') does not manage to open files created by cv2 when writing float64 arrays to tiff. Worked when i tested for 'f' and 'float32'.

@JohanRaniseth Thanks for pointing it out!

I didn't manage to get the pix2pix to work out-of-the-box using the pytorch source code. Instead, I referred to Jason Brownlee's code for a keras implementation of pix2pix here. The code worked wonderfully for me, with only minor changes like changing the image_shape under define_generator from (256,256,3) to (256,256,1). Aside from that, you also need to change the scaling of the image under the function load_real_samples, from X1 = (X1 - 127.5) / 127.5 to X1 = normalize(X1) (from the function given in this gist).

Give it a try and tell me how it goes!

So an update for people who might come looking here. Got the PyTorch pix2pix implementation to work with a few changes. I ended up doing all image pre-processing and post-processing outside the PyTorch implementation so that I just inserted my [-1,1] normalized images and got out [-1,1] normalized output images. You can probably do this inside the pix2pix code if you want.

In util.py the function tensor2im transforms a tensor to an image array BUT it also rescales your image since it assumes your images only have values from 0-255 which is usually not the case for tifs. So change the line image_numpy = (np.transpose(image_numpy, (1, 2, 0))+1) / 2.0 * 255.0 to image_numpy = (np.transpose(image_numpy, (1, 2, 0))) OR put in your own rescaling/post-processing. Also remove the lines

#if image_numpy.shape[0] == 1:  # grayscale to RGB
#    image_numpy = np.tile(image_numpy, (3, 1, 1))

I did the rescaling afterwards in a separate .py script which does the reverse of the normalization from @liuhh02

In the save_image function change #Original to #Changed

    # Original:
    # image_pil = Image.fromarray(image_numpy)
    # h, w, _ = image_numpy.shape

    # Changed
    image_numpy = np.squeeze(image_numpy) #Remove empty dimension
    image_pil = Image.fromarray(image_numpy, mode='F')
    h, w = image_numpy.shape
    image_path = image_path.replace(".png", ".tif") # Replace .png ending with .tif ending

In aligned_dataset.py your input image is put into a tensor and several transforms are applied to it.
Here you only want to change the way the image is loaded since you dont want it to convert into RGB.

AB = Image.open(AB_path).convert('RGB')
Changed to
AB = Image.open(AB_path)

Lastly in the base_dataset.py you want to change the function get_transform.
You want to remove if grayscale:. The Github page for the transforms.Grayscale is currently down while writing this, but if I remember correctly it tries to scale an [0,255] range image to [0,1] grayscale. And if you have already normalized to [-1,1] this transformation just reduced all your values to 0.

#if grayscale:
#       transform_list.append(transforms.Grayscale(1))

You also want to make changes to if convert:. If your images are already normalized you need to remove the normalization here. OR insert the normalization code from @liuhh02

    if convert:
        transform_list += [transforms.ToTensor()]
        if grayscale:
            transform_list += [transforms.Normalize((0.5,), (0.5,))]
        else:
            transform_list += [transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]

Should be changed to

    if convert:
        transform_list += [transforms.ToTensor()]

I think these were all the changes I made.

Best,
Johan

@JohanRaniseth Amazing work! Thank you for sharing it here :)

Hello @liuhh02, would it be possible to identify in your code what parameters I would need to change should I require a different scale of values other than [-1 1].

Hello @liuhh02, would it be possible to identify in your code what parameters I would need to change should I require a different scale of values other than [-1 1].

Sure! This line of code scales the values to [0, 1]:
scaled = np.array((arr-arr_min) / float(arr_range), dtype='f')

So you may modify
arr_new = -1 + (scaled * 2)
accordingly to scale it to the range you require.

@liuhh02 Nice approach, thank you for your input

Thanks to @liuhh02 and @JohanRaniseth for the work. For anyone visiting in the future: you have to normalize based on the min and max of your whole (training) dataset, not every image individually like in the code provided above (see #950). If you normalize individually, you will lose information and be unable to reverse the process later.

I also had to make another change to the tensor2im function in util.py by changing this line to just return image_numpy.