Background Removal with OpenCV - Attempt 1 (http://codepasta.com/site/vision/segmentation/)

segmentation1.py

import numpy as np
import cv2

def getSobel (channel):

    sobelx = cv2.Sobel(channel, cv2.CV_16S, 1, 0, borderType=cv2.BORDER_REPLICATE)
    sobely = cv2.Sobel(channel, cv2.CV_16S, 0, 1, borderType=cv2.BORDER_REPLICATE)
    sobel = np.hypot(sobelx, sobely)

    return sobel;

def findSignificantContours (img, sobel_8u):
    image, contours, heirarchy = cv2.findContours(sobel_8u, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

    # Find level 1 contours
    level1 = []
    for i, tupl in enumerate(heirarchy[0]):
        # Each array is in format (Next, Prev, First child, Parent)
        # Filter the ones without parent
        if tupl[3] == -1:
            tupl = np.insert(tupl, 0, [i])
            level1.append(tupl)

    # From among them, find the contours with large surface area.
    significant = []
    tooSmall = sobel_8u.size * 5 / 100 # If contour isn't covering 5% of total area of image then it probably is too small
    for tupl in level1:
        contour = contours[tupl[0]];
        area = cv2.contourArea(contour)
        if area > tooSmall:
            cv2.drawContours(img, [contour], 0, (0,255,0),2, cv2.LINE_AA, maxLevel=1)
            significant.append([contour, area])

    significant.sort(key=lambda x: x[1])
    return [x[0] for x in significant];

def segment (path):
    img = cv2.imread(path)

    blurred = cv2.GaussianBlur(img, (5, 5), 0) # Remove noise

    # Edge operator
    sobel = np.max( np.array([ getSobel(blurred[:,:, 0]), getSobel(blurred[:,:, 1]), getSobel(blurred[:,:, 2]) ]), axis=0 )

    # Noise reduction trick, from http://sourceforge.net/p/octave/image/ci/default/tree/inst/edge.m#l182
    mean = np.mean(sobel)

    # Zero any values less than mean. This reduces a lot of noise.
    sobel[sobel <= mean] = 0;
    sobel[sobel > 255] = 255;

    cv2.imwrite('output/edge.png', sobel);

    sobel_8u = np.asarray(sobel, np.uint8)

    # Find contours
    significant = findSignificantContours(img, sobel_8u)

    # Mask
    mask = sobel.copy()
    mask[mask > 0] = 0
    cv2.fillPoly(mask, significant, 255)
    # Invert mask
    mask = np.logical_not(mask)

    #Finally remove the background
    img[mask] = 0;

    fname = path.split('/')[-1]
    cv2.imwrite('output/' + fname, img);
    print (path)


segment('original-small.jpg')

As it might be interesting for some:
I changed (ceteris paribus) mean = np.mean(sobel) to mean = np.median(sobel) and the results changed from:

to

original:

@atraining This method, overall, works horribly on these image. Because

1. edges of object is touching the edges of the image...so contour detection goes wrong (using cv2.BORDER_CONSTANT with sobel may help a little bit...).
   And 2. Some images aren't "relatively plain background" as I mentioned in the blog. If there are objects in the background, then contours will go wrong again. Maybe random forest edge detection is better than sobel here (never tried it so far - http://docs.opencv.org/3.1.0/d0/da5/tutorial_ximgproc_prediction.html).

@Munawwar have you managed to solve the edges problem? cv2.BORDER_CONSTANT doesn't help.

How would you make the background transparent instead of black?

@iSWORD Nope, haven't solved the edges problem.
@Petah The code is dealing with RGB channels. There is no alpha/transparency channel. And I don't remember if opencv gives that.

Hi Munawwar,

I am working on image cleaning project for my curricular project. I need some help regarding this. Please provide me your email to discuss further.

Thanks
Saurabh

may i ask where to put the smoothing in the guide? http://www.codepasta.com/site/vision/segmentation/

is it inside the findSignificantContours()? please help thanks!