OpenCV with Python – 4

Introduction

So far, we have mentioned lots of image preprocessing technique. We're going to talk one more approach for elimanating noise and move to the application aspect.

Morphology

Morphology is an image processing theroy based on Lattice theory and topology. Here, we're not going to talk about what is morphology. In fact, we're focus only on dilation and erosion.

Dilation help us to make picture brightness and make the features more bigger then before.

Erosion can make the feature smaller then before.

Often, we will come across lots of noise or some less important feature. We can first apply erosion to erase noise or rescale the size and convert the wanted features back to the original one by dilation. Let's see how dilation and erosion do:

Dilation:
The value of the output pixel is the maximum value of all pixels in the neighborhood.
Erosion:
The value of the output pixel is the minimum value of all pixels in the neighborhood.

Notice that before doing the dilation or erosion, we should ensure that the image is in binary format, that is, 0 and 1.

Function in OpenCV







































import cv2
import numpy as np

img = cv2.imread("resources/shapes.jpg")

# Change the image to gray scale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Blur the image
img_blur = cv2.GaussianBlur(img_gray, (7, 7), 0)

# Convert to 0, 1 (Here just for demostration)
img_thresh = cv2.threshold(img_blur, 200, 1, cv2.THRESH_BINARY_INV)[1].astype(np.float)

# Define the kernel (The size of the neighboorhood)
kernel = np.ones((69, 69), np.uint8)

# First erase the noise
'''
kernel -> size for doing the erosion
iteratoin -> times to do the erosion
'''
img_eroded = cv2.erode(img_thresh, kernel, iterations=1)

# Back to the original feature
'''
kernel -> size for doing the dilation
iteratoin -> times to do the dilation
'''
img_dilation = cv2.dilate(img_eroded, kernel, iterations=1)

# Show the above image
result = np.hstack([img_thresh, img_eroded, img_dilation])

cv2.imshow("Result", result)

cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.waitKey(1)

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

We removed hexagon but also affect other shapes in the image.

Contour / shape detection

What is contours?

Contours is simply a curve joining all of the continuous points along the specific boundary. Let's say when we see a circle, how do we recognize it as a circle not the rectangle? That's a stupid question, but we should first know HOW humans do, then we can apply it to the computer. Below image show how we do.

Flow chart of finding contours

Due to the complexity of the function in this session, I will first go through the function going to use and demonstrate all of it later. Here is the flow chart to do the contour detection.

Step by step explanation

The complete code will be showed after this session.

Read Image


img = cv2.imread("resources/shapes.jpg")

Blur the image
Remove noise.



# Change the image to gray scale and blur the image
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

Convert to binary format
Using Canny or Threshold.


# Edge detection
img_canny = cv2.Canny(img_blur, 50, 50)

So far, we have finish the preprocessing. We combine them togetherand start to implement a function called get_countours. (Function starts at line 78, we will walk throught it step by step).


















'''1. Read image'''
img = cv2.imread("resources/shapes.jpg")

'''2. Blur image'''
# Change the image to gray scale and blur the image
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

'''3. Convert to binary image'''
# Edge detection
img_canny = cv2.Canny(img_blur, 50, 50)

# Copy the original image
img_contour = img.copy()

'''Other steps'''
# Get contours
get_contours(img_canny)

Find contours

cv2.findContours(img, mode, method, ...) Documentation
- img: Image to find contour.
- mode: Contour retrieval mode. See the detail in documentation.
- Method: Contour approximation method. See the detail in documentation.
















def getContours(img):
    '''
    4. Find contours
    In this session, we ignore the use of hierarchy
    '''
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL,
                                            cv2.CHAIN_APPROX_NONE)
    
    '''
    Print the contours to see the result by yourself.
    You should notice contours is a list with np.array.
    Each elements represent one contours you find. 
    Each contours include all the points on it.
    '''
    print(contours)
    print(contours.shape)

If mode is cv2.CHAIN_APPROX_SIMPLE, it will only return the points. Otherwise, cv2.CHAIN_APPROX_NONE will return the line. You can try it by yourself!

Drop the noise
Here, we use the size of the area to drop the noise. We use threshold to drop the noise.

cv2.contourArea(contour) Documentation
- contour: Contour get from cv2.findContours(). In fact, it could be point vectors.



















# Get the area of the closed contour
def get_contours(img):
    '''4. Find contours'''
    # Retreive the outer contours
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

    for cnt in contours:
        # Get the area of the closed contour
        area = cv2.contourArea(cnt)

        '''
        We can calculate the area for the bounding shapes.
        And drop the smaller area.
        '''
        print(area)

        # Clean noise by setting the threshold of area
        if area < 800:
            continue

Draw the contours

cv2.drawContours(img_output, contour, contourIdx, color, thickness) Documentation
- img_output: Draw on which image.
- contour: Point vectors (Same as above).
- contourIdx: Index for which contour to draw. If negative, draw all the contours.






















def get_contours(img):
    '''4. Find contours'''
    # Retreive the outer contours
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

    for cnt in contours:
        '''5. Drop the noise'''
        # Get the area of the closed contour
        area = cv2.contourArea(cnt)

        print(area)

        # Clean noise by setting the threshold of area
        if area < 800:
            continue

        # Draw the contour
        '''
        img_contour is the copy of origin image in line 74.
        You should add cv2.imshow by yourself to see the result.
        '''
        cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)

Recognize the shapes
Draw the bounding box and put the text to specify each shapes.

cv2.arcLength(contour, closed) Documentation
- closed: Flag indicating whether the curve is closed or not.
cv2.approxPolyDP(contour, epsilon, closed) Documentation
- epsilon: Specifying the approximation accuracy.
cv2.boundingRect(points) Documentation
- points: Point vectors.






































def get_contours(img):
    '''4. Find contours'''
    # Retreive the outer contours
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

    for cnt in contours:
        '''5. Drop the noise'''
        # Get the area of the closed contour
        area = cv2.contourArea(cnt)

        # Clean noise by setting the threshold of area
        if area < 800:
            continue

        '''6.1 Draw the contours'''
        cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)

        # Connt the peripheral 
        peri = cv2.arcLength(cnt, True)

        '''6.2 Regconize the shape'''
        # Get the vertex of the shape
        approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)

        # Count the number of the sides
        obj_cor = len(approx)

        # Get the bounding box of the shape
        x, y, w, h = cv2.boundingRect(approx)

        '''
        Print out the new info
        '''
        print("The peripheral is", peri)
        print("The vertex are:", approx)
        print("Therefore, there are {} sides.".format(obj_cor))
        print("The boundind box is", (x, y, w, h))
        print("-"*100)

Code
























































































import cv2
import numpy as np


def getContours(img):
    '''4. Find contours'''
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL,
                                                cv2.CHAIN_APPROX_NONE)
    
    # Process all the contours
    for cnt in contours:
        '''5. Drop the noise'''
        # Get the area of the closed contour
        area = cv2.contourArea(cnt)
        
        # Clean noise by setting the threshold of area
        if area < 800:
            continue
         
        '''6.1 Draw the contours'''
        cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)
        
        '''6.2 Regconize the shape'''
        # Count the peripheral 
        peri = cv2.arcLength(cnt, True)
        
        # Get the size of the shape
        approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)
        
        # Count the number of the sides
        obj_cor = len(approx)
        
        # Get the bounding box of the shape
        x, y, w, h = cv2.boundingRect(approx)
        
        # Detect the shape
        if obj_cor == 3:
            object_type = "Tri"
        elif obj_cor == 4:
            # Check whether is circle or rectangle
            asp_ratio = w / h
            
            # Error for getting the square
            if (asp_ratio > 0.95) & (asp_ratio < 1.05):
                object_type = "Square"
            else:
                object_type = "Rectangle"
        elif obj_cor == 5:
            object_type = "Pentagon"
        elif obj_cor == 6:
            object_type = "Hexagon"
        else:
            object_type = "Circle"
        
        # Drow the bounding box and put the shape text on it
        cv2.rectangle(img_contour, (x, y), (x + w, y + h), (0, 0, 255), 3)
        cv2.putText(img_contour, object_type,
                    (x + (w // 2) - 20, y - 15), 
                    cv2.FONT_HERSHEY_COMPLEX, 0.5,
                    (0, 0, 0), 2)


'''1. Read image'''
img = cv2.imread("resources/shapes.jpg")

'''2. Blur image'''
# Change the image to gray scale and blur the image
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

'''3. Convert to binary image'''
# Edge detection
img_canny = cv2.Canny(img_blur, 50, 50)

# Copy the original image
img_contour = img.copy()

'''Other steps'''
# Get contours
get_contours(img_canny)

# Show the image
img_stack = stackImages(0.5, [[img, img_blur], [img_canny, img_contour]])
cv2.imshow("Result", img_stack)

cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.waitKey(1)

Summary

Find contours can be one of the powerful tools for us. For example, we can use shape as a sign to tell the robot where to go! Also, the bounding box help us to highlight the object we're interested in. In terms of application, we can use it when we're doing object detection. We'll see more example in next article.

OpenCV with Python – 4

Introduction

Morphology

Contour / shape detection

What is contours?

Flow chart of finding contours

Step by step explanation

Summary

Reference

Read more

Graph -- Basic concept

Graph -- Shortest path algorithm

Graph -- Spanning Tree

OpenCV series