---
tags: OpenCV
---

# OpenCV with Python -- 4

[Back to book mode](https://hackmd.io/@Justin123/opencv)

## Introduction
So far, we have mentioned lots of image preprocessing technique. We're going to talk one more approach for elimanating noise and move to the application aspect.

## Morphology
Morphology is an image processing theroy based on **Lattice theory** and **topology**. Here, we're not going to talk about what is morphology. In fact, we're focus only on **dilation** and **erosion**.

**Dilation** help us to make picture brightness and make the features more bigger then before.

**Erosion** can make the feature smaller then before.

Often, we will come across lots of noise or some less important feature. We can first apply **erosion to erase noise or rescale the size** and convert the **wanted features back to the original one by dilation**. Let's see how dilation and erosion do:

* **Dilation**:
The value of the output pixel is the ==maximum== value of all pixels in the neighborhood.
* **Erosion**:
The value of the output pixel is the ==minimum== value of all pixels in the neighborhood.

Notice that before doing the dilation or erosion, we should ensure that the image is in **binary format**, that is, 0 and 1.

* **Function in OpenCV**

import cv2
import numpy as np

img = cv2.imread("resources/shapes.jpg")

# Change the image to gray scale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Blur the image
img_blur = cv2.GaussianBlur(img_gray, (7, 7), 0)

# Convert to 0, 1 (Here just for demostration)
img_thresh = cv2.threshold(img_blur, 200, 1, cv2.THRESH_BINARY_INV)[1].astype(np.float)

# Define the kernel (The size of the neighboorhood)
kernel = np.ones((69, 69), np.uint8)

# First erase the noise
'''
kernel -> size for doing the erosion
iteratoin -> times to do the erosion
'''
img_eroded = cv2.erode(img_thresh, kernel, iterations=1)

# Back to the original feature
'''
kernel -> size for doing the dilation
iteratoin -> times to do the dilation
'''
img_dilation = cv2.dilate(img_eroded, kernel, iterations=1)

# Show the above image
result = np.hstack([img_thresh, img_eroded, img_dilation])

cv2.imshow("Result", result)

cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.waitKey(1)


We removed hexagon but also affect other shapes in the image.

## Contour / shape detection

### What is contours?
Contours is simply a **curve joining all of the continuous points** along the specific boundary. Let's say when we see a circle, how do we recognize it as a circle not the rectangle? That's a stupid question, but we should first know **HOW** humans do, then we can apply it to the computer. Below image show how we do.
<center><img src=https://i.imgur.com/ky5zhsF.jpg width=450 /></center>

### Flow chart of finding contours
Due to the complexity of the function in this session, I will first go through the function going to use and demonstrate all of it later. Here is the flow chart to do the contour detection.
<center><img src=https://i.imgur.com/WEiit8g.jpg width=600 /></center>

### Step by step explanation

The complete code will be showed after this session.

1. **Read Image**

    img = cv2.imread("resources/shapes.jpg")

2. **Blur the image**
Remove noise.

    # Change the image to gray scale and blur the image
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

3. **Convert to binary format**
Using **Canny** or **Threshold**.

    # Edge detection
    img_canny = cv2.Canny(img_blur, 50, 50)

So far, we have finish the preprocessing. We combine them togetherand start to implement a function called `get_countours`. (Function starts at ==line 78==, we will walk throught it step by step).

'''1. Read image'''
img = cv2.imread("resources/shapes.jpg")

'''2. Blur image'''
# Change the image to gray scale and blur the image
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

'''3. Convert to binary image'''
# Edge detection
img_canny = cv2.Canny(img_blur, 50, 50)

# Copy the original image
img_contour = img.copy()

'''Other steps'''
# Get contours
get_contours(img_canny)

4. **Find contours**
* `cv2.findContours(img, mode, method, ...)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#findcontours)

* img: Image to find contour.
* mode: Contour retrieval mode. See the detail in documentation.
* Method: Contour approximation method. See the detail in documentation.

    def getContours(img):
        '''
        4. Find contours
        In this session, we ignore the use of hierarchy
        '''
        contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL,
                                                cv2.CHAIN_APPROX_NONE)
        
        '''
        Print the contours to see the result by yourself.
        You should notice contours is a list with np.array.
        Each elements represent one contours you find. 
        Each contours include all the points on it.
        '''
        print(contours)
        print(contours.shape)

> If mode is cv2.CHAIN_APPROX_SIMPLE, it will only return the points. Otherwise, cv2.CHAIN_APPROX_NONE will return the line. You can try it by yourself!

5. **Drop the noise**
Here, we use the size of the area to drop the noise. We use ==threshold to drop the noise==. </br>
* `cv2.contourArea(contour)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#contourarea)

* contour: Contour get from cv2.findContours(). In fact, it could be point vectors.

    # Get the area of the closed contour
    def get_contours(img):
        '''4. Find contours'''
        # Retreive the outer contours
        contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

        for cnt in contours:
            # Get the area of the closed contour
            area = cv2.contourArea(cnt)

            '''
            We can calculate the area for the bounding shapes.
            And drop the smaller area.
            '''
            print(area)

            # Clean noise by setting the threshold of area
            if area < 800:
                continue

6. **Draw the contours**
* `cv2.drawContours(img_output, contour, contourIdx, color, thickness)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#drawcontours)

* img_output: Draw on which image.
* contour: Point vectors (Same as above).
* contourIdx: Index for which contour to draw. If negative, draw all the contours.

    def get_contours(img):
        '''4. Find contours'''
        # Retreive the outer contours
        contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

        for cnt in contours:
            '''5. Drop the noise'''
            # Get the area of the closed contour
            area = cv2.contourArea(cnt)

            print(area)

            # Clean noise by setting the threshold of area
            if area < 800:
                continue

            # Draw the contour
            '''
            img_contour is the copy of origin image in line 74.
            You should add cv2.imshow by yourself to see the result.
            '''
            cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)

6. **Recognize the shapes**
Draw the ==bounding box== and put the text to specify each shapes.
* `cv2.arcLength(contour, closed)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#arclength)

* closed: Flag indicating whether the curve is closed or not.

* `cv2.approxPolyDP(contour, epsilon, closed)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#approxpolydp)
* epsilon: Specifying the approximation accuracy.
* `cv2.boundingRect(points)` [Documentation](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#boundingrect)
* points: Point vectors.
</br>

    def get_contours(img):
        '''4. Find contours'''
        # Retreive the outer contours
        contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

        for cnt in contours:
            '''5. Drop the noise'''
            # Get the area of the closed contour
            area = cv2.contourArea(cnt)

            # Clean noise by setting the threshold of area
            if area < 800:
                continue

            '''6.1 Draw the contours'''
            cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)

            # Connt the peripheral 
            peri = cv2.arcLength(cnt, True)

            '''6.2 Regconize the shape'''
            # Get the vertex of the shape
            approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)

            # Count the number of the sides
            obj_cor = len(approx)

            # Get the bounding box of the shape
            x, y, w, h = cv2.boundingRect(approx)

            '''
            Print out the new info
            '''
            print("The peripheral is", peri)
            print("The vertex are:", approx)
            print("Therefore, there are {} sides.".format(obj_cor))
            print("The boundind box is", (x, y, w, h))
            print("-"*100)

</br>

**Code**

import cv2
import numpy as np


def getContours(img):
    '''4. Find contours'''
    contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL,
                                                cv2.CHAIN_APPROX_NONE)
    
    # Process all the contours
    for cnt in contours:
        '''5. Drop the noise'''
        # Get the area of the closed contour
        area = cv2.contourArea(cnt)
        
        # Clean noise by setting the threshold of area
        if area < 800:
            continue
         
        '''6.1 Draw the contours'''
        cv2.drawContours(img_contour, cnt, -1, (255, 0, 0), 3)
        
        '''6.2 Regconize the shape'''
        # Count the peripheral 
        peri = cv2.arcLength(cnt, True)
        
        # Get the size of the shape
        approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)
        
        # Count the number of the sides
        obj_cor = len(approx)
        
        # Get the bounding box of the shape
        x, y, w, h = cv2.boundingRect(approx)
        
        # Detect the shape
        if obj_cor == 3:
            object_type = "Tri"
        elif obj_cor == 4:
            # Check whether is circle or rectangle
            asp_ratio = w / h
            
            # Error for getting the square
            if (asp_ratio > 0.95) & (asp_ratio < 1.05):
                object_type = "Square"
            else:
                object_type = "Rectangle"
        elif obj_cor == 5:
            object_type = "Pentagon"
        elif obj_cor == 6:
            object_type = "Hexagon"
        else:
            object_type = "Circle"
        
        # Drow the bounding box and put the shape text on it
        cv2.rectangle(img_contour, (x, y), (x + w, y + h), (0, 0, 255), 3)
        cv2.putText(img_contour, object_type,
                    (x + (w // 2) - 20, y - 15), 
                    cv2.FONT_HERSHEY_COMPLEX, 0.5,
                    (0, 0, 0), 2)


'''1. Read image'''
img = cv2.imread("resources/shapes.jpg")

'''2. Blur image'''
# Change the image to gray scale and blur the image
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 1)

'''3. Convert to binary image'''
# Edge detection
img_canny = cv2.Canny(img_blur, 50, 50)

# Copy the original image
img_contour = img.copy()

'''Other steps'''
# Get contours
get_contours(img_canny)

# Show the image
img_stack = stackImages(0.5, [[img, img_blur], [img_canny, img_contour]])
cv2.imshow("Result", img_stack)

cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.waitKey(1)

<center><img src=https://i.imgur.com/VQgqr71.jpg/></center>

## Summary
Find contours can be one of the powerful tools for us. For example, we can use shape as a sign to tell the robot where to go! Also, the bounding box help us to highlight the object we're interested in. In terms of application, we can use it when we're doing object detection. We'll see more example in next article.

## Reference
1. [Contours: Getting started](https://docs.opencv.org/4.4.0/d4/d73/tutorial_py_contours_begin.html)
2. [LEARN OPENCV in 3 HOURS with Python (2020)](https://youtu.be/WQeoO7MI0Bs)
3. [【影像處理】形態學 Morphology](https://jason-chen-1992.weebly.com/home/-morphology)