# Daily Note 02/07/2020
###### tags: `Daily Notes` , `OpenCV`
## Name : Christofel Rio Goenawan
## University : Bandung Institute of Technology (ITB)
## Schedule:
1. Read about explanation of Open CV works and implementation.
2. Create simple Python model using Open CV.
3. Testing and Debugging the code.
4. Analyze the result.
## Outcome :
1. Explain what and works of Open CV.
2. Create simple image classification using Open CV in Python.
3. Analyzing result.
## Further Plan :
- Create more complex code using Open CV in Python.
- Understand work of Android Studio.
- Know how to make simple app using Android Studio.
## Daily Log

### 1. Read about explanation of Open CV works and implementation. <mark>(9.00)</mark>
- Learn code and concept from
https://medium.com/the-andela-way/lets-make-memes-a1d7b6dfea18
- Learn from another example in Kaggle for simple image detection.

### 2. Create simple Python model using Open CV. <mark>(12.00)</mark>
- Modify the code in reference with another function.

### 3. Testing and Debugging the code. <mark>13.00</mark>
- First there are some problem in the code, after checking there are some version problem with the reference code. After resolve this the program works well.

### 4. Analyze the Result. <mark>13.30</mark>
- Testing performance accuracy of 3 different architecture and compared the result.

## Report
### 1. What is Open CV
From reference, Open CV is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. OpenCV makes it easy for businesses to utilize and modify the code.


The advantage of open CV are available free, it is quite fast because written in C/C++ language, Low RAM usage (approx 60–70 mb), and very portable as OpenCV can run on any device that can run C language. The disadvantage is because Open CV has a flann library of its own, its causes conflict issues when you try to use OpenCV library with the PCL library

### 2. Create and Testing Open CV Package using Python
Writer found simple Open CV code that easier to understand and implement in reference [3].

The coding consist of data preprocessing to make data picture become vector hence it can be input to the model. Then the model implemented by this code.

for i,im_file in enumerate(im_list[9:15], start=1):
    df = boxes_df.query('ImageID == @im_file.stem').copy()
    img = cv2.imread(str(im_file)) # read file
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    
    # Add boxes
    h0, w0 = img.shape[:2]
    coords = ['XMin', 'YMin', 'XMax', 'YMax']
    df[coords] = (df[coords].to_numpy() * np.tile([w0, h0], 2)).astype(int)

    for tup in df.itertuples():
        cv2.rectangle(img, (tup.XMin, tup.YMin), (tup.XMax, tup.YMax),
                      color=(0,255,0), thickness=2)
        cv2.putText(img, tup.Label, (tup.XMin+2, tup.YMax-2),
                    fontFace=cv2.FONT_HERSHEY_DUPLEX,
                    fontScale=1, color=(0,255,0), thickness=2)
    
    # Add segmentation masks
    mask_files = [m for m in mask_list if im_file.stem in m.stem]    
    mask_master = np.zeros_like(img)
    np.random.seed(10)
    for m in mask_files:
        mask = cv2.imread(str(m))
        mask = cv2.resize(mask, (w0,h0), interpolation = cv2.INTER_AREA)
        color = np.random.choice([0,255], size=3)
        mask[np.where((mask==[255, 255, 255]).all(axis=2))] = color
        mask_master = cv2.add(mask_master, mask)
    img = cv2.addWeighted(img,1, mask_master,0.5, 0)    
    
    plt.subplot(cols, rows, i)    
    plt.axis('off')
    plt.imshow(img)

plt.show()

In that code first image dataset read by ***imgread()*** function in Open CV package. Than we convert image colour into some optional using ***cvtColor()*** function to convert image to RGB color basis. Than each image in dataset we create rectangle for ***image segmentation*** and using ***rectangle()*** function. Then we regonize each image using ***putText()*** function that using Open CV API to recognize the image, and write the text in the box. Then writer create mask for image segmentation by using interpolation in ***INTER_AREA()*** function.

The second approach is to use image detection first then create box segmentation. One of famous method is to use **Blob** in image recognition. The work of image recognition using blob can be shown in diagram below.


The detailed work can be seen in following link.
https://www.pyimagesearch.com/2017/11/06/deep-learning-opencvs-blobfromimage-works/

First writer create blob array to recognize the image by DNN model using ***dnn.blobFromImage()*** function in Open CV and get each confidence value of blob. The we decide the prediction using treshold 0.2 in this model. The model is implemented using this code.

%%time

from skimage import io

im_url = urls.loc[urls.ImageID==im_list[11].stem, 'OriginalURL'].squeeze()
img = io.imread(im_url)

height,width,channels = img.shape

# Make a blob array and run it through the network
blob = cv2.dnn.blobFromImage(img,0.00392,(416,416),(0,0,0),True,crop=False)
net.setInput(blob)
outs = net.forward(outputlayers)

# Get confidence scores and objects
class_ids=[]
confidences=[]
boxes=[]
for out in outs:
    for detection in out:
        scores = detection[5:]
        class_id = np.argmax(scores)
        confidence = scores[class_id]
        if confidence > 0.2:   # threshold
            print(confidence)
            center_x= int(detection[0]*width)
            center_y= int(detection[1]*height)
            w = int(detection[2]*width)
            h = int(detection[3]*height)
            x=int(center_x - w/2)
            y=int(center_y - h/2)
            boxes.append([x,y,w,h]) #put all rectangle areas
            confidences.append(float(confidence)) #how confidence was that object detected and show that percentage
            class_ids.append(class_id) #name of the object tha was detected
            
# Non-max suppression
indexes = cv2.dnn.NMSBoxes(boxes,confidences,0.4,0.6)
print(indexes, boxes, class_ids)


The example image result can be shown below. <br>

### 3. Analyze the Result
Some image recognition result can be shown below.




It is shown that the model can recognize object in image very well. It also only use very low memory ( about 120 MB RAM in my memory ) because low- throughput of Open CV and using Open CV API to create prediction. The masking also work well to highlight object segment. There is a lot of improvement for this model because writer only use default algorithm and haven't tuning the parameter yet.

## Reference
1. https://medium.com/the-andela-way/lets-make-memes-a1d7b6dfea18
2. https://opencv.org/about/
3. https://www.kaggle.com/jpmiller/open-images-eda