The easiest method for image matching (find small pictures in the big picture): find_template of aircv

In my open source project SimpleRPA , aircv’s find_template method has been used for image matching. The most common scenario is to find a pre-cut small image from the screenshot to determine the next mouse click position:

For example, in the main page of Dingding PC terminal below:

Let's find the location of the corporate logo:

aircv is a small project maintained by NetEase. There are several image processing tool functions in it. Its installation is very simple:

pip install aircv

The most used one is the find_template method. The basic usage is as follows:

import aircv as ac

match_result = ac.find_template(image_origin, image_template, threshold, bgremove)

Several parameters represent

• image_origin : The source image to be searched (the screenshot of the main page of DingTalk in the example above). Note that the width and height of the source image must be greater than or equal to the template image
• image_template : the small image of the template to be found (the logo in the example above)
• threshold : the minimum confidence level, between 0 and 1. Because image matching does not need to be exactly the same for each pixel, it can be fuzzy matching, so the higher the value is set, the closer the area found will be to the template image, but if it is set too high, it may not be found.
• bgremove : Whether to remove the background. If this value is set to True, then the function will use the Canny operator to extract the contour of the image first, and then do the search

The return value match_result is a dict structure:

match_result:{
    'result': (x,y),        #tuple，表示识别结果的中心点
    'rectangle':[            #二位数组，表示识别结果的矩形四个角
        [left, top],
        [left, bottom],
        [right, top],
        [right, bottom]
    ],
    'confidence': percentage   #识别结果的置信度,在0-1之间，越大越精准
}

In this way, we can use the return value to mark the position of the rectangular box on the picture:

import cv2
rect = match_result['rectangle']
cv2.rectangle(img_result, (rect[0][0], rect[0][1]), (rect[3][0], rect[3][1]), (0, 0, 220), 2)

The find_template method only returns the most credible result. If there are multiple template images in the source image, they all want to get it back.
Then you need the find_all_template function, the parameters are exactly the same, but the return value is an array of match_result.

But this library hasn't been maintained for a long time. I submitted a PR to it, and no one paid any attention to it. There are still some pitfalls. I will talk about it later.