Three steps to implement license plate recognition in OpenHarmony

Introduction This license plate recognition project is based on the open source project EasyPR (Easy to do Plate Recognition). EasyPR is an open source Chinese license plate recognition system developed based on the OpenCV open source library.
This project uses Runhe HiSpark Taurus AI Camera (Hi3516DV300) camera development board kit (hereinafter referred to as Hi3516) to take pictures of license plates and display the recognition results. The system used is OpenAtom OpenHarmony (referred to as "OpenHarmony") 3.1 Release small system.
First, aim the camera in Hi3516 at the license plate, and its distance is about 60cm~70cm as shown below:

After running the program, press 1 to take a photo, and press 2 to output the recognition result as shown below:

Development Process

This license plate recognition project is implemented using the media subsystem in OpenHarmony. The code is based on local license plate recognition in parking scenarios. To explain, the code structure is as follows:
Three-party library transplantation
The EasyPR implementation is based on the OpenCV implementation, so the implementation of EasyPR must first port OpenCV. The way of transplantation is to use Gn to call the Shell script, and the Shell script to call the Makefile for implementation.

 ├── BUILD.gn
├── include 
│   ├── camera.h                                       // 摄像头定义
│   ├── local_net_communication.h          // 设备协同主要功能定义
│   ├── local_net_def.h                             // 设备协同打印日志
│   ├── local_net_dlist.h                            // 设备协同设备列表定义
│   ├── local_net_message.h                    // 设备协同传输消息定义
│   ├── local_net_udp.h                             // 设备协同udp协议定义
│   ├── local_net_utils.h                            // 设备协同通用工具定义
│   ├── log.h                                              // 打印日志定义
│   └── wpa_work.h                                   // wifi设置定义
└── src
    ├── base64.cpp                                     // 图片转base64格式功能代码 
    ├── camera.cpp                                     // 摄像头实现
    ├── local_net_communication.c            // 设备协同主要功能实现
    ├── local_net_dlist.c                              // 设备协同设备列表实现
    ├── local_net_message.c                      // 设备协同传输消息实现
    ├── local_net_udp.c                              // 设备协同udp协议实现
    ├── local_net_utils.c                              // 设备协同通用工具实现
    ├── main.cpp                                         // 主程序
    └── wpa_work.c                                     // wifi设置实现

The general process of porting is described below. For details, please refer to Running the Open Source Project License Plate Recognition on a Small System and Porting the OpenCV Library.
Porting OpenCV
Download the source code Get the source code Put the OpenCV library source code under third_party in the OpenHarmony root directory:

Generate Makefile
Create a new build directory in the root directory of the OpenCV source code to generate the Makefile file:
Use cmake-gui to configure the build environment:

 cd build
make-gui ..

The displayed UI interface is as follows:

Click Configure to configure, select the fourth option to configure, as shown below:
Configure the toolchain:
Click Generate to generate the Makefile.

Create a shell script
Add build_opencv.sh to the root directory of the OpenCV source code:

 touch build_opencv.sh
chmod 777 build_opencv.sh
vim build_opencv.sh
##添加如下内容
#!/bin/sh
processor=`cat /proc/cpuinfo|grep processor | sort -u | wc -l`
cd build
make -j$processor
cp lib/* $1/libs/

Create a Gn file Add BUILD.gn in the root directory of the OpenCV source code Add the OpenCV library to the build:

Porting EasyPR
Download the source code Get the source code The EasyPR library source code is placed under third_party in the source code root directory:

Generate Makefile
Create a new build directory in the EasyPr source code root directory:

 mkdir build
cd build
cmake-gui ..

The displayed UI interface is as follows:

Click Configure to configure, select the fourth option to configure, as shown below:
Configure the toolchain:
Click Generate to generate the Makefile.

Create Shell Script Add build_easypr.sh in the root directory of EasyPR source code:

Create Gn file Add BUILD.gn in the root directory of EasyPR source code to compile and build:

 vim BUILD.gn

#BUILD.gn中添加如下内容
import("//build/lite/config/component/lite_component.gni")
import("//build/lite/ndk/ndk.gni")

root_build = rebase_path(root_build_dir)

build_ext_component("easypr_lib") {
    command = "sh build_easypr.sh $root_build"
    exec_path = "$root_build/../../../third_party/EasyPR"
}

lite_component("easypr") {
    deps = [
        "//third_party/opencv:opencv",
        ":easypr_lib"
    ]
    features = []
}

The final OpenCV and EasyPR are in the third_party directory as shown below:

Implementing EasyPR in OpenHarmony needs to be divided into the following three steps:

GN build, add EasyPR to the build build;
Take a picture, call the OpenHarmony camera interface, and take a picture of the license plate;
EasyPR local recognition, call EasyPR to recognize the license plate interface and return the recognition result.

GN build
GN build includes EasyPR header file path, link EasyPR dynamic library, compile and depend on EasyPR. As follows:

The photo function The photo function is developed based on the official document photo development guidance, and its demo sample is in the following directory:

In the parking scene, QR code recognition and license plate recognition share the same photo code. In order to improve the QR code recognition rate, the resolution must be set to 1280*720 when initializing the photo. This change will not affect the license plate recognition. The initial photo code is as follows:
Set the photo save path under the file camera.h:
Because QR code scanning and license plate recognition will call the camera interface in the parking scene, s_runAi is used to distinguish:

 int main(int argc,char **argv)
{
    int ret;
    char licensePlate[32] = {0};
    char input;
    InitCamera();
    PlateInit();
    while(cin >> input) {
        switch (input) {
            case '1':
                RunAICamera();                                                  // 拍照
                break;
            case '2':
                memset(licensePlate, 0, sizeof(licensePlate));
                ret = GetPlateString(IMG_PATH, licensePlate);   // 识别车牌
                SAMPLE_INFO("ret -> %d, licensePlate->%s", ret, licensePlate);
                break;
            case 's':
                PlateDeinit();
                ExitCamera();
                return 0;
            default:
                SAMPLE_ERROR("input Error");
                break;
        }
    }
    return 0;
}

After taking a photo, it will enter the photo data processing. When s_runAi is false, it means that it is QR code recognition. You can directly call the QR code recognition interface. When s_runAi is true, the photo data must be saved as a picture:

Save the photo data as a picture as "/sdcard/CaptureAi.jpg".

EasyPR local recognition Write the main program main.cpp to set the program function to press 1 to take a picture and press 2 to display the result:

Compile and burn

The previous article roughly outlines the migration steps of OpenCV and EasyPR. For more details on the steps of environment construction, burning and project source code construction, please refer to the reference article Local License Plate Recognition.

Summarize

Write the external interface of the license plate recognition library. For the use of related interfaces, please refer to the author's article introduction; the source code of this article refers to the local license plate recognition. The rich and varied OpenHarmony development samples are inseparable from the contributions of partners and developers. If you want to share your own development samples, you are welcome to submit the samples to the OpenHarmony Knowledge System SIG warehouse to build development samples together. Please refer to How to Build a Development Sample.

License plate recognizer (OpenCV version)
https://gitee.com/openharmony-sig/knowledge_demo_temp/tree/master/docs/GreyWolf_ImageRecognition_LocalAI

license plate reader
https://gitee.com/openharmony-sig/knowledge_demo_temp/blob/master/docs/GreyWolf_EasyPR/readme.md

Photo development guide
https://gitee.com/openharmony/docs/blob/master/en-us/device-dev/guide/device-camera-control-demo-photoguide.md

Author Article List
https://gitee.com/link?target=https%3A%2F%2Fwww.cnblogs.com%2Fsubconscious%2Fp%2F3979988.html

Source code reference
https://gitee.com/openharmony-sig/knowledge_demo_temp/tree/master/docs/GreyWolf_ImageRecognition_LocalAI

Build development samples
https://gitee.com/openharmony-sig/knowledge/blob/master/docs/co-construct_demos/README_en.md