Children’s Day, see me using ModelArts to make 8090 dream of childhood

[Recommendation in this issue] Children's Day for 8-year-olds is a bit hard-core. Let's meet these little programmers and see how they can code a unique childhood.

Abstract: If there is another Children's Day...

This article is shared from the HUAWEI CLOUD community " Childhood Dreams and Enjoy Food"--ModelArts + Wechaty Recognize Childhood Food ", the original author: HU Qi, an EI expert at HUAWEI CLOUD

Hello, everyone, I’m Hu Qi, the Copy Siege Lion. Last night I finally ran through ModelArts + Wechaty, which has been holding back for a long time, to realize an interesting food image recognition application. Although calling AI-related services in conjunction with Wechaty has been implemented by thousands of people many years ago, as a novice, I can use the one-stop AI development platform ModelArts to create data sets, train models and deploy them, and finally develop through the best WeChat The library Wechaty implements a chat assistant that calls AI services, which is still very interesting. Of course, I can't wait to share my practice with you, and I hope to get the guidance of you guys!

Data set production

We started from the AI development process. When I determined that my goal was to combine ModelArts + Wechaty to realize childhood food recognition, basically my selection of technology was also determined. In view of my poor academic skills, I can get started with zero foundation. One-stop AI development platform ModelArts and four lines of code can realize the Wechaty development library Wechaty for chatbots, which became my only choice. At the stage of data preparation, I was fortunate to encounter the code contributed by the predecessors, so I can quickly get the pictures I want from a certain picture. For specific operations, please check my historical article " How to quickly get the data set ". I used the same method to get pineapple beer, white rabbit toffee, crispy noodles, fruit tan peel, hamburger gummies, monkey king dan, peanut nougat, Jianlibao, bonbon chocolate, roasted sweet potatoes, spicy bars, old popsicles, three colors Ice cream, Sachima, popping candy, Wahaha AD calcium, Wangwang crushed ice, Wangzi milk, watermelon bubble gum, senbei, fish skin peanut, corn jelly, Zhenzhi stick, Mimi shrimp cracker, 24 categories, from 7200 The pictures were sorted one by one, and finally 3543 pictures were labeled, which realized a data set that seemed to be the case for the time "160bf0e71dc258 "Children's Day Special"-8090's Childhood Food ".

However, from the current label distribution of the data, it is unreasonable. The number of labels that are labeled the most is as many as 268, while the number of labels that are the least labeled is only 23. The difference between the two is an order of magnitude, so the trained model is large. The probability will go wrong.

In fact, there are certain skills when performing data annotation on ModelArts. For example, I have obtained a large number of pictures locally and uploaded them to the OBS bucket through the command line tool of OBS. Since this annotation is for image classification, I can quickly filter out the current images through the OBS directory where the pictures are stored during annotation. Sorting labels and marking. At present, the ModelArts data set can display up to 60 pictures on a single page during processing, which also provides strong support for me to select pictures carefully. Secondly, when selecting a picture, you can proceed from the last page. Each time you delete a picture, the source file is also removed from the OBS, and the removal is carried forward until the first page is processed. Finally, you can select all the current pages for labeling. It took less than 2 hours for the entire data set from data acquisition to data processing to data set release. I personally feel that I am very friendly to the novice Xiaobai!

Training and evaluation model

If you are interested in the "Children's Day Special"-8090's Childhood Foods data set, you can download it with one click from the AI Gallery, which can be downloaded to OBS or directly to the ModelArts data set.

Based on the data set, in addition to developing algorithms and training models on the ModelArts platform, you can also use AI Gallery's subscription algorithms or directly use ModelArts' automatic learning to train models. For easier operation, this big lion directly uses simple and rude automatic learning to realize it. I mainly want to see what kind of results the current data set can train (PS: In fact, it is mainly free, "Please put free in Comment area"), are you looking forward to it?

After creating the automatic learning task and running for 9 minutes, I got a model. From the training results, it is still very ideal, but whether a mule is a horse depends on the final deployment effect.

The table is the reference value corresponding to each label. The F1 value is the weighted and harmonic average of the accuracy and recall of the model. It is used to evaluate the quality of the model. When the F1 value is high, it means that the model has a better effect; the accuracy refers to It is the ratio of samples correctly predicted by the model in all samples predicted by the model to be a certain category, reflecting the model’s ability to distinguish negative samples; recall rate refers to all samples marked by users as a certain category, the model correctly predicted as The sample ratio of this classification reflects the model's ability to recognize positive samples.

Deployment model

Currently ModelArts supports a free deployment service instance, because my previous practice has already occupied the quota, so when I deploy the model in automatic learning again, it will prompt the error in the following figure, so I can only go to "Deployment online-- Online Service" manually create a service instance.

When deploying the model, we can add some personalized settings or other functions according to our own needs, such as data collection, screening of difficult cases, and so on.


After seeing the successful deployment of the service, I can't wait to debug it first, and I just found a picture of "jumping candy" and let the AI recognize it. Oh, it's not bad, it feels better than I thought. The development part of ModelArts came to an end, and then we proceeded with the joint development with Wechaty in accordance with the debugging guide of the online service, "Talk is cheap. Show me the code."

At this time, the information we have obtained: the online address of the AI service and the input parameter images in the form of file.

Wechaty development

Let's take a look at our final rendering effect, that is, like a WeChat robot sending a specified keyword and then sending a picture, the WeChat robot will call ModelArts' online service to recognize the picture and return the result to WeChat. Here we need to use Wechaty, a powerful WeChat development library. For the details of Wechaty, you can visit Wechaty's official website- https://wechaty.js.org/. From the perspective of the domain name, this development library should be related to JavaScript. After reading the documentation, I learned that novices only need four lines of code to use Wechaty:

npm install qrcode-terminal --save
npm install wechaty 
npm install wechaty-puppet-wechat --save // 这个依赖是关键
export WECHATY_PUPPET=wechaty-puppet-wechat // 这里也是关键，需要配置你使用的puppet

We can create a new folder, execute npm init to initialize a project, then execute the above code, then create a new index.js, write:

const { Wechaty } = require('wechaty');
const name = 'wechat-puppet-wechat';
let bot = '';
bot = new Wechaty({
    name, // generate xxxx.memory-card.json and save login data for the next login
});

//  二维码生成
function onScan(qrcode, status) {
    require('qrcode-terminal').generate(qrcode); // 在console端显示二维码
    const qrcodeImageUrl = [
        'https://wechaty.js.org/qrcode/',
        encodeURIComponent(qrcode),
    ].join('');
    console.log(qrcodeImageUrl);
}

// 登录
async function onLogin(user) {
    console.log(`贴心小助理${user}登录了`);
    //   if (config.AUTOREPLY) {
    //     console.log(`已开启机器人自动聊天模式`);
    //   }
    // 登陆后创建定时任务
    // await initDay();
}

//登出
function onLogout(user) {
    console.log(`小助手${user} 已经登出`);
}

bot.on('scan', onScan);
bot.on('login', onLogin);
bot.on('logout', onLogout);
bot
    .start()
    .then(() => console.log('开始登陆微信'))
    .catch((e) => console.error(e));

Basically completed the development of the Wechaty part, execute node index.js to display a QR code on the console, which is similar to the Wechat we log in to the desktop. Then we mainly need to solve the problem:

1. Listen to WeChat messages - you can use bot.on('message') to achieve;
2. Docking with ModelArts - can use token authentication to access;
3. File transfer - data can be converted through form-data.
   The basic code is implemented as follows:

// 获取Token
async function getToken() {
    let token = ''
    const data = {
        "auth": {
            "identity": {
                "methods": [
                    "password"
                ],
                "password": {
                    "user": {
                        "domain": {
                            "name": config.IAMDomain
                        },
                        "name": config.IAMUser,
                        "password": config.IAMPassword
                    }
                }
            },
            "scope": {
                "project": {
                    "name": config.IAMProject
                }
            }
        }
    }
    await axios.post(config.TokenURL, data).then(res => {
        token = res.headers['x-subject-token']
    }).catch(err => {
        console.log(err)
        token = ''
    });

    return token
}

// 识别美食
async function sendImage(fileName) {
    let resp = {}
    await getToken().then(async res => {
        // console.log(res)
        const form = new FormData();
        form.append('images', fs.createReadStream(fileName));
        console.log(form.getHeaders())
        await axios.post(config.URL, form, { headers: Object.assign(form.getHeaders(), { "X-Auth-Token": res }) }).then(res => {
            console.log(res.data)
            resp = res.data
        }).catch(err => {
            console.log(err)
            if (err.response && err.response.data) {
                resp = err.response.data
            }
        })
    }).catch(err => {
        console.log(err)
        resp = err.data
    })
    return resp
}

For the complete code, please refer to: https://github.com/hu-qi/modelarts-wechaty

Reminder: Please evaluate the related risks of using Wechaty.

1. Using this method to log in to WeChat may be controlled by WeChat's official risk, and those who are serious can be blocked;
2. Not all accounts can use this scheme for login/operation. If your account cannot be logged in through the interface provided by Wechaty, it cannot be used.

For more AI related data, algorithms, models and other AI assets, please click "Learn More" , AI Gallery is waiting for you!

Click to follow and learn about Huawei Cloud's fresh technology for the first time~