Best Practice丨Cloud Development CloudBase Content Audit Ability

This article introduces the usage scenarios of cloud development content review and demonstrates how to use the content review function.

*This article mainly introduces the functional applications on the development side of Tencent Cloud Cloud. Please refer to the user guide for the development side of WeChat Cloud:

https://developers.weixin.qq.com/miniprogram/dev/wxcloud/basis/Consecurity.html

Content review introduction

What is content moderation?

Content review is based on image, text, audio and video detection technology, which can automatically detect vulgar, illegal, pornographic, advertising, graphic violations and other content, and review the content of pictures, text, audio and video uploaded by users to meet upload requirements , To help customers reduce the risk of business violations.

Common content review methods are:

• Image content review

Image content review, using deep neural network models to detect image content, accurately identifying sensitive illegal elements, illegal content, etc. in the image, helping businesses avoid the risk of violations.

• Text content review

Text content review, using artificial intelligence text detection technology to effectively identify violations of laws and regulations, advertisements, abuse, contraband and water-filled text content, and provide customized text-sensitive content review programs.

Introduction to Cloud Development Content Audit

Cloud development content review provides fast content review services for content hosted in cloud development (data stored in cloud databases, files stored on cloud storage). You can quickly set content review rules (review scope, review scenarios, automatic ban processing after violations), asynchronous review after database document changes, and complete review and processing of text content within seconds.

Note: Currently, only documents in cloud databases are supported for content review.

Common scenarios for content review

In the current Internet environment, content review can be said to be ubiquitous. Take a forum website as an example: as small as a user’s user name registration content review, user reviews in the comment area or message area, and as large as a user’s posting Content review of the entire post.

Any place that allows users to freely enter content needs to be reviewed, so as to prevent the forum from being surrounded by a large amount of bad information, causing the waste of website server resources, seriously affecting the image of the website, and what is more, the webmaster may Undertake relevant legal responsibilities.

Use the cloud to develop content review

Prerequisites

There is already an available as-you- environment.

https://cloud.tencent.com/document/product/876/41391

Steps

go environment and enter the 161d5a47017ffd environment> content audit> security configuration page. On the security configuration page, you can see two parts: audit rules and audit details.

Step 1: Create audit rules

Before content review, you need to create review rules, that is, formulate which content to conduct what kind of rules for review.

1. Click create to start creating audit rules.

2. Choose which collection in the cloud database to conduct content review. For example, if you need to review the content of user comments, select the collection that stores user comments, as follows:

Note: The collections listed in the drop-down box are all collections in the database. If there is no collection in the current drop-down box, you need to create a collection in the database on the left navigation bar.

3. After selecting the collection, you need to fill in which field under the collection to review. For example, if you need to review the content of user comments, fill in the comment content field in the comment collection, as follows:

4. Draw up a shielded copy. When there is illegal content, the information used to prompt the user, the default is: the content has been violated and will not be displayed temporarily. as follows:

5. Turn on the automatic blocking switch for illegal content. After opening, if the system detects a content violation, it will automatically block the content and rewrite it with the custom block copy prepared in 4. If it is not turned on, only the content will be reviewed and scored, and the blocking will not be automatically rewritten. It is generally recommended to turn it on, as follows:

6. After opening, you need to select the review scene, that is, what type of illegal content should be reviewed, such as verbal abuse, advertising, illegality, pornography, etc. It is generally recommended to select all, as follows:

Note: The review score after the violation type refers to: when the system reviews the content, the reviewed content will be scored. The higher the score, the higher the degree of violation of the content. When the score is greater than the entered review score, The system will automatically block the content.

The audit score can be understood as the "tolerance" of the current type of violation. The lower the score, the lower the tolerance. For example, there is zero tolerance for illegal information, based on the principle of "I would rather kill a thousand mistakes than miss one", that is You can fill in the score for the illegal type a little lower (minimum 60 points), so that once the system finds that the content is suspected to be illegal information, it will be blocked immediately.

7. After filling in the review scenario, finally click to check the terms.

Click confirm . A new audit rule can be created.

Step 2: Test the audit rules

After the audit rules are created, you can test whether the created audit rules are effective.

1. Click on the left navigation bar database page, enter the above selected set, click add documents , first add a normal content:

As shown in the figure below, the content that does not violate the rules will be inserted into the collection normally.

2. Add the document again and try to deliberately enter some illegal content:

As shown in the figure below, the inserted document field contains illegal content, which has been automatically blocked and rewritten.

The audit rule created by the above description has taken effect.

Step 3: View audit details

1. Click on the left navigation bar content review after page, and then click the top of audit details , to see the contents of the audit and the audit results of the audit platform, as follows:

2. When testing the audit rules, the second deliberate input of the violation content has been successfully monitored by the audit system, and the type of violation and the review score of the content have been accurately identified, as follows:

3. In some specific demand scenarios, if an item is found to be normal content in a specific scenario, but the audit system misjudges as illegal content:

Violation of the article can record rollback .

After the rollback, the processing status of the content in the audit record has also changed to Has been rolled back , and the record can also be displayed normally in the database.

*This article mainly introduces the functional applications on the development side of Tencent Cloud Cloud. For the user guide on the development side of WeChat Cloud, please refer to:

https://developers.weixin.qq.com/miniprogram/dev/wxcloud/basis/Consecurity.html