Interpretation of e-commerce search capabilities-entity recognition (NER)

About search is the most important function in the business transformation of the e-commerce industry. Do you know the technical implementation principles? Let's learn about the ability of entity recognition in the e-commerce industry based on years of comprehensive data and knowledge base optimization of Taobao, and what optimization effects can be brought in query analysis and category prediction

Introduction to Entity Recognition Function

Entity recognition, the full name of named entity recognition (Named Entity Recognition, referred to as NER), refers to the identification of the brand, category, category modification, model, style, etc. of the semantic entities with specific meaning in the e-commerce query words. Kinds of categories. Query analysis rewrites the query term according to the recognition result and the weight of the entity type, so that the recalled document meets the intent of the query.

<span class = "Lake-fontSize-. 11"> Type </ span>
<span class = "Lake-fontSize-. 11"> common words </ span>	<span class ="lake-fontsize-11">Material</span>	<span class="lake-fontsize-11">Style</span>	<span class="lake-fontsize-11">Style elements< /span>
<span class="lake-fontsize-11">Color</span>	<span class="lake-fontsize-11">Brand</span>	<span class=" lake-fontsize-11">Function and function</span>	<span class="lake-fontsize-11">Size specification</span>
<span class="lake-fontsize-11">Quality Color</span>	<span class="lake-fontsize-11">scene</span>	<span class="lake-fontsize-11" > Population </ span>	<span class = "Lake-fontSize-. 11"> Kit </ span>
<span class = "Lake-fontSize-. 11"> Time Season </ span>	< span class="lake-fontsize-11">Model</span>	<span class="lake-fontsize-11">New product</span>	<span class="lake-fontsize-11"> series </span>
<span class="lake-fontsize-11">Marketing services</span>	<span class="lake-fontsize-11">Location area</	class="lake-fontsize-11">Person’s name</span>	<span class="lake-fontsize-11">Recreation books</span>
<span class="lake-fontsize- 11">Organization entity</span>	<span class="lake-fontsize-11">Film and TV name</span>	<span class="lake-fontsize-11">Game name</span>	<s pan class="lake-fontsize-11">Number</span>
<span class="lake-fontsize-11">unit</span>	<span class="lake-fontsize-11" >Category</span>	<span class="lake-fontsize-11">new words</span>	<span class="lake-fontsize-11">modification</span>
span class="lake-fontsize-11">proper nouns</span>	<span class="lake-fontsize-11">category modifiers</span>	<span class="lake-fontsize- 11">Symbol</span>	<span class="lake-fontsize-11">prefix</span>
<span class="lake-fontsize-	<span class="lake-fontsize-11">Gift</span>	<span class="lake-fontsize-11">Negative</span>	<span class="lake-fontsize-11"> Agent</span>

## Open search entity recognition advantages Based on the full data and knowledge base of Amoy Department for many years, the entity recognition ability of the e-commerce industry is deeply optimized, and the problems of rapid brand update, big ambiguity, category modification relationship, and brand category matching relationship are solved. Solve the difficulties of boundary ambiguity, semantic ambiguity, nested ambiguity, etc., especially in Chinese due to the lack of natural separators. ## The role of entity recognition in query analysis 1. Act on query rewrite: Open search query analysis can rewrite two queries. The first query is more accurate, and the second query reduces the terms involved in the recall. When the number of more accurate recall results is insufficient, the second query is used to expand the recall. Query rewriting is mainly based on the importance of the entity. The entity words with high importance are retained when recalling. The low importance part does not affect the recall, but only affects the algorithm ranking. Implementation : Entity importance is currently divided into three levels: high, medium, and low. Among them, "brand, category" is in the high-end, which is the most important; secondly, "style, style, color, season, crowd, location..." is in the mid-range; finally "size, modifiers, influence service, series, unit..." Low-end, can be discarded without participating in the recall. 2. Use with category prediction Different entities in the query have different effects on the category. Therefore, when the original query does not have the result of category prediction, according to certain rules, after removing the words that are irrelevant to the category intent or having low relevance, the category is performed. Project prediction, this will be of great help to the category prediction of long-tail query. example: " Yang Mi (person's name) Same style (suffix) Spring (time season) Slim (style elements) sorted according to the priority of the word query Spring slim dress Spring dress dress dress The system will query the results of category predictions in the order described above More category prediction functions are introduced in detail : https://help.aliyun.com/document\_detail/69036.html ## E-commerce industry enhanced entity recognition capabilities are upgraded Open Search creates a unique industry intelligent search solution, and launches the e-commerce industry enhanced version , which has been upgraded in entity recognition capabilities to make the entity marking effect better, and can more accurately locate the user’s search intent to meet the product Search requirements. * existing framework knowledge base update Rebuild the full knowledge base based on the old model link, F1 69 -> 74 * labeled training data, using neural network model Labeling 100,000 pieces of data, it takes 4 months, BiLSTM-CRF model, F1 74 -> 78 * neural network model combined with knowledge base Technological innovation GraphNER framework combines supervision model and knowledge base, F1 78 -> 82 ## Entity Recognition Intervention Dictionary In different business scenarios, entity recognition semantics are also different. For the convenience of users, Open Search provides entity recognition intervention dictionaries for users to customize the semantics of words. At present, entity recognition interventions mainly include two: one is the entity recognition result itself Intervention, the second is the intervention of the importance of entity type. When the result of entity recognition is inaccurate, intervention should be carried out through the configuration of the intervention dictionary of entity recognition. By creating an entity recognition intervention dictionary, and configuring the corresponding intervention dictionary for entity recognition in the query analysis, the entity recognition function can be intervened. For example: business special proper nouns that require entity recognition intervention. The user searches for " Bear Baby Nail Care Set 160dec07e4b638". The entity recognition result of "Bear" and "bear" is: ordinary words, but in actual business, "Bear" is a maternal and child brand. Use the intervention dictionary and set it to "brand" to affect the effect of recall ranking. ## Search performance comparison after practice 1. Search "Netherlands Mei Su Jiaer infant milk powder" word segmentation effect • Before: "Netherlands Mei Su Jiaer infant milk powder milk powder for infants and young children" • After: "Netherlands (location area) Mesocare (brand) Infant (crowd) milk powder (category)" 2. Search "Summer New Sandals for Boys" • Before: 3 recall results, • After: 22 recall results **Using entity recognition function: locate search keywords Entity recognition: "Summer" season, "boys" crowd; "new" new products;; "sandals" category;** Query1: "Sandals" "Summer" "Boys"; Query2: "Sandals" * ## Alibaba Cloud search related product recommendations OpenSearch: a one-stop intelligent search business development platform based on a large-scale distributed search engine independently developed by Alibaba. It currently provides search service support for the core businesses of Alibaba Group including Taobao and Tmall. With built-in capabilities such as query semantics understanding of various industries and machine learning sorting algorithms, fully open engine capabilities are provided to help developers quickly build smart search services with higher performance and higher search baseline effects Learn about product details: https://www.aliyun.com/product/opensearch Learn more about e-commerce industry search solutions: https://www.aliyun.com/page-source//data-intelligence/activity/opensearch > Copyright Notice: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.