Practice of Multi-Service Modeling in Meituan Search Sorting

The search on the Meituan homepage is the largest traffic distribution portal for various life services on the Meituan App, providing various services to tens of millions of users every day. Search ordering is a typical multi-service mixed ordering modeling problem, and this kind of multi-service scenario search has many challenges. This article focuses on the modeling and optimization of multi-business sorting in multi-business scenarios of stores and merchants, hoping to be helpful to students engaged in related work.

introduction

Meituan’s mission is to “help everyone eat better and live better”. The businesses carried by the Meituan App include food delivery, in-store dining, grocery shopping, selection, hotels, travel, leisure and entertainment and other life services. The search on the Meituan homepage is the largest traffic distribution portal for various life services on the Meituan App, providing various services to tens of millions of users every day. Meituan search sorting is a typical multi-service hybrid sorting modeling problem. A typical multi-service search scenario is when users search for a location, such as "Wangjing", and the user's needs are not very clear. At this time, the search result page is as follows As shown in Figure 1, the list of businesses below will contain the results of various businesses such as catering, movies, leisure and entertainment, and hotels near Wangjing. This is a multi-business mixed sorting scenario.

However, there are several challenges in the multi-service scenario:

1. Because of the commonalities and characteristics between businesses, how to make the model take into account these two characteristics to achieve better data learning. For example, visiting restaurants and restaurants are very sensitive to distance characteristics, while tourist attractions business is relatively insensitive to distance characteristics.
2. Businesses naturally have high-frequency and low-frequency characteristics (such as food delivery and travel), resulting in an imbalance in the number of multi-business samples in the model's training data.
3. Each business often has its own different main goals. How to meet the goals of different businesses can ultimately improve the user experience of search.

This article shares the optimization work of multi-business sorting modeling in Meituan Search. We mainly focus on the multi-business scenarios of in-store merchants. The follow-up content will be divided into the following four parts: The first part is the hierarchical structure of Meituan search sorting. Brief introduction; the second part will introduce the multi-service integration modeling on the multi-channel integration layer; the third part will introduce the multi-service sequencing modeling of the fine-ranking model; the last part is the summary and outlook. I hope to inspire or help students who are engaged in related work.

Introduction to the sorting process

The flow of the Meituan search system is shown in Figure 2 below. The overall flow is divided into data layer, recall layer, sorting layer and display layer. The sorting layer is divided into the following sub-parts:

1. coarse sorting layer : Use a relatively simple model to preliminarily filter the recall candidate set to achieve trade-off of sorting effect and performance.
2. Multi-channel fusion layer : Use query word features and context scene features to construct a quota model, control the number of different service candidate sets, and achieve accurate understanding of user needs.
3. fine ranking layer : uses a deep learning model with billion-level features to capture various explicit and implicit signals to achieve accurate estimation of Item ranking scores.
4. rearrangement layer : Use small models and various mechanisms to adjust the order of the results after fine-ranking to achieve fine-directed optimization.
5. Heterogeneous sorting layer achieve high multi-service load.

The multi-layer sorting architecture is designed to balance the sorting effect and performance. The multi-service modeling optimization work mentioned later in this article is mainly introduced from the multi-channel fusion layer and the fine scheduling layer.

Multi-service modeling practice

Multi-service quota model (multi-channel integration layer)

With the development of Meituan's business, Meituan Search has access to the business of dining, shopping, hotel, and tourism. For search terms with ambiguous business intentions, such as a user's search for "Wudaokou", it is necessary to comprehensively judge the user's business intentions based on multiple factors such as the user, the query word, and the scene. In order to integrate the recall results of different businesses and refine a candidate set with a suitable proportion for L2, we designed a multi-business quota model to balance the proportion of multi-business recalls. This method of merging multiple recall results based on quotas is very common in search and recommendation scenarios, such as Taobao homepage search and Meituan recommendation.

In order to provide flexible access to multiple recalls and adapt to the development of Meituan's search business, we continue to iterate the search quota model. The following will introduce in detail the iterative process of Meituan’s multi-business quota model. In the subsequent parts of the article, the multi-business quota model (Multi-Business Quota Model) will be referred to as MQM for short.

One-dimensional target multi-service quota

Taking into account that there are multiple recalls of different businesses in the big search results, in order to characterize the user's search query's intention to recall the three-way business, we adopt a multi-target modeling method, and take whether each recall is clicked or placed as the goal. Modeling, the first version of multi-service quota model MQM-V1 was realized. The model outputs the joint probability of clicks and orders for each channel of recall as the final quota distribution. At the feature level, we use Query dimensional features, Context dimensional features, Cross dimensional features, and User dimensional features to characterize the real-time personalized needs of users in different scenarios. The MQM-V1 model structure is shown in Figure 4 below.

After the MQM-V1 version was launched, the overall online click rate was +0.53%, and the purchase rate of each business was basically the same.

Two-dimensional target multi-service quota

With the continuous iteration of Dasou's recall strategy, Dasou not only introduced a recall method split by business, but also introduced heterogeneous recall methods across multiple businesses, such as vector retrieval and geographic proximity retrieval, which led to the continuous increase of Daxou recall strategies. , The multi-service quota model also faces the cold start problem brought about by new recall sources. At the same time, in order to strengthen the personalization of the multi-service quota model, we refer to the method of user behavior sequence modeling in [6]. In summary, the differences between this version of MQM-V2 and MQM-V1 are as follows:

• In terms of modeling goals, the one-dimensional target that is clicked in the recall mode is upgraded to the two-dimensional target of the cross-multiplication business in the recall mode, which makes the granularity of the multi-channel fusion more finer and higher.
• The behavior sequence modeling module introduces Transformer Layer.
• In order to solve the cold start problem of the new recall source access, we have introduced an artificial experience layer, including business prior and historical statistics, and the comprehensive model output determines the quota for each recall.

After the MQM-V2 version was launched, the rate of various business indicators has increased, among which the travel purchase rate +2%, the meal visit purchase rate +0.57%, and the general and hotel visit purchase rate remain the same.

Multi-service ranking model (fine ranking layer)

Upgraded from the Meituan search precision model to the DNN model, until the end of 2019, the Meituan search model structure is the mainstream Embedding&MLP paradigm structure in the industry. During this period, we also tried the model structure proposed by the industry such as PNN[1], DeepFM[2], DCN[3], AutoInt[4], FiBiNet[5], etc.

As the iteration progresses, we find that it is difficult for the optimization of a specific business to play a role in the fine-ranking model. In order to take into account the characteristics of each business and support more effective targeted iterative optimization of each business, it is necessary to explore a model structure to adapt to the United States. Multi-business scenarios such as group search. The following will specifically introduce the development history of the fine-ranking model in multi-business modeling. In the subsequent part of the article, the multi-business fine-ranking model (Multi-Business Network) will be referred to as MBN for short.

Independent subnet split

Considering that hotels and tourism account for relatively small traffic in the Meituan Dasou ranking strategy, and related optimization for small traffic is difficult to reflect in the current unified Embedding&MLP model structure, we tried manual customization as shown in Figure 6. Multi-tower model MBN-V1 structure: The main network reuses the current model structure. For detailed information, please refer to the behavioral sequence modeling part in [6], adding independent hotel and tourism sub-networks; the input of the hotel sub-network includes the unique characteristics of the hotel and the scoring output of the main network, and the input of the tourism sub-network includes the unique characteristics of tourism. , The scoring output of the main network, the last layer of the main network FC, the input of the hotel and the tourism sub-tower is different because the business logic is different, which leads to a large difference in data distribution. This is the result of practice. The final output is a weighted calculation of the three outputs. with.

Regarding the weight part of the weighted summation, we have adopted two ways to set the weight:

• The first method is to use hard segmentation, that is, the weight vector is a one-hot sparse vector: predict the hotel business, only select the output of the hotel sub-network, and the rest can be deduced by analogy.
• The second method is to use soft segmentation and use the output of the multi-service quota model as the weight value.

Online experiments found that the second method is better than the first. We believe that the use of hard segmentation will cause the parameters of the sub-tower branch to be updated only by the data of the corresponding business, and the uneven proportion of the data of each business leads to poor learning. Soft segmentation will achieve a kind of knowledge transfer effect. The final online effect is compared with the unified Embedding& MLP model. The overall tourism has achieved a positive effect: the overall click rate is +0.17%, and the other business visit rate effects are basically the same.

Self-learning of sub-network weights

Based on the first version of the multi-service fine-ranking model, we have achieved preliminary positive results. We continue to add gourmet business sub-towers. At the same time, we consider that MBN-V1 relies on the output of the quota model. This will lead to changes in the quota model that may affect the fine-ranking model. The effect has an impact. In response to these factors, we launched the second version of the multi-service model MBN-V2. The model structure is shown in Figure 7. The differences compared to MBN-V1 are as follows:

• Add an independent sub-network for the gourmet business.
• Decoupling the fine-ranking model and the quota model, and integrating the weight-generating sub-network in the fine-ranking model. The input of the sub-network is mainly the characteristics of some Query dimensions and Context dimensions.

Online experiment effect: Compared with MBN-V1, MBN-V2 has an overall click rate of +0.1%, and the effect of business visit rate is basically the same.

Sub-network feature adaptation

On the basis of the second version of the model, we further added to the comprehensive service sub-tower. With the increase of sub-networks, the input of the sub-networks is currently manually designed. This method requires a lot of time for offline experiments. Considering that the current multi-service sub-tower structure is very similar to the industry's multi-task learning, we also try to introduce the industry's multi-task learning structure; at the same time, we analyze the output of the weight sub-network in MBN-V2 and find that the output weight is different The output of business merchants is similar, so the targeted optimization of the business will not be obvious. Based on the above parts, we iterated the third edition of the multi-service refinement MBN-V3, the structure is shown in Figure 8 below, and the improvements are as follows:

• Supplement to the comprehensive sub-network, the MMoE [7] multi-task learning structure is used to automatically learn the feature representation and output to the upper sub-network, thereby replacing the input of the artificially designed sub-network.
• In addition to the main LambdaLoss calculated by the user's online feedback, the loss function of the fine-ranking model additionally adds the category cross-entropy Loss of the business to achieve the goal of predicting the item score of a business item with the largest weight of the corresponding business sub-tower.

Online experiment effect: Compared with MBN-V2, MBN-V3 has the same overall click-through rate effect, food business visit rate +0.36%, comprehensive business visit rate +1.07%, hotel business visit rate +0.27%, travel business Purchase rate +0.35%.

Multi-service feature expression optimization

Although the MMoE multi-task learning structure has been applied in many scenarios in the industry, and has been effectively verified in our multi-service modeling scenarios, we continue to follow up with the industry's frontiers and combine business scenarios to implement them.

We tried the PLE[8] structure proposed by Tencent, and iterated MBN-V4 for multi-service refinement. PLE can be regarded as an improved version of MMoE. It has its own specific expert layer for each task. There is a shared expert layer between different tasks. Compared with MMoE, it is the weighted sum of all expert outputs. The input of the PLE subtask is The weighted sum of the output of the unique expert and the shared expert of the subtask makes it easier to learn the characteristics of the business; at the same time, based on performance considerations, we selected a single-layer PLE, or CGC structure, as shown in Figure 9 below:

Online experiment effect: Compared with MBN-V3, MBN-V4 has an overall click rate of +0.1%, a gourmet business visit rate of +0.53%, and the fluctuation of the visit rate of other businesses is the same; we visualize the expert weights of MMoE and CGC as shown in the figure below As shown in 10, the analysis found that the expert-level weights of the CGC structure have smaller variance and more stability than the expert weights of the MMoE among multiple samples of the same business, indicating that CGC has more advantages in feature representation than MMoE.

Summary and outlook

Since the end of 2019, in order to solve the actual multi-service recall ordering problem, Meituan Search has conducted a lot of explorations, enriching the support for multi-services at all levels from engineering to algorithm to product form. Among them, the sorting algorithm level is mainly optimized in the multi-path recall fusion layer and the fine ranking layer.

The multi-channel fusion layer mainly completes the screening process of search results from relevance to high-quality results. It needs to solve the problem of fusion and truncation of different recall methods (text recall, recommendation recall, vector recall) and different business recall results, which directly determines that users can browse The result candidate set. The most important issue is to determine the strength of user demand for each business and the recall quality of each business, and to determine the appropriate fine-ranking admission criteria for each business result and recall result.

The multi-service quota model integrates user real-time demand, Query historical statistics, search context information, and the quality of each recall source, and gives the proportion of each recall and each business that should be refined. This model ensures the diversity and quality of the results of the refined candidate set in different scenarios, realizes the less intrusive access of new services and new recall methods, and reduces the cost of service and recall access. At the same time, it also provides a priori weight for merging the results of each service sub-network for the network structure of the fine-ranking layer sub-service.

The fine ranking layer further performs refined sorting modeling and scoring on the multi-service search results on the basis of the multi-channel fusion layer. The needs of users are as diverse as Meituan’s business. In order to fully model the needs in various scenarios, the fine-ranking multi-service ordering model has carried out multiple operations from the underlying data (rich features of sub-services), model structure, and integration of business goals. Round iterations. Among them, the model structure and the corresponding target integration directly model various large and small businesses, scenarios and corresponding business goals in fragments, which effectively alleviates the problem of small business and small scenes being overwhelmed by large business samples in unified modeling. At the same time, the model supports the rapid iteration of new and old businesses, and each business can easily and independently iterate features, model structure and corresponding goals.

The above optimization covers all online traffic, and the search user experience and the value of each business have been significantly improved, but there is still a lot of work to continue to optimize.

• business unique features use : currently we use to add business unique features to some businesses, and other businesses give default values for these missing unique features, but this will bring a lot of redundant calculations, regardless of whether this part is from There is room for optimization in terms of effect and performance.
• sample imbalanced learning : The amount of data for different businesses is very different in Meituan search. How to make the model better learn the distribution of small businesses, we are exploring methods such as transfer learning and Meta-Learning.
• Multi-objective optimization : Meituan Search must take into account the user's search experience and serve the strategic goals of Meituan's various businesses, so the main optimization indicators of each business may not be the same. Multi-objective optimization is also a direction of continuous exploration.

The work described in this article focuses on the search and ranking of Meituan’s multi-business merchants. At the same time, with the development of commodity businesses such as optimization, grocery shopping, group good goods, flash sales, etc., we are also carrying out a mixed arrangement of commodity-type multi-business and merchant products. Structure multi-business mixed arrangement work.

Reference

• [1] Product-based neural networks for user response prediction
• [2] DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• [3] Deep & Cross Network for Ad Click Predictions
• [4] AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• [5] FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
• [6] Transformer's Practice in
• [7] Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
• [8] Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations

About the Author

Peihao, Xiao Yao, Xiaojiang, Jiaqi, Chen Sheng, Yunsen, Yongchao, Liqian, etc., all come from the search and NLP department of the Meituan platform.

Read more technical articles from the

the front | algorithm | backend | data | security | operation and maintenance | iOS | Android | test

| in the menu bar dialog box of the official account, and you can view the collection of technical articles from the Meituan technical team over the years.

| This article is produced by the Meituan technical team, and the copyright belongs to Meituan. Welcome to reprint or use the content of this article for non-commercial purposes such as sharing and communication, please indicate "the content is reproduced from the Meituan technical team". This article may not be reproduced or used commercially without permission. For any commercial activity, please send an email to [e-mail] to apply for authorization.