Cold Start Algorithm Series - A Preliminary Study on Cold Start of Cloud Music Songs

Image credit: https://revolutionmotors.ca/blog/car-batteries-cold-weather/

Author: Origin, Zhengchao

1. Introduction to the cold start problem

1 What is a cold start

In the recommendation system, there are thousands of users and thousands of items. The essential task of the recommendation system is to recommend items of interest to users. In this, users and items are constantly updated. How to recommend items of interest to new users, and how to recommend new items to users who are interested in them, is the cold start problem of the recommendation system.
Therefore, the cold start problem of the recommendation system mainly includes two categories: user cold start and item cold start.

2 The importance of cold start

The flow and uncertainty of users are objective facts, and the listing, update, and removal of items are also objective facts. In today's information overload era, the uncertainty of users is more obvious. How to deal with these uncertainties? users recommend good items, which is one of the main functions of the recommender system. Now that users and items are constantly being created, which is the norm on the Internet, the cold start problem accompanies the entire life cycle of a product.

Every product on the Internet is paying attention to MAU and DAU. In this era when traffic is king, users play a crucial role in whether a product can survive and survive well. Whether new users are satisfied with the product and whether they can be retained is directly related to the user growth and revenue growth of a product. In the field of business, the customer is God. On the Internet, this sentence still applies, and this sentence is vividly expressed.

In addition, whether a product can innovate its items is the key to attracting users. In a sense, the quality of an item directly determines the quality of a product.

Therefore, how to solve the problem of new users and new items, that is, the cold start problem, is very important for the recommendation system.

3 methods of cold start

According to the different characteristics of users and items, different cold start methods will be adopted, which will be explained separately below.

3.1 User cold start

non-personalized recommendations

Popular recommendation is a good method. Although it is not personalized, many people have a herd mentality. According to the 28 principle, recommending popular items to new users can meet the needs of 80% of users. For example, the recommendation of popular movies, popular songs, and popular short videos.

Referral using registration information

Many apps now require users to register before they can be used, so personalized recommendations can be made based on these registration information. For example, marriage and love websites can recommend beautiful women to men and handsome guys to beautiful women.

In addition, portraits can be formed according to the registered age, region, occupation, education, income and other information, and then personalized recommendations can be made based on these portraits.

Recommendations based on points of interest

Now some apps require users to select their own points of interest before using them, so that the recommendation system can make good recommendations. For example, news apps require users to select tags of interest, game apps require users to select game types, music categories, etc. The app asks the user to select a genre of interest, etc.

Recommendations based on a small number of behaviors

Some users are less active and have fewer behaviors, but they can make personalized recommendations based on these rare user behaviors. For example, if a user has watched a short video, they can make recommendations based on the short video they have watched.

Recommendation using heuristics

The detection and utilization method is one of the commonly used methods in recommender systems. First randomly recommend a few items to the user, and then obtain the user's interest based on the user's feedback. This method is mainly suitable for apps that consume less user time, and can quickly locate the interests of users, such as news and short video apps.

Recommendation based on interest transfer strategy

Some companies have relatively mature apps, or there are multiple types of recommendations on an app, and they can use other user interests to make recommendations for migration. For example, some apps recommend music and short videos, which can be recommended according to the user’s interest in music. Related short videos.

3.2 Item cold start

Recommendation based on side information

Items naturally have some attribute information, such as the merchant, classification, price, etc. of the product, and then the language, style, genre, musical instrument, etc. of the music. The recommendation system can recommend it to the corresponding interested users based on these basic information.

Recommendations based on a small number of behaviors

Some items have a small amount of behavior information, and personalized recommendations can be made based on this small amount of behavior information. For example, if a user has played a short video completely, the short video can be recommended to similar users.

Recommendation using heuristics

The method of detection and utilization is also applicable to the cold start of the item. First, a cold start item is randomly distributed to a group of users, and recommended to the corresponding interested users according to the user's feedback.

4 Evaluation indicators of the cold start method

To evaluate the quality of a cold start method, the following three points are mainly considered:

Coverage

The first evaluation indicator that needs to be considered is coverage. The size of coverage directly determines the quality of online effects. If the coverage is too low, no matter how good the online coverage is, the overall effect will be greatly reduced. For the methods described above, both the item cold start method based on side information and the non-personalized user cold start method have high coverage, which can cover almost 100%. On the other hand, the user cold start method based on a small number of behaviors and the user cold start method based on interest migration are relatively strict, and their coverage is not so high.

Accuracy

The second evaluation indicator to be considered is accuracy. For example, the user cold start method based on interest migration has a relatively high recommendation accuracy because there are more user information, while the method based on side information, although the coverage Very high, but its accuracy is not that high.

explainable

The interpretability of recommendations in recommender systems is very important for both users and recommender systems, and many recommender systems are now paying more and more attention to the interpretability of recommendations. Similarly, for the cold start problem, good interpretability is also conducive to improving the accuracy of the recommendation. For example, the user cold start method based on points of interest can explain the recommended items to the user very well.

From the above evaluation indicators, no one method occupies all the advantages, so the recommendation system in practice also coexists with multiple cold-start methods to achieve the effect of complementary advantages of multiple methods.

Above we briefly introduced the definition of the cold start problem, the general methods for solving the two types of cold start problems, and the criteria for evaluating the quality of the cold start method. Next, we will introduce our practical solution to the problem of cold start of songs in the music recommendation system.

2. Cold Start Practice of Cloud Music Songs

1 Business Background

At present, there are more than 400,000 independent musicians in NetEase Cloud Music. Independent musicians release a large number of excellent new works every day. How to quickly and accurately distribute these excellent new works to the target audience's playlist, complete the cold start of the song and then Entering the song growth system is an important problem to be solved by NetEase Cloud Music's recommendation system.

Due to its particularity, it is difficult to directly apply the recommendation model established for non-cold-start songs to cold-start songs, so it is necessary to establish an effective song recommendation model for cold-start songs.

2 Problems faced by song cold start

2.1 The lack of song features

The fundamental problem faced by cold-start songs is the lack of historical user interaction data with songs, resulting in the lack of features and samples.

Missing song stats

Including various behaviors of songs, such as the number of times and conversion rate characteristics of playing, downloading, collecting, sharing and other behaviors, such characteristics are usually an important part of song recall and sorting models, and cold-start songs cannot be used directly because they do not contain these characteristics. existing model.

Missing samples to train the embedding vector for cold-start songs

The song embeddings in the recall and ranking models in the recommender system are usually obtained by end-to-end training, and the cold-start songs do not exist in the vocabulary, and the corresponding song embedding representation cannot be directly obtained.

2.2 Business Explainability

The ultimate goal of the song cold start system is to serve the business. In addition to the goal of successfully distributing cold start songs, it is also hoped that the process of cold start should be as interpretable as possible. An interpretable cold-start system will better help businesses to answer questions such as what songs are more likely to be successful in cold-start, thus providing successful experience for subsequent cold-start songs.

3 Solutions

The core idea of solving the cold start of songs is to increase the available data as much as possible. The most widely used method is the cold start method using side information. This kind of method is usually simple to implement, has low requirements for data characteristics, and has good business interpretability. .

The following will introduce the cold-start scheme based on content tags from the perspectives of cold-start song recall and cold-start song sorting.

3.1 Cold Start Recall

Since the user interaction behavior records of cold-start songs cannot be collected, it is usually impossible to recall cold-start songs using conventional i2i or vectors, but the next best thing is to use the content tags of cold-start songs to recall.

In the first step, the recall part preprocesses and normalizes the content tags corresponding to the cold-start songs, including retaining only the main genre, normalizing minor languages, and unifying album artists and singing artists.

In the second step, the cold-started songs are classified according to their content tags, and each category is given a recall candidate score according to the conversion rate with time decay. The calculation method of the candidate score is as follows:

consists of three parts

Part 1: Time decay factor, days represents the number of cold start days, and T represents the half-life. The overall meaning of this part is that the more recent the new song has, the higher the score.

Part 2: Smooth tag conversion rate, calculated as

like7d and play7d represent the number of song collectors and players on the platform in the last 7 days, respectively, and k is used to adjust the initial conversion rate. This part represents the conversion rate using the label dimension of the cold start song to represent the cold start song conversion rate.

Part 3: The smooth conversion rate of the cold start song itself. The effect achieved by the overall calculation process is: 1) Use the smooth conversion rate as the recall candidate score; 2) If there is no cold start song conversion data at the initial stage of the cold start, the corresponding cold start song is used. 3) With the advancement of the cold start distribution time, the effect of the conversion rate of song content tags decreases, and is gradually replaced by the conversion effect of the actual song.

3.2 Cold Start Sequence

Similar to the recall part, the features of cold-start songs cannot be directly used for sorting, but can be degraded to use the label features of cold-start songs, and borrow the genre, language and artist statistical features and embedding vectors trained by the non-cold-start song recommendation ranking model. to represent the missing features of cold-start songs.

The sorting part of the model structure is as follows

Contains song-side and user-side feature construction, and ranking models for user and cold-start songs.

song side

Export the genre, the embedding vector corresponding to the language and artist, and the statistical features corresponding to the song label in the ranking model trained for non-cold-start songs. Then, according to the genre, language and artist label of the cold start song, the embedding and statistical features are respectively queried.

User side

Calculate the user's preference sequence for the genre, language and artist dimension of non-cold-start songs, query the embedding and do pooling to express the user's label preference as the corresponding preference vector.

Sorting model: In this part, the inner product of the three-dimensional vectors of the user's genre, language, and artist is used to calculate the matching score, and then the statistical features of the song tag dimension and the attribute features of the user dimension are added to the logistic regression model to predict whether the user is a user. The training of the model is carried out on the user's interactive samples of non-cold-start songs, and the weight output of each is obtained after the training is completed.

online prediction

Since the user-side preference vector, user attribute features, and cold-start song-side label conversion rate and song label vector can all be completed offline, online prediction only needs to query the embedding according to the song label, and then match with the user preference label respectively. Multiplying each dimension weight input sigmoid in the logistic regression model can get the user's ranking score for cold start songs. Online prediction requires less computation and faster prediction.

Finally, after obtaining the user's score for the candidate cold-start songs, the top songs are taken and inserted into the distribution flow of the existing song recommendation system to complete the distribution of the cold-start songs.

4 business results

The most direct indicator to measure whether a cold start song is successfully distributed is the exposure coverage rate of the cold start song, that is, how many users who have exposed songs have exposed cold start songs. However, since cold-start songs are not necessarily all high-quality, recommending low-quality songs to users may affect the user's playing experience, so cold-start songs cannot be distributed without restrictions. A good song cold start system should have no or as little impact on the existing song recommendation system as possible, so that more cold start songs can be distributed.

The figure below compares the online experiments, adding the recall and sorting recommended by cold start as the experimental group, and the experimental effect of no cold start recommendation as the benchmark group. During the experiment, without affecting the user experience indicators of the existing song recommendation system, the experimental group that added the song cold start system achieved a relative increase of more than 40% in the exposure coverage of cold start songs compared with the baseline group.

5 Summary

This part introduces the tag-based song cold-start recommendation method. This method is simple and effective. It is an important part of the NetEase Cloud Music song recommendation system. It is an important cornerstone for building a healthy music distribution ecosystem. Growth provides the first step.

In the follow-up content of this series, we will continue to share more solutions and practical experience of cloud music on the classic problem of cold start, so stay tuned.

This article is published from the NetEase Cloud Music technical team, and any form of reprinting of the article is prohibited without authorization. We recruit all kinds of technical positions all year round. If you are ready to change jobs and happen to like cloud music, then join us at [e-mail]