oCPX Introduction-"Unmanned Driving" Technology in Advertising

oCPX birth history

When Internet advertisements first appeared, advertisers would buy out advertisements for a placement according to the length of time, or place advertisements according to the number of times the advertisement was displayed. This method is similar to traditional TV advertising. The banners of some websites and the opening ads of most apps that we see now belong to this advertising method, and these advertisers are all advertising for the purpose of brand promotion.

This type of advertisement is characterized by poor personalization. All users will see this advertisement when watching this TV show or opening this web page. Therefore, many men will see cosmetics advertisements on TV, and women will see Nike advertisements, but in fact most of them will see cosmetics advertisements on TV. Under circumstances, they are not interested in this branded product, and advertising traffic will be wasted more. Advertisers who place brand advertisements also don't care much about the waste of traffic, and do not require users to buy goods immediately after reading the advertisement, which enhances the public's awareness of the brand from a longer-term perspective.

Generally, those who can put brand advertisements are large advertisers, who pay more attention to long-term brand effects. But for many small and medium-sized advertisers, they even hope that every input has a point of output. After the emergence of commercial search engines such as Google and Baidu, the performance advertisements with deductions per click have been widely promoted and applied, and have become the main delivery method of Internet advertisements for a long time.

After the advertiser puts the advertisement, the advertisement will appear on the search result page or content recommendation page. The advertisement display itself is free of charge, and the advertisement will be deducted only after the user clicks on the advertisement. This kind of advertisement will be more controllable for advertisers, and an advertisement deduction will bring a user visit, instead of just paying for the advertisement, all other aspects are not guaranteed. And generally speaking, users are interested in clicking on advertising links, and users will have a certain relevance to the advertising content, which will improve advertising conversion and user experience to a certain extent.
For advertising platforms, the exposure of different users can be sold to different advertisers, which is more profitable than selling all user exposures to the same advertiser. On the other hand, it also puts forward higher requirements on platform technology. The platform needs to accurately estimate the click-through rate (ctr) of users for each candidate advertisement, sort them according to ctr*bid, and continuously optimize the ctr estimation model to more reasonably connect users and Advertisers are matched to increase platform revenue.

For a long time, Internet advertising has been a way of advertising based on clicks and charges (CPC). The platform side is more of a diversion function, directing users to advertisers, and using the advertiser's page or App How to market later is the advertiser's own responsibility. In this way, for advertisers, the real conversion cost is actually not so controllable. After clicking on the advertising link, they may not buy anything from the advertiser in the end. On the other hand, some advertisers do some headline party or SEO advertisements, so that the model predicts that the pCtr is high, so as to acquire users at a low price. In fact, it destroys the user experience and also destroys the advertising ecology.

The conversion funnel of an Internet advertisement is very deep. Take a game app as an example. The entire advertisement conversion process simply includes 6 stages: request, exposure, click, download, registration, and payment. We generally deduct advertising costs at the stage of advertising clicks. Advertisers actually want users to pay, and they want to be able to advertise based on how much a paying user pays.

In reality, such fees are charged in the middle of house sale and lease, and a certain percentage of the platform commission is deducted according to the house transaction amount. In this way, no matter how many people are brought to see the room, there is no charge, and only when the transaction is completed. The CPC diversion method in this scenario is: the intermediary brings people to see the house (diversion), if someone is willing to come to see the house, the diversion is successful, and each time it can bring people to the house and charge a diversion fee of 3~5 yuan, finally Whether the transaction is made depends on the homeowner's own marketing methods. Relatively speaking, homeowners will be more inclined to deal with commissions.

We call the method of deducting fees based on conversions as CPA, and there are more problems in operating CPA on the Internet.

1. The conversion behavior is delayed, and the deduction may not be possible when the conversion behavior occurs.
2. Accurate conversion data is in the hands of advertisers, and Internet scenarios cannot accompany the entire process like real intermediaries.
3. Many small and medium-sized advertisers need to be connected, and their technical level is not high, and the process of returning conversion data is prone to problems.

Based on the above 3 points, it is difficult to implement the CPA method on the Internet. Increasing the conversion rate of the back-end advertising and reducing the conversion cost has always been the direction of everyone’s efforts. During the period, advertisers were allowed to choose crowd-targeted tags for advertising, thereby increasing the conversion rate. , But the actual landing also has some unsatisfactory places.

In recent years, leading companies such as Facebook and Toutiao have introduced oCPC delivery methods to protect back-end conversion costs. The model is still advertising deduction when the user clicks, but the deduction price here is not a previous fixed price, but a price calculated according to the conversion rate of the advertising backend; bid=CPA*pCvr, where CPA is the user conversion bid, For example, for a paying user of 100 yuan, pCvr is the probability of a click-to-target conversion (paid) estimated by the model. Because the fee is still deducted when clicking, advertisers are no longer motivated to cheat on returning conversion data. The overall risk is shared by the platform and the customer. Messing up the conversion data will only make the model effect worse and the conversion effect is uncontrollable. At the same time, at the principle level, the cost can be controlled, which can meet the needs of advertisers, and from the perspective of the platform, the traffic distribution is more reasonable and the benefits are greater, and the advertising ecology is better. It is just that the actual engineering of the platform will be more complicated and technically challenging, and the actual conversion cost will deviate from the CPA to a certain extent.

In addition to oCPC, there are also oCPM, oCPD and other delivery methods. The principles are similar and collectively referred to as oCPX.

oCPX algorithm technical architecture

First, it is necessary to establish a real-time conversion data return attribution module to store the conversion data returned by advertisers into categories, and correct attribution (associate the conversion data with the click that brings the conversion). Data is the cornerstone of the algorithm system. Without data, there will be no entire algorithm system behind. The real-time data flow must be stable and reliable.

The second is our model training module, where we will follow up the ad click and conversion data, and model the click conversion rate, that is, establish a model to estimate the conditional probability P (conversion=1|click=1). The model feature level reuses most of the CTR model features, adding cvr statistical features and landing page features to express the relationship between users and conversion behavior. The model will be pushed to the online service to estimate the click conversion rate pCvr of each candidate advertisement.

In order to ensure that the advertising cost is controllable and correct the model's estimated deviation in the advertising dimension, we also need a real-time cost control module. It will count the actual cost of each advertisement on the day and the advertiser's expected cost in real time, and use industrial control algorithms to adjust the overall bid for each advertisement. If the cost of an advertisement is too high, the overall bid of the advertisement will be lowered, so that the actual conversion cost of the advertisement will be consistent with the advertiser's expected cost, and vice versa.

On the online ranking service (ranker), we need to estimate the conversion rate pCvr of each candidate advertisement according to its delivery goal (may be download, activation, registration, payment...), combined with the advertiser's conversion bid CPA and The advertising bid control coefficient k calculated by the control module is used to calculate the intelligent bidding bid for this time the advertisement is clicked bid=pCvr CPA k. After getting the click bid, it will conduct the overall integrated advertising bidding together with other CPC advertisements according to the method of score=pCtr*bid. After several sorting strategies, the winning advertisement is finally selected and sent to the client for advertisement display.

The complexity of the entire data and algorithm system is relatively high, and problems may occur in every link. A debug and case analysis module is essential. Here, some debug data will be reported on the ranker side to enter our case analysis module, which can follow up actual observations and feedback from advertisers for tracking and positioning, and timely adjustment and subsequent optimization of the system.

Introduction to CVR Modeling Technology

The Cvr prediction model is the core of the oCPX system, and its effect will greatly affect the final platform revenue and the deviation of advertising costs.

Before the Cvr model was modeled, there was actually a Ctr prediction model, but one was to model click conversion, and the other was to model exposure clicks. In fact, Cvr modeling still has many technical difficulties to overcome.

1. Higher estimation accuracy requirements
   For pCtr, there is a certain deviation in the technical estimation results, as long as the deviation does not affect the final bidding results, there is no problem; while pCvr directly affects the click bid and cost of the advertisement, even if the deviation is 5%, it will reflect the advertiser's In terms of conversion costs, advertisers have a clear perception. pCvr needs to be estimated more accurately.
   Corresponding to the problem of estimated deviation. Generally speaking, in addition to model optimization, pCvr calibration is required. The pCvr value estimated by the model is compared with the true statistical Cvr value, and the order-preserving regression model is used for calibration.
   
2. The problem of sample sparseness and bias is more serious
   The samples of our training model are all trained based on whether the user clicks or not. These samples are the case of winning the final bid and being clicked by the user, which proves that the user and the advertisement are matched well. In online estimation, all candidate advertisements are estimated. In this way, it is as if we are to make a model for estimating the national income, and the modelling training sample only uses the Shenzhen sample, the problem of sample bias is more serious.
   The sample bias problem can be alleviated and optimized through joint modeling. There are two mainstream models: ESMM and MMoE. At present, it is actually found that the joint modeling of MMoE multiple conversion types can improve the model effect to a certain extent. The effect of ESMM is not obvious.
   

3. Delayed conversion
   At present, the model is updated by incremental training. The delay of deep conversion behavior is relatively large. If special processing is not done, it is easy for a click behavior to occur first, and enter the model as a negative sample for training. After a few days, the user pays and then uses a positive sample. Enter model training. On the one hand, the training is inaccurate, and on the other hand, the conversion rate of new online advertisements is prone to underestimated. This delay problem will be even greater for the Ctr model.
   

Here we believe that 7 days is the maximum window for the return of conversion behavior (which can cover 95%+). First, we will incrementally train a T-7 model, because the conversion data has been fully returned at this time, so this model does not have the problem of delayed conversion.

Then the positive and negative sample data of the last 7 days will be refreshed every day, and the sample weight will be adjusted according to the daily conversion reflux ratio to balance the positive and negative sample ratio. Then we will base the T-7 model, and use the weighted sample data of the last 7 days for incremental training to obtain the T-0 model. In this way, the T-0 model can even out the problem of delayed conversion in the sample ratio.

Introduction to Cost Control Strategy

Targeted means will make the cvr model estimate more accurate, but it is inevitable that there will be deviations in the estimate, especially when the conversion data is relatively sparse, and the actual conversion volume is prone to large fluctuations. Therefore, a cost control module is needed to provide a guarantee for advertisers to achieve cost. Here are a few concepts:

Actual consumption: the amount deducted by the advertiser on the day
Advertiser value: In the case of CPA billing, the amount that should be deducted on the day
Consumption ratio=actual consumption/advertiser value. The control module makes the consumption ratio approach 1.0, so that the actual consumption approaches the advertiser value.
The control strategy module calculates the advertising dimension bid coefficient k, bid=CPA pCvr k, to adjust the advertising dimension bid.

At the beginning, we adopted a proportional control method, k=1.0/consumption ratio. The idea is relatively simple. If the actual consumption is twice the advertiser's value, then every time you bid at half the previous price, the advertiser's expected cost can be achieved. But in fact, the regulation process also affects the consumption ratio. In the end, the above situation can only be used to control the consumption ratio to 1.414 through mathematical derivation, and the consumption ratio cannot be finally brought to 1.0. The overall cost situation of the proportional control has been improved to a certain extent, and the control The algorithm is relatively simple and the control robustness is better.

In the second edition of the control strategy, we adopted the PID control algorithm widely used in industrial control.

The algorithm principle allows the control error to approach 0 through the P term (proportion), I term (integral), and D term (differential), which can effectively solve the problem of proportional control. In fact, the PID algorithm has also achieved a certain cost control effect improvement after the line, but the robustness is slightly worse than that of the P control, and more rules are required for stable operation.

Author profile
Zhu Taipeng OPPO Machine Learning Stargazing Lab Leader
8 years of Internet algorithm work experience, joined OPPO in 2016 to be responsible for information flow advertising algorithms, and in 2020 responsible for the construction of recommendation/advertising machine learning infrastructure.

Get more exciting content: follow the [OPPO Internet Technology] public account