App quality control

Based on the experience of China and Taiwan, this article focuses on how to do the quality stability of the App. As one of the core services of App, business abnormality monitoring is of course also very important. This is not the focus of this article.

Current status of quality issues

Regarding quality issues, it is directly unfolded in the form of short stories. The following are some thoughts on the quality review of China Mobile's annual review

1. Technical solution level reflects test cases
   For business projects, there will be testing resources, smoking use cases, precision testing, business regression of new QA businesses, UI automation of core businesses, and manual QA regression in the high-speed rail stage. Here is a brief talk about these words. For new business projects, there will be test resources, in short, QA. After the new project passes PRD, MRD, demand discussion meeting, Kick-off, after the technical plan review, it will go through test cases. The result of the review is the use case guide, and then QA will assign it to the corresponding development on the use case platform.
   Under the idea of agile development, business needs follow the car, instead of driving for business projects, the high-speed rail is created every Monday, and you need to buy a ticket and then get on the car. Before getting on the car, for your development branch, accurate testing will be carried out, accurate test reports will be produced, and test reports will be analyzed. If the coverage rate is relatively low, it is necessary to analyze whether there is too much code or QA has not been fully implemented. You can combine the latter for the latter. Use cases, whether there are omissions, and then go to push QA and then return.
   For unchanging business, automated use cases precipitated will go through UI automation testing. During the offline performance monitoring will find some performance problems. QA on duty every week will return to business without distinction.

But ah, most of these are for business. If it is the capabilities and performance of the basic SDK, most of the problems cannot be located. Therefore, there may not be test resources for the technical SDK. It is necessary for developers in the middle and Taiwan to think about the basic SDK itself in the SDK stage. Core use cases, use cases need to think about functional use cases and performance use cases, as well as some issues such as switch situations and version upgrades.
So the first topic is mainly for the basic SDK, but for business projects, in the technical solution stage, what we think about is not test cases, but Skynet alarms (report of business abnormalities), business information (core data), etc.

2. The official component introduces BetterMR
   It is agreed that the business code can be merged from the personal branch to the dev branch only after the business code has been tested (note that the dev branch is not a market branch, and the release branch is a market branch). The submitted MR must be at least +2 before they can be merged. One of them is an old driver of the same technology stack, and the other is the business development of the same project to achieve alignment.

Code quality is directly related to product quality. Code Review is one of the most obvious and feasible measures to ensure code quality, and BetterMR is one of our ways to explore the best Code Review.

Agreement and suggestion
[Convention] All subsequent projects and daily routines will follow the betterMR process by default. If it is relatively simple, you can apply for not to follow the betterMR process;
[Convention] MR classification, the default is ordinary MR, review is completed within 24H; the submitter can choose to be urgent MR, review is to be completed within 2H;
[Agreed] In the follow-up planning, the architect will reserve a certain amount of time on the work assignment to CR;
[Convention] When you are busy and unable to review by @'s reviewer, you can choose to review @ a backup for yourself in the comments;
[Suggestion] After the emergency MR is issued, please bring the MR students to contact and urge the reviewer to respond quickly orally or on the corporate WeChat;
[Suggestion] When reviewer is busy, you can +1 merge first, and then review suggestions later;

Number and choice of reviewers
Agreement and suggestion
[Convention] Each MR @ has two classmates, including the owner of the business domain, and another suitable classmate (familiar with the business or familiar with the code);
[Convention] MR not @ more than two classmates;

Can the small MR process be faster
Agreement and suggestion
[Convention] Quality is the core issue, so for the time being, all projects and routines that go to betterMR will stick to the logic of +2; until our quality data has significantly improved, it means that our quality awareness has improved significantly, and then consider lightweighting;
[Suggestion] Students and reviewers who propose MR can speed up the review process through more effective description, annotation, and communication, such as faster review in the UI part, and focus on the logic part in the review, etc.;

MR code size and effective split
Agreement and suggestion
[Convention] In the technical review and kick-off phases, the workload is logically split MR tasks. The business domain owner checks these two phases, and the split tasks are as granular as possible;
[Convention] Try to submit the relevant logic completely in an MR, which is conducive to the overall review of the code;
[Convention] In the technical review stage, the business domain owner will conduct an internal review of the technical plan and split, and be familiar with the changes and design details in advance to avoid logical omissions outside the code submitted by MR;
[Suggestion] On the premise of ensuring the integrity of the subtask MR logic, try to constrain the code amount of each MR to ensure the review effect;

3. Accurate testing of business SDK access, output reports must be analyzed
   In the first part of the business project, I explained what test actions will be taken for the business. From the perspective of the middle station, I think about how the middle station (such as goods and news) can ensure the quality. A test report for further analysis. If the coverage rate is relatively low, you need to analyze whether there is too much code or QA has not been fully implemented. For the latter, you can combine the use cases to see if there are any omissions, and then go to push QA and go back.

The business middle-stage project that the technical middle-stage is responsible for, that is, the business SDK, also needs to be strictly controlled, otherwise it will be an abnormal business, which will cause online problems or online asset losses.

4. Business projects must be connected to Skynet alarms, and the key processes of the basic SDK are connected to Skynet alarms
   App quality and stability are divided into: performance and quality stability, and business stability. When the business is unstable, it is easy to cause online problems or loss of assets. In response to business exceptions, we have sorted out the online problem attribution, which can generally be divided into:
   The parameter data type of the method or interface is incorrect, the parameter value is not in the legal range, the boundary case is not covered, and other (historical bugs, caused by the three-party SDK upgrade, insufficient communication requirements at the 2 ends are not aligned);
   If we solve the first and second types of problems, online problems will be significantly improved. This is exactly the original intention of Skynet Alarm. Skynet Alarm is used for business abnormal monitoring and alarm. Skynet alarm monitoring is not actively monitored by SDK like APM, but requires developers to sort out key business processes and business scenarios in the current modules, current development projects, and current development daily iterations. For some Report any abnormal cases that may exist.

Therefore, the specification is formulated: business projects must be connected to Skynet to report on alarms. If there is a problem with the basic SDK such as IM, merchandise, and Socket links, then it is an online problem, and it must be a business abnormality. So such key links must be sorted out and reported

5. The coverage rate of the new SDK UT is more than 90%, and the old SDK is approved based on BDD
   Due to limited resources, the legacy SDK may not be able to sort out and write single tests. The old SDK can give behavior to write BDD test cases. I will not describe what BDD is and how to practice it here. For the new SDK in the technical solution stage, it is necessary to think about the test cases and reflect them. The UT coverage rate in the development stage must be greater than 90%.
6. SDK must be passed by Lint
   The lint here is not OCLint for grammar, lock-in, etc., but pod lint. Because there have been some cases, after the MR is submitted, go to the packaging system packaging stage. The packaging fails due to the problem of the pod SDK, so the lint of the pod must be passed to solve the problem in advance.
7. SDK Warning cleanup
   Try to clean up the warning inside the SDK. For example, UIWebView or a certain API used is marked as obsolete by Apple. If you do not modify it on time, if a certain function used by the user is abnormal after going online, then GG
8. Sort out SDK core use cases to ensure access to App integration testing
   The old SDK sorts out core use cases to facilitate BDD testing. All functions of the SDK need to be connected to at least 2 business line apps to verify whether the functions and performance meet expectations
9. SDK Demo must reflect development capabilities, and multi-terminal Demo must be aligned
   The functional design of the SDK and the APIs of the classes are aligned at multiple ends, and the capabilities are consistent. And the core functions can be reflected on the Demo
10. Dirty and poor governance and optimization
    At the end of the year, we count the causes of online problems. It is often found that there are some legacy or inherited online problems, no matter whether it is a business line or a middle office. Therefore, no matter what the reason is, Owner awareness is required, and the mess is sorted out to fix the problem.
11. Ensure that the test case smokes and passes
    The test cases assigned by QA must pass through with smoke. If the smoke is returned to be very serious, this is not serious about quality and disrespect for QA work.
12. The key functions are limited, and the old drivers do the development to avoid the formation of stuck points (schedule) or affect the quality. If it is too busy, at least the old and the new
    For core business functions, it is difficult for newcomers to assess all impact areas and edge cases, so the old drivers are given priority to develop, or the newcomers sort out and evaluate the plan, and the old drivers review and check. Avoid slow progress or online quality problems due to unfamiliarity
13. Basic SDK cross test
    Business projects have QA resources, and the basic SDK does not necessarily have test resources. Developers need to think about test cases, including functions and performance. Finally, they can cross-test. Android and iOS test each other to ensure quality.