Meituan App page view testability reform practice

The dynamic container technology that writes multiple runs at one time has greatly improved the efficiency of research and development, but for the test process that still requires multi-end verification, it faces great challenges in terms of efficiency. This article focuses on the dynamic layout technology in the dynamic container and explains how to improve the test efficiency through the testability transformation. I hope to bring some inspiration and help to students who also need to test dynamic pages.

Page Features of Meituan App

For different users, the Meituan App page is presented in a variety of ways, which is the so-called "Thousand Faces". Take the "Guess you like" module on the Meituan homepage as an example. There are many different forms such as single column, tab, double column, etc. for different users. With so many different page style requirements, if the development, testing, and online process are to be completed within one day, the R&D team is also facing great challenges. Therefore, test engineers need to rely heavily on automated testing to form a rapid acceptance mechanism.

Technical challenges in the implementation of automated testing

Next, this article will explain from the three dimensions of the page element cannot be positioned, the principle of Appium element positioning, AccessibilityNodeInfo and Drawable.

Page element cannot be positioned

At present, Meituan App client automation mainly relies on Appium (an open source, cross-platform testing framework that can be used to test native and hybrid mobile applications) to realize the positioning and operation of page elements. When we use Appium Inspector to perform page During element review, the only information that can be found through element review is the outer frame and the two buttons below, and other information cannot be identified (as shown in Figure 2 above). Neither the central image nor the text information in the upper left corner can be located and parsed through existing UI automation solutions. If you can't locate the element, you can't perform page operations and assertions, which seriously affects the implementation of automation.

After further investigation, we found that a large number of Drawable objects were used in these page cards to draw page information, which caused elements to be unable to be positioned. Why can't the Drawable object be positioned? Let's study together the principles of UI automation element positioning.

The principle of Appium element positioning

The current UI automation test uses Appium to locate and operate page elements. As shown in the figure below, AppiumServer and UiAutomator2's mobile phone terminal completes the operation of the element after communicating.

By reading the Appium source code, it is found that the process of completing a positioning is shown in the following figure:

• First, Appium locates elements by calling findElement .
• Then, call the findObject method of the UIDevice
• Finally, the element search is completed PartialMatch.accept

Next, let's take a look at how this PartialMatch.accept completes element positioning. Through source code , we found that the element information is stored in an object AccessibilityNodeInfo node.getXXX method is used in the source code. Are you familiar with it? This information is actually the attributes of UI elements that can be obtained in our daily automated testing.

Drawable cannot obtain element information. Is it related to AccessibilityNodeInfo We further explore the relationship between Drawable and AccessibilityNodeInfo

AccessibilityNodeInfo and Drawable

Through the study of the source code, we have drawn the following class diagram to explain the relationship between AccessibilityNodeInfo and Drawable

View implements the AccessibilityEventSource interface and implements a onInitializeAccessibilityNodeInfo to fill in the information. We also found a description of this information in the Android document

onInitializeAccessibilityNodeInfo(): This method provides information about the view state for the accessibility service. The default View implementation has a set of standard view properties, but if your custom view provides TextView or Button , you should replace this method and set other information about the view to this AccessibilityNodeInfo object processed by the method.

And Drawable does not implement the corresponding method, so it cannot be found by automated testing. After exploring the principle of element search, we are about to begin to solve the problem.

Reconstruction of page view testability-XraySDK

Positioning plan comparison

Now that I know that Drawable not filled with AccessibilityNodeInfo , it means that I cannot access the current automated test solution to complete the page content acquisition. Then we can think of the following three solutions to solve the problem:

Implementation plan	Sphere of influence
Transform Appium positioning method so that Drawable can be recognized	Need to change the underlying AccessibilityNodeInfo obtain(View, int) method and add AccessibilityNodeInfo to Drawable so that it needs to be compatible with all Android systems, and the scope of influence is too large
Use View instead of Drawable	Dynamic layout cards use Drawable for drawing because Drawable uses less resources than View and has better drawing performance. Abandoning the use of Drawable is equivalent to giving up performance improvements.
Use image recognition for positioning	There are many images in dynamic cards that contain text, and multiple lines of text will have a great impact on the accuracy of image recognition.

At present, none of the above three solutions can effectively solve the problem of dynamic card element positioning. How to achieve the goal of obtaining view information under the premise of a small influence? Next, we will further study the implementation of dynamic layout.

View information acquisition and storage-XrayDumper

Our application scenario is very clear. The automated test obtains the ability to interact with the client by integrating the Client, and the client sends instructions to the App to obtain page information. Then we can consider embedding an SDK (XraySDK) to complete the view acquisition, and then provide a client (XrayClient) to the automation to complete this part of the function.

The function division of XraySDK is shown in the following table:

Module name	Functional division	Operating environment	Product form
Xray-Client	1. Interact with Xray-Server to send instructions and receive data 2. Expose external APIs to automation or other systems	Inside the app	Client SDK (AAR and Pod-Library)
Xray-SDK	1. Obtain and structure page information (Xray-Dumper) 2. Receive user instructions for structured data output (Xray-Server)	Inside automation or inside a three-party system	JAR package or dependent package based on other languages

How can XraySDK get the Drawable information we need? Let's first study the implementation of dynamic layout.

The view presentation process of dynamic layout is divided into: parsing template->binding data->calculation layout->page drawing. After the calculation layout is over, the position of the element on the page has been determined, so as long as the information at this stage is intercepted, it can be achieved Obtaining view information.

Through the study of the code, we found that the view layout behavior is controlled com.sankuai.litho.recycler.AdapterCompat class, and the final layout and calculation of the view are completed bindViewHolder First, we intercept the layout information by inserting a custom listener here.

public final void bindViewHolder(BaseViewHolder<Data> viewHolder, int position) {
        if (viewHolder != null) {
            viewHolder.bindView(context, getData(position), position);

            //自动化测试回调
            if (componentTreeCreateListeners != null) {
                if (viewHolder instanceof LithoViewHolder) {
                    DataHolder holder = getData(position);
                    //获取视图布局信息
                    LithoView view = ((LithoViewHolder<Data>) viewHolder).lithoView;
                    LayoutController layoutController = ((LithoDynamicDataHolder) holder).getLayoutController(null);
                    VirtualNodeBase node = layoutController.viewNodeRoot;
                    //通过监听器将视图信息向外传递给可测性SDK
                    componentTreeCreateListeners.onComponentTreeCreated(node, view.getRootView(), view.getComponentTree());
                }
            }
        }
    }

Then, complete the initialization of the listener by exposing a static method to the testability SDK.

public static void setComponentTreeCreateListener(ComponentTreeCreateListener l) {
        AdapterCompat.componentTreeCreateListeners = l;
        try {
            // 兼容mbc的动态布局自动化测试，为避免循环依赖，采用反射调用
            Class<?> mbcDynamicClass = Class.forName("com.sankuai.meituan.mbc.business.item.dynamic.DynamicLithoItem");
            Method setComponentTreeCreateListener = mbcDynamicClass.getMethod("setComponentTreeCreateListener", ComponentTreeCreateListener.class);
            setComponentTreeCreateListener.invoke(null, l);

        } catch (Exception e) {
            e.printStackTrace();
        }

        try {
            // 搜索新框架动态布局自动化测试
            Class<?> searchDynamicClass = Class.forName("com.sankuai.meituan.search.result2.model.DynamicItem");
            Method setSearchComponentTreeCreateListener = searchDynamicClass.getMethod("setComponentTreeCreateListener", ComponentTreeCreateListener.class);
            setSearchComponentTreeCreateListener.invoke(null, l);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

Finally, automation completes the acquisition and storage of view information by setting up a custom listener.

//通过静态方法设置一个ComponentTreeCreateListener来监听布局事件
AdapterCompat.setComponentTreeCreateListener(new AdapterCompat.ComponentTreeCreateListener() {
            @Override
            public void onComponentTreeCreated(VirtualNodeBase node, View rootView, ComponentTree tree) {
                //将信息存储到一个自定义的ViewInfoObserver对象中
                ViewInfoObserver vif = new ViewInfoObserver();
                vif.update(node, rootView, tree);
            }
        });

We store the view information in an object like ViewInfoObserver.

public class ViewInfoObserver implements AutoTestObserver{
    public static HashMap<String, View> VIEW_MAP = new HashMap<>();
    public static HashMap<VirtualNodeBase, View> VIEW = new HashMap<>();
    public static HashMap<String, ComponentTree> COMPTREE_MAP = new HashMap<>();
    public static String uri = "http://dashboard.ep.dev.sankuai.com/outter/dynamicTemplateKeyFromJson";

    @Override
    public void update(VirtualNodeBase vn, View view,ComponentTree tree) {
        if (null != vn && null != vn.jsonObject) {
            try {
                String string = vn.jsonObject.toString();
                Gson g = new GsonBuilder().setPrettyPrinting().create();
                JsonParser p = new JsonParser();
                JsonElement e = p.parse(string);

                String templateName = null;
                String name1 = getObject(e,"templateName");
                String name2 = getObject(e,"template_name");
                String name3 = getObject(e,"template");
                templateName = null != name1 ? name1 : (null != name2 ? name2 : (null != name3 ? name3 : null));

                if (null != templateName) {
                //如果已经存储则更新视图信息
                    if (VIEW_MAP.containsKey(templateName)) {
                        VIEW_MAP.remove(templateName);
                    }
                    //存储视图编号
                    VIEW_MAP.put(templateName, view);
                    if (VIEW.containsKey(templateName)) {
                        VIEW.remove(templateName);
                    }
                    //存储视图信息
                    VIEW.put(vn, view);
                    if (COMPTREE_MAP.containsKey(templateName)) {
                        COMPTREE_MAP.remove(templateName);
                    }
                    COMPTREE_MAP.put(templateName, tree);
                    System.out.println("autotestDyn：update success");

                }

            } catch (Exception e) {
                System.out.println(e.toString());
                System.out.println("autotestDyn：templateName not exist!");
            }
        }
    }

When you need to query this information, you can use XrayDumper to complete the output of the information.

public class SubViewInfo {
    public JSONObject getOutData(String template) throws JSONException {
        JSONObject outData = new JSONObject();
        JSONObject componentTouchables = new JSONObject();

        if (!COMPTREE_MAP.isEmpty() && COMPTREE_MAP.containsKey(template) && null != COMPTREE_MAP.get(template)) {
            ComponentTree cpt = COMPTREE_MAP.get(template);
            JSONArray componentArray = new JSONArray();

            ArrayList<View> touchables = cpt.getLithoView().getTouchables();
            LithoView lithoView = cpt.getLithoView();
            int[] ls = new int[2];
            lithoView.getLocationOnScreen(ls);
            int pointX = ls[0];
            int pointY = ls[1];

            for (int i = 0; i < touchables.size(); i++) {
                JSONObject temp = new JSONObject();
                int height = touchables.get(i).getHeight();
                int width = touchables.get(i).getWidth();
                int[] tl = new int[2];
                touchables.get(i).getLocationOnScreen(tl);
                temp.put("height",height);
                temp.put("width",width);
                temp.put("pointX",tl[0]);
                temp.put("pointY",tl[1]);

                String url = "";
                try {
                    EventHandler eh = (EventHandler) getValue(getValue(touchables.get(i), "mOnClickListener"), "mEventHandler");
                    DynamicClickListener listener = (DynamicClickListener) getValue(getValue(eh, "mHasEventDispatcher"), "listener");
                    Uri clickUri = (Uri) getValue(listener, "uri");
                    if (null != clickUri) {
                        url = clickUri.toString();
                    }
                } catch (Exception e) {
                    Log.d("autotest", "get click url error!");
                }

                temp.put("url",url);
                componentArray.put(temp);
            }
            componentTouchables.put("componentTouchables",componentArray);
            componentTouchables.put("componentTouchablesCount", cpt.getLithoView().getTouchables().size());

            View[] root = (View[])getValue(cpt.getLithoView(),"mChildren");
            JSONArray allComponentArray = new JSONArray();
            if (root.length > 0) {
                for (int i = 0; i < root.length; i++) {
                    try {
                        if (null != root[i]) {
                            Object items[] = (Object[]) getValue(getValue(root[i], "mMountItems"), "mValues");
                            componentTouchables.put("componentCount", items.length);
                            for (int itemIndex = 0; itemIndex < items.length; itemIndex++) {
                                getMountItems(allComponentArray, items[itemIndex], pointX, pointY);
                            }
                        }
                    } catch (Exception e) {

                    }
                }
            }
            componentTouchables.put("componentUntouchables",allComponentArray);
        } else {
            Log.d("autotest","COMPTREE_MAP is null!");
        }
        outData.put(template,componentTouchables);
        System.out.println(outData);
        return outData;
    }
    }
}

View information output-XrayServer

We have obtained the information, and then we must consider how to pass the view information to the automated test script. We refer to the design of Appium.

Appium starts a SocketServer through the InstrumentsClient installed on the mobile phone to complete the data communication between the automation and the underlying test framework through the HTTP protocol. We can also learn from the above ideas and start a WebServer in the Meituan App to complete the output of information.

In the first step, we have implemented a function that inherits the Service component, so that you can start and stop testability easily through the command line.

public class AutoTestServer extends Service  {
    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
    ....
        return super.onStartCommand(intent, flags, startId);
    }
}

The second step is to expose the communication interface through HttpServer.

public class AutoTestServer extends Service  {
    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        // 创建对象，端口通过参数传入
        if (intent != null) {
            int randNum = intent.getIntExtra("autoTestPort",8999);
            HttpServer myServer = new HttpServer(randNum);
            try {
                // 开启HTTP服务
                myServer.start();
                System.out.println("AutoTestPort:" + randNum);
            } catch (IOException e) {
                System.err.println("AutoTestPort:" + e.getMessage());
                myServer = new HttpServer(8999);
                try {
                    myServer.start();
                    System.out.println("AutoTestPort:8999");
                } catch (IOException e1) {
                    System.err.println("Default:" + e.getMessage());
                }
            }
        }
        return super.onStartCommand(intent, flags, startId);
    }
}

The third step is to register the listener set before.

public class AutoTestServer extends Service  {
    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
    //注册监听器
        AdapterCompat.setComponentTreeCreateListener(new AdapterCompat.ComponentTreeCreateListener() {
            @Override
            public void onComponentTreeCreated(VirtualNodeBase node, View rootView, ComponentTree tree) {
                ViewInfoObserver vif = new ViewInfoObserver();
                vif.update(node, rootView, tree);
            }
        });

        // 创建对象，端口通过参数传入
        .....
        return super.onStartCommand(intent, flags, startId);
    }
}

Finally, through different paths in HttpServer to receive different instructions.

private JSONObject getResponseByUri(@Nonnull IHTTPSession session) throws JSONException {
        String uri = session.getUri();
        if (isFindCommand(uri)) {
            return getResponseByFindUri(uri);
        }
}

@Nonnull
private JSONObject getResponseByFindUri(@Nonnull String uri) throws JSONException {
    String template = uri.split("/")[2];
    String protocol = uri.split("/")[3];
    switch (protocol) {
        case "frame":
            TemplateLayoutFrame tlf = new TemplateLayoutFrame();
            return tlf.getOutData(template);
        case "subview":
            SubViewInfo svi = new SubViewInfo();
            return svi.getOutData(template);
        //省略了部分的代码处理逻辑    
        ....
        default:
            JSONObject errorJson = new JSONObject();
            errorJson.put("success", false);
            errorJson.put("message", "输入find链接地址有误");
            return errorJson;
    }
}

The overall functional structure of the SDK

The automated script accesses a specific port of the device (for example: http://localhost:8899/find/subview), via XrayServer, and forwards the request to XrayDumper through the access path for information extraction and output. Then the layout parser serializes the layout information into JSON data, and then transmits it to the automated test script via the XrayServer via the network in the form of HTTP response.

View information enhancement

In addition to the conventional location, content, type and other information, we also check the time listener to further determine whether the view elements can interact, and further enhance the effective information of the page view structure.

// setGestures
ArrayList<String> gestures = new ArrayList<>();
if (view.isClickable()){
   gestures.add("isClickable");
}
if (view.isLongClickable()){
   gestures.add("isLongClickable");
}
//省略部分代码
.....

The benefits of dynamic layout automation

Based on the improvement of view testability, the automated test coverage of Meituan's dynamic cards has been greatly improved. From the original inability to do automated testing, more than 80% of the current dynamic cards have achieved automated testing, and the efficiency has also been achieved. Significant improvement.

Future outlook

As one of the most basic and important attributes of client-side testing, page view information is a code-level representation of user visual information. It plays a very important role in machine identification of page element information, and its measurability transformation will bring great benefits to the technical team. We will list several exploration directions for view testability transformation for your reference only.

Use view analysis principle to solve WebView element positioning

Applying the same idea, we can also use it to solve the problem of WebView element positioning.

Through the SDK running inside the App, the corresponding WebView instance can be obtained. By obtaining the root node, cyclically traversing from the root node, and storing the information of each node at the same time, all the view information can be obtained.

Is there an equally suitable root node in WebView? Based on the understanding of HTML, we can think that all the tags in HTML are hung under the BODY tag, and the BODY tag is the root node we need to select. We can get the attributes through WebElement["attrName"].

More application scenarios for view testability transformation

• improves the reliability of functional testing. : In functional testing automation, the stability of automated testing can be effectively improved through the more stable and rapid internal view information output. Avoid automated test failures due to unavailable elements or slow access to elements.
• Improve the efficiency of reliability testing : For reliability tests that rely on random or random page operations based on view information, rely on the filtering of view information, or only operate elements that can be interacted (by filtering whether the element event listener is empty) . In this way, the efficiency of reliability testing can be effectively improved, and more pages can be tested within a unit time.
• adds compatibility test detection means : In terms of page compatibility, scan the page for unreasonable stacking, blank areas, abnormal shapes and other UI presentation abnormalities through the location information and attributes of page components. You can also obtain content information, such as pictures, text, to check whether there is inappropriate content presentation. Can be used as an effective supplement to the image comparison program.

Job Offers

The Meituan Platform Quality Technology Center is responsible for the basic technical quality of the Meituan App business and the front-end (mobile client and Web front-end), precipitating process specifications and supporting tools, and improving R&D efficiency. The team has first-class skills and a good atmosphere. Interested students can send their resumes to: [e-mail]

Read more technical articles from the

the front | algorithm | backend | data | security | operation and maintenance | iOS | Android | test

| in the public account menu bar dialog box, and you can view the collection of technical articles from the Meituan technical team over the years.

| This article is produced by the Meituan technical team, and the copyright belongs to Meituan. Welcome to reprint or use the content of this article for non-commercial purposes such as sharing and communication, please indicate "the content is reproduced from the Meituan technical team". This article may not be reproduced or used commercially without permission. For any commercial activity, please send an email to [e-mail] to apply for authorization.