Use Arthas to troubleshoot open source Excel components

Introduction to actual use of 160e574203a7be, it is inevitable that Arthas can dynamically monitor our code while the program is running? With this question, let's take a look at the implementation principle of Java Agent technology.

​

Background introduction

​

com.github.dreamroute excel-helper tool is used in the project to assist in the parsing of Excel files. The code at the time of error is written like this: as shown below (non-source code)

     try {            excelDTOS = ExcelHelper.importFromFile(ExcelType.XLSX, file, ExcelDTO.class);        } catch (Exception e) {            log.error("ExcelHelper importFromFile exception msg {}", e.getMessage());        }

​

Because the e.getMessage() method is used when printing the exception information, the exception information is not printed. And the local recurrence has not reproduced either. So we can only consider using arthas to help troubleshoot this problem.
​

Investigation process

​

1. Install Arthas on the online server.
https://arthas.aliyun.com/doc/install-detail.html

2. Use the watch command to monitor the specified method and print out the abnormal stack information. The command is as follows:

watch com.github.dreamroute.excel.helper.ExcelHelper importFromFile '{params,throwExp}' -e -x 3

Call the method again, and capture the exception stack information as follows:

The exception has been caught, and the stack information is printed out.
​

3. According to the corresponding stack information, locate the specific code, as follows:

The code is very simple. It can be clearly seen from the code that if the specified headerInfo is not obtained from the headerInfoMap, this exception will be thrown. There are only two cases:
​

• The information stored in headerInfoMap is incorrect.
• The columnIndex in the cell exceeds the normal range, resulting in the corresponding HeaderInfo not being obtained.

For the second case, first check whether there is any problem with the uploaded Excel file, and test the Excel file locally, and there is no problem. The local test is also successful, so subjective judgment is that the second case is unlikely.

Therefore, it is mainly to check whether the first situation occurs. At this time, you can look at the first line of code of the method.
​

MapheaderInfoMap = processHeaderInfo(rows,cls);

You can see that the headerInfoMap is obtained through processHeaderInfo. Find the code of processHeaderInfo as shown below.

public static MapproceeHeaderInfo(Iteratorrows, Class cls) {
    if (rows.hasNext()) {
        Row header = rows.next();
        return CacheFactory.findHeaderInfo(cls, header);
    }
    return new HashMap<>(0);
}
public static MapfindHeaderInfo(Class cls, Row header) {
    MapheaderInfo = HEADER_INFO.get(cls);
    if (MapUtils.isEmpty(headerInfo)) {
        headerInfo = ClassAssistant.getHeaderInfo(cls, header);
        HEADER_INFO.put(cls, headerInfo);
    }
    return headerInfo;
}
public static MapgetHeaderInfo(Class cls, Row header) {
    IteratorcellIterator = header.cellIterator();
    Listfields = ClassAssistant.getAllFields(cls);
    MapheaderInfo = new HashMap<>(fields.size());
    while (cellIterator.hasNext()) {
        org.apache.poi.ss.usermodel.Cell cell = cellIterator.next();
        String headerName = cell.getStringCellValue();
        for (Field field : fields) {
            Column col = field.getAnnotation(Column.class);
            String name = col.name();
            if (Objects.equals(headerName, name)) {
                HeaderInfo hi = new HeaderInfo(col.cellType(), field);
                headerInfo.put(cell.getColumnIndex(), hi);
                break;
            }
        }
    }

    return headerInfo;
}

​

It is mainly generated by the findHeaderInfo of the CacheFactory class. In the findHeaderInfo method, a HEADER\_INFO variable modified by static final is used for caching. When it is called, it will first go to HEADER\_INFO to check. If it has, it will return directly. If it is not, it will be recreated. (That means that the same Excel file only initializes HeaderInfo once). The steps of creation are in the ClassAssistant.getHeaderInfo() method.

Take a brief look at the generation process of HeaderInfo, compare each Cell value in the first row of the Excel file with the annotation of the custom entity class. If the name is the same, it will be saved as a key-value pair (the data structure of HeaderInfo is HashMap).

4. At this time, you need to confirm the ExcelDTO.class-related HeaderInfo saved in HEADER\_INFO. Use the ognl command or getstatic command to view. The ognl command is used here.
​

ognl '#value=new com.tom.dto.ExcelDTO(),#valueMap=@com.github.dreamroute.excel.helper.cache.CacheFactory@HEADER_INFO,#valueMap.get(#value.getClass()).entrySet().iterator.{#this.value.name}'

The results are as follows: Under normal circumstances, this Excel file has 6 columns of information, why only 4 key-value pairs are generated? If an error is stored in HEADER\_INFO, from the above logic, the correct Excel file uploaded later will throw an error during parsing.

5. I asked a colleague who found this problem at the time, and learned that the Excel file he uploaded for the first time was faulty, and I wanted to correct it later, but the problem occurred when I uploaded it again. The problem is found here.
​

The principle of Arthas

​

After actual use, I can't help but think of how Arthas can dynamically monitor our code while the program is running? With this question, let's take a look at the implementation principle of Java Agent technology.

​

Java Agent technology

​

Agent is a specific program running on the target JVM. Its responsibility is to obtain data from the target JVM and then pass the data to an external process. The time to load the Agent can be when the target JVM is started, or it can be loaded when the target JVM is running, and loading the Agent when the target JVM is running is dynamic.
​

Basic concepts

​

• JVMTI (JVM Tool Interface): It is a collection of interfaces exposed by the JVM for user extension. JVMTI is event-driven. Every time the JVM executes a certain logic, it will call some event callback interfaces (if any). These The interface can be used by developers to extend their own logic.
• JVMTIAgent (JVM Tool Interface): It is a dynamic library that uses some interfaces exposed by JVMTI to help us load the Agent into the target JVM by using the JVM Attach mechanism when the program is started or when the program is running.
• JPLISAgent (Java Programming Language Instrumentation Services Agent): Its role is to initialize all agents written through the Java Instrumentation API, and it also assumes the responsibility of implementing the API exposed in Java Instrumentation through JVMTI.
• VirtualMachine: Provides the Attach action and the Detach action, allowing us to remotely connect to the JVM through the attach method, and then register an agent agent to the JVM through the loadAgent method, and an Instrumentation instance will be obtained in the agent's agent. This instance can Change the bytecode of the class before the class is loaded, or reload it after the class is loaded.
• Instrumentation: The bytecode of the class can be changed before the class is loaded (premain), or it can be reloaded after the class is loaded (agentmain).

​

Implementation process

​

Write a Demo

​

Through javassist, change the code of the specified method at runtime, and add custom logic before and after the method.
​

1. Define the Agent class. Currently Java provides two ways to inject code into the JVM. Here, our Demo chooses to use the agentmain method.

premain: Inject the agent into the specified JVM through the javaagent command at startup.
agentmain: activate the specified agent through the attach tool at runtime.
​

/**
 * AgentMain
 *
 * @author tomxin
 */
public class AgentMain {

    public static void agentmain(String agentArgs, Instrumentation instrumentation) throws UnmodifiableClassException, ClassNotFoundException {
        instrumentation.addTransformer(new InterceptorTransformer(agentArgs), true);
        Class clazz = Class.forName(agentArgs.split(",")[1]);
        instrumentation.retransformClasses(clazz);
    }
}

/**
 * InterceptorTransformer
 *
 * @author tomxin
 */
public class InterceptorTransformer implements ClassFileTransformer {

    private String agentArgs;

    public InterceptorTransformer(String agentArgs) {
        this.agentArgs = agentArgs;
    }

    @Override
    public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
        //javassist的包名是用点分割的，需要转换下
        if (className != null && className.indexOf("/") != -1) {
            className = className.replaceAll("/", ".");
        }
        try {
            //通过包名获取类文件
            CtClass cc = ClassPool.getDefault().get(className);
            //获得指定方法名的方法
            CtMethod m = cc.getDeclaredMethod(agentArgs.split(",")[2]);
            //在方法执行前插入代码
            m.insertBefore("{ System.out.println(\"=========开始执行=========\"); }");
            m.insertAfter("{ System.out.println(\"=========结束执行=========\"); }");
            return cc.toBytecode();
        } catch (Exception e) {

        }
        return null;
    }
}

2. Use Maven to configure the MANIFEST.MF file, which can specify the main method of the Jar package.

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.3.1</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                        </manifest>
                        <manifestEntries>
                            <Agent-Class>com.tom.mdc.AgentMain</Agent-Class>
                            <Can-Redefine-Classes>true</Can-Redefine-Classes>
                            <Can-Retransform-Classes>true</Can-Retransform-Classes>
                        </manifestEntries>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>

3. Define the Attach method, and specify the class to be agented through VirtualMachine.attach(#{pid}).
​

import com.sun.tools.attach.VirtualMachine;

import java.io.IOException;

/**
 * AttachMain
 *
 * @author tomxin
 */
public class AttachMain {
    public static void main(String[] args) {
        VirtualMachine virtualMachine = null;
        try {
            virtualMachine = VirtualMachine.attach(args[0]);
            // 将打包好的Jar包，添加到指定的JVM进程中。
            virtualMachine.loadAgent("target/agent-demo-1.0-SNAPSHOT.jar",String.join(",", args));
        } catch (Exception e) {
            if (virtualMachine != null) {
                try {
                    virtualMachine.detach();
                } catch (IOException ex) {
                    ex.printStackTrace();
                }
            }
        }
    }
}

​

4. Define the test method
​

package com.tom.mdc;
import java.lang.management.ManagementFactory;
import java.util.Random;
import java.util.concurrent.TimeUnit;

/**
 * PrintParamTarget
 *
 * @author toxmxin
 */
public class PrintParamTarget {

    public static void main(String[] args) {
        // 打印当前进程ID
        System.out.println(ManagementFactory.getRuntimeMXBean().getName());
        Random random = new Random();
        while (true) {
            int sleepTime = 5 + random.nextInt(5);
            running(sleepTime);
        }
    }

    private static void running(int sleepTime) {
        try {
            TimeUnit.SECONDS.sleep(sleepTime);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println("running sleep time " + sleepTime);
    }
}

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.