Fastjson deserialization randomness fails

This article mainly talks about a "random" deserialization error!

foreword

As a high-performance JSON serialization framework, Fastjson has many usage scenarios, but there are also some potential bugs and deficiencies. This article mainly talks about a "random" deserialization error!

problem code

In order to clearly describe the ins and outs of the entire error report, paste the relevant code, and at the same time, in order to execute it locally, take a look at the actual effect.

StewardTipItem

 package test;

import java.util.List;

public class StewardTipItem {

    private Integer type;

    private List<String> contents;

    public StewardTipItem(Integer type, List<String> contents) {
        this.type = type;
        this.contents = contents;
    }
}

StewardTipCategory

Fails on deserialization, this class has two special features:

• Returns the build method of the StewardTipCategory (ignoring the return null value).
• The constructor "C1" Map<Integer, List<String>> items parameter has the same name as the List<StewardTipItem> items property, but the type is different!

 package test;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class StewardTipCategory {

    private String category;

    private List<StewardTipItem> items;

    public StewardTipCategory build() {
        return null;
    }

    //C1 下文使用C1引用该构造函数
    public StewardTipCategory(String category, Map<Integer, List<String>> items) {
        List<StewardTipItem> categoryItems = new ArrayList<>();
        for (Map.Entry<Integer, List<String>> item : items.entrySet()) {
            StewardTipItem tipItem = new StewardTipItem(item.getKey(), item.getValue());
            categoryItems.add(tipItem);
        }
        this.items = categoryItems;
        this.category = category;
    }

    // C2 下文使用C2引用该构造函数
    public StewardTipCategory(String category, List<StewardTipItem> items) {
        this.category = category;
        this.items = items;
    }

    public String getCategory() {
        return category;
    }

    public void setCategory(String category) {
        this.category = category;
    }

    public List<StewardTipItem> getItems() {
        return items;
    }

    public void setItems(List<StewardTipItem> items) {
        this.items = items;
    }
}

StewardTip

 package test;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class StewardTip {

    private List<StewardTipCategory> categories;

    public StewardTip(Map<String, Map<Integer, List<String>>> categories) {
        List<StewardTipCategory> tipCategories = new ArrayList<>();
        for (Map.Entry<String, Map<Integer, List<String>>> category : categories.entrySet()) {
            StewardTipCategory tipCategory = new StewardTipCategory(category.getKey(), category.getValue());
            tipCategories.add(tipCategory);
        }
        this.categories = tipCategories;
    }

    public StewardTip(List<StewardTipCategory> categories) {
        this.categories = categories;
    }

    public List<StewardTipCategory> getCategories() {
        return categories;
    }

    public void setCategories(List<StewardTipCategory> categories) {
        this.categories = categories;
    }
}

JSON string

 {
    "categories":[
        {
            "category":"工艺类",
            "items":[
                {
                    "contents":[
                        "工艺类-提醒项-内容1",
                        "工艺类-提醒项-内容2"
                    ],
                    "type":1
                },
                {
                    "contents":[
                        "工艺类-疑问项-内容1"
                    ],
                    "type":2
                }
            ]
        }
    ]
}

FastJSONTest

 package test;

import com.alibaba.fastjson.JSONObject;

public class FastJSONTest {

    public static void main(String[] args) {
        String tip = "{\"categories\":[{\"category\":\"工艺类\",\"items\":[{\"contents\":[\"工艺类-提醒项-内容1\",\"工艺类-提醒项-内容2\"],\"type\":1},{\"contents\":[\"工艺类-疑问项-内容1\"],\"type\":2}]}]}";
        try {
            JSONObject.parseObject(tip, StewardTip.class);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

stack info

When executing the main method of FastJSONTest, an error is reported:

 com.alibaba.fastjson.JSONException: syntax error, expect {, actual [
  at com.alibaba.fastjson.parser.deserializer.MapDeserializer.parseMap(MapDeserializer.java:228)
  at com.alibaba.fastjson.parser.deserializer.MapDeserializer.deserialze(MapDeserializer.java:67)
  at com.alibaba.fastjson.parser.deserializer.MapDeserializer.deserialze(MapDeserializer.java:43)
  at com.alibaba.fastjson.parser.deserializer.DefaultFieldDeserializer.parseField(DefaultFieldDeserializer.java:85)
  at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:838)
  at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:288)
  at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:284)
  at com.alibaba.fastjson.parser.deserializer.ArrayListTypeFieldDeserializer.parseArray(ArrayListTypeFieldDeserializer.java:181)
  at com.alibaba.fastjson.parser.deserializer.ArrayListTypeFieldDeserializer.parseField(ArrayListTypeFieldDeserializer.java:69)
  at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:838)
  at com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer.deserialze(JavaBeanDeserializer.java:288)
  at com.alibaba.fastjson.parser.DefaultJSONParser.parseObject(DefaultJSONParser.java:672)
  at com.alibaba.fastjson.JSON.parseObject(JSON.java:396)
  at com.alibaba.fastjson.JSON.parseObject(JSON.java:300)
  at com.alibaba.fastjson.JSON.parseObject(JSON.java:573)
  at test.FastJSONTest.main(FastJSONTest.java:17)

Troubleshoot

There are two difficulties in the troubleshooting process:

• The key, position or other valuable prompt information of the JSON string at the time of the exception cannot be obtained from the error message.
• Error reporting does not happen every time it is executed, and there is randomness. It may report two or three errors after ten executions, and there is no statistical failure rate.

After many executions, some clues were found! The whole process is briefly described below in combination with the source code, and finally, how to debug the code when an error is reported will be given.

JavaBeanInfo: line 285

When clazz is StewardTipCategory.class, ask the following two questions: Q1: What is the return value of the Constructor[] constructors array? Q2: What is the order of the elements of the constructors array? Refer to the annotation of java.lang.Class#getDeclaredConstructors to get A1:

• A1
  public test.StewardTipCategory(java.lang.String,java.util.Map<java.lang.Integer, java.util.List<java.lang.String>>)『C1』
  public test.StewardTipCategory(java.lang.String,java.util.List<test.StewardTipItem>) "C2"
• A2
  The order of the build() method, C1 constructor, and C2 constructor in the Java source file determines the order of the elements of the constructors array!

The following table is a set of data obtained after many experiments. Because it is triggered manually and the number of times is small, 100% accuracy cannot be guaranteed, but it is only a high probability event.

The underlying implementation of java.lang.Class#getDeclaredConstructors is native getDeclaredConstructors0. This part of the JVM code has not been read, so it is currently impossible to explain the reason for this phenomenon.

forward	middle	back	array element order
build()	C1	C2	random
C1	build()	C2	C2,C1
C1	C2	build()	C2,C1
build()	C2	C1	random
C2	build()	C1	C1,C2
C2	C1	build()	C1,C2
C1		C2	C2,C1
C2		C1	C1,C2

It is precisely because java.lang.Class#getDeclaredConstructors returns the randomness of the array element order that the randomness of the deserialization failure is caused!

• [C2,C1] Deserialization succeeded!

• [C1,C2] Deserialization failed!
  [C1, C2] order to explore the path of code execution when deserialization fails.

  JavaBeanInfo: line 492

  
  The com.alibaba.fastjson.util.JavaBeanInfo#build() method has a large amount of code, ignoring irrelevant code on the execution path.

• [C1, C2] The code will be executed to line 492 and executed twice (StewardTipCategory#category, StewardTipCategory#items are executed once each).
• Create a com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer after the end.

JavaBeanDeserializer: line 49

Two important properties of JavaBeanDeserializer:

• private final FieldDeserializer[] fieldDeserializers;
• protected final FieldDeserializer[] sortedFieldDeserializers;

Details of fieldDeserializers when deserializing test.StewardTipCategory#items.

com.alibaba.fastjson.parser.deserializer.DefaultFieldDeserializer
com.alibaba.fastjson.parser.deserializer.DefaultFieldDeserializer#fieldValueDeserilizer
(The attribute value is null, and the runtime will obtain the specific implementation class according to the fieldType)
com.alibaba.fastjson.util.FieldInfo#fieldType
(java.util.Map<java.lang.Integer, java.util.List<java.lang.String>>)

Created and executed com.alibaba.fastjson.parser.deserializer.JavaBeanDeserializer#deserialze(com.alibaba.fastjson.parser.DefaultJSONParser, java.lang.reflect.Type, java.lang.Object, java.lang.Object, int, int [])

JavaBeanDeserializer: line 838

DefaultFieldDeserializer: line 53

com.alibaba.fastjson.parser.ParserConfig#getDeserializer(java.lang.Class<?>, java.lang.reflect.Type) Set the specific implementation class of com.alibaba.fastjson.parser.deserializer.DefaultFieldDeserializer#fieldValueDeserilizer according to the field type .

DefaultFieldDeserializer: line 34

The actual type of the test.StewardTipCategory#items property is List<StewardTipItem>.

The implementation class of fieldValueDeserilizer obtained according to the C1 constructor during deserialization is com.alibaba.fastjson.parser.deserializer.MapDeserializer.
An error is reported when com.alibaba.fastjson.parser.deserializer.MapDeserializer#deserialze(com.alibaba.fastjson.parser.DefaultJSONParser, java.lang.reflect.Type, java.lang.Object) is executed.

MapDeserializer: Line 228

JavaBeanDeserializer: line 838

java.lang.Class#getDeclaredConstructors returns [C2,C1] order,
The implementation class of fieldValueDeserilizer obtained according to the C2 constructor during deserialization is com.alibaba.fastjson.parser.deserializer.ArrayListTypeFieldDeserializer, and the deserialization is successful.

problem solved

code

• Remove the C1 constructor and create the StewardTipCategory in another way.
• Modify the C1 constructor parameter name and type to avoid misleading Fastjson.

debugging

 package test;

import com.alibaba.fastjson.JSONObject;

import java.lang.reflect.Constructor;

public class FastJSONTest {

    public static void main(String[] args) {
        Constructor<?>[] declaredConstructors = StewardTipCategory.class.getDeclaredConstructors();
        // if true must fail！
       if ("public test.StewardTipCategory(java.lang.String,java.util.Map<java.lang.Integer, java.util.List<java.lang.String>>)".equals(declaredConstructors[0].toGenericString())) {
          String tip = "{\"categories\":[{\"category\":\"工艺类\",\"items\":[{\"contents\":[\"工艺类-提醒项-内容1\",\"工艺类-提醒项-内容2\"],\"type\":1},{\"contents\":[\"工艺类-疑问项-内容1\"],\"type\":2}]}]}";
            try {
                JSONObject.parseObject(tip, StewardTip.class);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}

Summarize

In the development process, try to follow the specifications/protocols, and don't be maverick

The StewardTipCategory constructor C1 method signature is obviously not a good choice. In addition to attribute assignment, the method body also does some additional type/data conversion, which should be avoided as much as possible.

Professional with depth

Developers should have in-depth research on the technologies and frameworks used, especially the underlying principles, and cannot stay at the use level. Something obscure can cause incredible problems: java.lang.Class#getDeclaredConstructors.

fastjson

The framework should be implemented rigorously, and the error message should be as clear as possible. The reason for the failure of StewardTipCategory deserialization is that fastjson only checks the property name and the number of constructor parameters without further checking the property type.

<<Refactoring: Improve the design of existing code>> It is recommended that the code method blocks be as short and concise as possible, and the methods of some modules of Fastjson are too bloated.

Life has a limit but knowledge not