A troubleshooting of GSON time issues

Problem Description

Recently, an old project frequently encountered GSON deserialization time issues in the test environment. The error stack is as follows:

Exception in thread "main" com.google.gson.JsonSyntaxException: 2021-05-14 14:59:37
    at com.google.gson.internal.bind.DateTypeAdapter.deserializeToDate(DateTypeAdapter.java:81)
    at com.google.gson.internal.bind.DateTypeAdapter.read(DateTypeAdapter.java:66)
    at com.google.gson.internal.bind.DateTypeAdapter.read(DateTypeAdapter.java:41)
    at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
    at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
    at com.google.gson.Gson.fromJson(Gson.java:795)
    at com.google.gson.Gson.fromJson(Gson.java:761)
    at com.google.gson.Gson.fromJson(Gson.java:710)
    at com.google.gson.Gson.fromJson(Gson.java:682)
    at com.gson.GsonDate.main(GsonDate.java:17)
Caused by: java.text.ParseException: Unparseable date: "2021-05-14 14:59:37"
    at java.text.DateFormat.parse(DateFormat.java:366)
    at com.google.gson.internal.bind.DateTypeAdapter.deserializeToDate(DateTypeAdapter.java:79)
    ... 9 more

The error description is also very detailed, that is, when GSON deserializes a Json string, because the string cannot be deserialized at a certain time, the final Json deserialization fails;

Background note

Because the data volume of this system is relatively large, all the data from six months ago will be archived in Hbase. When archiving, the data in the database will be serialized into json format and then saved in Hbase; if it is the recent six months of data, it will be directly queried. In the database, Hbase will only be queried if the data is very early, so the probability of occurrence is relatively low;

problem analysis

Local reproduction

In order to facilitate the analysis, directly copy the Json string to the local, and then reproduce it locally, and then analyze the problem. The Json string is relatively long, and the following Json string is used instead:

{"date":"2021-05-14 14:59:37"}

Prepare the relevant code as follows:

public class GsonDate {
    public static void main(String[] args) {
        String json = "{\"date\":\"2021-05-14 14:59:37\"}";
        GsonDateBean date = new Gson().fromJson(json, GsonDateBean.class);
        System.out.println(date);
    }
}

@Data
class GsonDateBean {
    private Date date;
}

The result of the execution is that the reverse sequence can be successful without the above error. In order to find out the reason, we need to analyze the relevant source code of Gson time conversion;

Source code analysis

The source code of Gson time conversion is relatively simple. Part of the code of DateTypeAdapter

  private final DateFormat enUsFormat
      = DateFormat.getDateTimeInstance(DateFormat.DEFAULT, DateFormat.DEFAULT, Locale.US);
  private final DateFormat localFormat
      = DateFormat.getDateTimeInstance(DateFormat.DEFAULT, DateFormat.DEFAULT);
  private final DateFormat iso8601Format = buildIso8601Format();

  private static DateFormat buildIso8601Format() {
    DateFormat iso8601Format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.US);
    iso8601Format.setTimeZone(TimeZone.getTimeZone("UTC"));
    return iso8601Format;
  }

  private synchronized Date deserializeToDate(String json) {
    try {
      return localFormat.parse(json);
    } catch (ParseException ignored) {
    }
    try {
      return enUsFormat.parse(json);
    } catch (ParseException ignored) {
    }
    try {
      return iso8601Format.parse(json);
    } catch (ParseException e) {
      throw new JsonSyntaxException(json, e);
    }
  }

Gson has prepared three DateFormat , namely: localFormat, enUsFormat, iso8601Format; the conversion is carried out in this order during conversion, and whichever can be converted will be returned directly. The above problems indicate that the three types of DateFormat have not been converted successfully; local debugging can be directly converted. Debug comes in, you can find that the conversion is successful when you use localFormat directly, and you can view each pattern ;

• localFormat：yyyy-M-d H:mm:ss
• enUsFormat：MMM d, yyyy h:mm:ss a
• iso8601Format：yyyy-MM-dd'T'HH:mm:ss'Z'

The above date format completely conforms to the yyyy-M-d H:mm:ss format, so it can be directly converted successfully; it can be found that the localFormat is actually related to the local system's language environment, so there will be inconsistencies between the local running results and the server running results;

Reproduce again

You can set the language environment directly through the code, and set the environment to Locale.US

public class GsonDate {
    public static void main(String[] args) {
        System.out.println("默认："+Locale.getDefault());
        System.out.println("重置语言环境：Locale.US");
        Locale.setDefault(Locale.US);
        String json = "{\"date\":\"2021-05-14 14:59:37\"}";
        GsonDateBean date = new Gson().fromJson(json, GsonDateBean.class);
        System.out.println(date);
    }
}

Running the above code, the same anti-sequence time problem as the server appears:

默认：zh_CN
重置语言环境：Locale.US
Exception in thread "main" com.google.gson.JsonSyntaxException: 2021-05-14 14:59:37
    at com.google.gson.internal.bind.DateTypeAdapter.deserializeToDate(DateTypeAdapter.java:81)

It can be found that our local environment is generally zh_CN , corresponding to Locale.CHINA ;

problem solved

System Configuration

You can directly change the system language environment, liunx can be configured /etc/sysconfig/i18n

英文版系统：
LANG="en_US.UTF-8"
中文版系统：
LANG="zh_CN.UTF-8"

You can view the currently configured locale:

[root@Centos ~]# echo $LANG
en_US.UTF-8

Code

You can set the default log conversion format for Gson:

Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss").create();
GsonDateBean date = gson.fromJson(json, GsonDateBean.class);

Expand

Similarly, if you use other Json serialization tools, such as fastjson whether there is such a problem, you can simply do a test:

Locale.setDefault(Locale.US);
String json = "{\"date\":\"2021-05-14 14:59:37\"}";
String json2 = "{\"date\":\"2021年05月14日 14:59:37\"}";
JacksonDateBean date = JSON.parseObject(json, JacksonDateBean.class);

The result is not only yyyy-MM-dd HH:mm:ss format can be resolved, the Chinese included can be resolved date; If you view the source code can be found, fastjson did not directly use DateFormat do date format conversion, but realized ISO 8601 standard , It also provides support for common date formats in China; you can directly view scanISO8601DateIfMatch method in the JSONScanner
Another point to note is that the above GSON uses the 2.2.2 version. The latest version 2.8.6 also provides support for the ISO 8601 standard. For details, you can view the ISO8601Utils class.

Thanks for attention

You can follow the WeChat public "1609f869ab4774 roll back code ", read the first time, the article is continuously updated; focus on Java source code, architecture, algorithm and interview.