Java Hot Update Groovy Practice and Stepping Guide

What is Groovy?

Apache's Groovy is an object-oriented programming language designed on the Java platform. This dynamic language has features similar to those in Python, Ruby, and Smalltalk, and can be used as a scripting language for the Java platform. Code and library to interoperate.

Due to its nature of running on the JVM, Groovy can use libraries written in other Java languages. Groovy's syntax is very similar to Java, and most Java code also conforms to Groovy's syntax rules, albeit possibly with different semantics. Groovy 1.0 was released on January 2, 2007, and Groovy 2.0 was released in July 2012. Since version 2, Groovy can also be statically compiled, providing type inference and Java-like performance. Groovy 2.4 was the last major release sponsored by Pivotal Software as of March 2015. Groovy has changed its governance structure to the Apache Software Foundation's Project Management Committee (PMC) [1].

Why does Java need Groovy?

Groovy features are as follows:

• Syntactically supports dynamic types, closures and other new-generation language features
• Seamless integration with all existing Java class libraries
• Supports both object-oriented programming and procedural programming
• The execution method can compile the source file written by groovy into a class bytecode file, and then hand it over to the JVM for execution, or directly interpret and execute the groovy source file.
• Groovy can be perfectly integrated with Java, and can use all Java libraries

The advantages of Groovy are as follows:

• agile

  • Groovy adds a lot of syntactic sugar to the grammar. Many Java strict writing grammars can be implemented in Groovy with only a small amount of syntactic sugar.
• The flexibility of Groovy is that it can be used as both a programming language and a scripting language
• Learn Groovy at 0 cost, perfectly adapted to Java syntax

Design and Implementation of Hot Deployment Technology

scenes to be used

I will introduce the following common scenarios suitable for hot update of Groovy scripts for you to learn

Risk Control Security - Rule Engine

The rule engine of risk control is very suitable for groovy. To fight against black production, the strategists will produce interception rules every day. If the version needs to be released every time, after the observation is completed, the wool of the slaughtered wool will be smashed by the black production. Gone.

Therefore, the dynamic analysis and execution of the groovy script engine is used, and the rule script is used to abstract the inspection and interception rules, which can be deployed quickly and improve efficiency.

monitoring Center

In a large-scale Internet system, with the entry of massive data, personnel at all levels need to pay attention to the indicators of various dimensions of the business at all times. At this time, an abnormal indicator cannot be achieved by human flesh alone. At this time, the monitoring center needs to intervene and deploy the change rules in advance. When an abnormality occurs, the monitoring center sends an alarm notification to the corresponding rule creator, so as to find out the cause as soon as possible and recover the capital loss.

At this time, it is necessary to ensure that the monitoring center is extremely flexible, and can meet the needs of business personnel or R&D personnel to configure monitoring indicators anytime, anywhere. For testing, we can use Groovy conditional expressions to meet the needs of flexible monitoring rule configuration.

event marketing

Campaign configuration is one of the most complex businesses in my opinion. There are various activity templates, thousands of people and thousands of faces, and different groups of people see different activity styles or "prizes". And the activity should be launched quickly, the effect recovery, the input-output ratio, etc. should be able to be observed immediately.

At this time, it is necessary to abstract the entire activity template from the engineering side, and embed the Groovy script in the places that need to be changed, which reduces the time for testing and release, and enables the activity to be configured online.

Technical realization

Script loading/updating

Code implementation display:

 /**
 * 加载脚本
 * @param script
 * @return
 */
public static GroovyObject buildScript(String script) {
    if (StringUtils.isEmpty(script)) {
        throw new RuntimeException("script is empty");
    }

    String cacheKey = DigestUtils.md5DigestAsHex(script.getBytes());
    if (groovyObjectCache.containsKey(cacheKey)) {
        log.debug("groovyObjectCache hit");
        return groovyObjectCache.get(cacheKey);
    }

    GroovyClassLoader classLoader = new GroovyClassLoader();
    try {
        Class<?> groovyClass = classLoader.parseClass(script);
        GroovyObject groovyObject = (GroovyObject) groovyClass.newInstance();
        classLoader.clearCache();

        groovyObjectCache.put(cacheKey, groovyObject);
        log.info("groovy buildScript success: {}", groovyObject);
        return groovyObject;
    } catch (Exception e) {
        throw new RuntimeException("buildScript error", e);
    } finally {
        try {
            classLoader.close();
        } catch (IOException e) {
            log.error("close GroovyClassLoader error", e);
        }
    }
}

Focus:

• Scripts enable cache processing: otherwise multiple updates may cause Metaspace OutOfMemery

  script execution

 // 程序内部需要关联出待执行的脚本即可
try {
    Map<String, Object> singleMap = GroovyUtils.invokeMethod2Map(s.getScriptObject(), s.getInvokeMethod(), params);
    data.putAll(singleMap);
} catch (Throwable e) {
    log.error(String.format("RcpEventMsgCleanScriptGroovyHandle groovy error, guid: %d eventCode: %s",
            s.getGuid(), s.getEventCode()), e);
}

// 三种执行方式，看 脚本内部返回的结果是什么
public static Map<String, Object> invokeMethod2Map(GroovyObject scriptObject, String invokeMethod, Object[] params) {
    return (Map<String, Object>) scriptObject.invokeMethod(invokeMethod, params);
}

public static boolean invokeMethod2Boolean(GroovyObject scriptObject, String invokeMethod, Object[] params) {
    return (Boolean) scriptObject.invokeMethod(invokeMethod, params);
}

public static String invokeMethod2String(GroovyObject scriptObject, String invokeMethod, Object[] params) {
    log.debug("GroovyObject class: {}", scriptObject.getClass().getSimpleName());
    return (String) scriptObject.invokeMethod(invokeMethod, params);
}

Production Stepping Guide

Java8 lambda and Groovy syntax issue

It is said that Groovy is perfectly compatible with Java syntax, that is, directly copying the Java code into the Groovy file can also compile successfully.
Is this really the case, let's look at the code executed as follows:

 Set<String> demo = new HashSet<>();
demo.add("111");
demo.add("222");

for (String s : demo) {
    executor.submit({ -> 
        println "submit: " + s;                 
    });
}

for (String s in demo) {
    executor.submit({ -> 
        println "sp submit: " + s;                 
    });
}


// 输出结果
// submit: 222
// sp submit: 222
// submit: 222
// sp submit: 222

At this time, the code does not output 111, 222 as expected, why is this?

Answer: The semantics of lambda syntax in Groovy are inconsistent with those in Java. Although the compilation is not wrong, the semantics expressed are inconsistent in Groovy, which means the concept of closure. Those who are not familiar here can Google to learn more about Groovy syntax.

GroovyClassLoader loading mechanism causes frequent gc problems

Usually the Groovy class code is loaded as follows:

 GroovyClassLoader groovyLoader = new GroovyClassLoader();
Class<Script> groovyClass = (Class<Script>) groovyLoader.parseClass(groovyScript);
Script groovyScript = groovyClass.newInstance();

Every time groovyLoader.parseClass(groovyScript) is executed, in order to ensure that each execution is a new script content, Groovy will generate a class file with a new name each time, which has been explained in the previous section. When this method is executed every time for the same script, the phenomenon will be that more and more Classes will be loaded, which will cause PermGen to be full.

At the same time, there is also a performance bottleneck problem. If you analyze this code, you will find that 90% of the time is occupied by Class.

In the actual combat process as above, solutions have been given:

• Cache the Class object generated after parseClass, the key is the md5 value of the groovyScript script

The script takes a long time to execute for the first time

When the initial plan was launched, the stress test showed that the performance of the script loaded for the first time was slow, and the execution speed of the subsequent script was very fast. I guess it may be that Groovy did other checks when the script was first loaded (I haven't followed up on this yet). block, if any readers are interested, you can break the point to see where the link takes in detail)

For the slow loading problem for the first time, the solution is as follows:

 // 1.加载脚本，并缓存
GroovyObject object = loadClass(classSeq);
cacheMap.put(md5(classSeq), object);

// 2.预热
// 模拟方法调用
cacheMap.get(md5(classSeq)).invoke();

// 3.开放给线上流量使用

Wonderful past

• Performance tuning - a small log is a big pit
• Performance Optimization Essentials - Flame Graph
• Flink implements real-time features in risk control scenarios

Welcome to the official account: Gugu Chicken Technical Column Personal technical blog: https://jifuwei.github.io/

refer to:
[1] Groovy Wiki