LOG4J2 configuration: detailed getting started guide

LOG4J2 configuration: detailed getting started guide

Eric Dietrich

September 5, 2018

One of the most annoying aspects of software development is definitely logging. If a non-trivial application lacks logging, then those who maintain it will encounter difficulties, and post-mortem debugging will be mainly a guessing game. Today, we have a log4j2 configuration tutorial to help solve this situation.

There are many ways to log a Java program. You can use the manual method, but the recommended method is to use a dedicated logging framework. This is why we want to introduce Apache log4j2, which is an improved version of log4j and one of the industry standard logging frameworks.

We will first quickly review our previous article on Java logging and introduce some things about log4j2. Then we write a sample application to demonstrate the use of log4j2.

Next, we enter the focus of the article. You will learn how to configure log4j2, starting from the basics, and gradually learning more advanced topics such as log format, additional programs, log levels, and log hierarchy.

let's start.

Logging with Log4j2: not the first time

In this tutorial, we used log4j version 2, which is a logging framework from the Apache project.

Let's learn more about the Java application log and view the log4j2 configuration. Today we will introduce the basic aspects of log4j2 configuration to help you get started.

The features of Log4j make it one of the most popular logging frameworks in Java. It can be configured for multiple logging destinations and various log file formats.

Log messages can be filtered and directed at the individual class level, allowing developers and operations personnel to fine-tune application messages.

Let's check these mechanisms by configuring log4j using a command-line Java application.

Sample application

Let's use log4j for logging applications.

package com.company;

import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;

public class Main {

  private static final Logger logger = LogManager.getLogger(Main.class);

  public static void main(String[] args) {
      String message = "Hello there!";
      logger.trace(message);
      logger.debug(message);
      logger.info(message);
      logger.warn(message);
      logger.error(message);
      logger.fatal(message);
  }
}

We log the same messages at each log4j predefined log level: trace, debug, information, warning, error, and fatal.

We will use log4j's YAML file format, so you need to add some additional dependencies to pom.xml (or build.gradle).

<dependencies>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.12.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.12.1</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.dataformat</groupId>
        <artifactId>jackson-dataformat-yaml</artifactId>
        <version>2.10.0</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.10.0</version>
    </dependency>
</dependencies>

Set up this code so that you can build and run it with your favorite Java tool.

Basic Log4j2 configuration

default allocation

Let's run our application without log4j configuration file. If you already have one, delete it or move it to another file name so that log4j will ignore it.

When we run the application, we will see on the console:

09:38:14.114 [main] ERROR com.company.Main - Hello there!
09:38:14.119 [main] FATAL com.company.Main - Hello there!

Two of the six log messages, the messages designated as "error" and "fatal" are sent to the console.

Log4j has a default configuration. It will log to the console and display messages categorized as "error" or higher.

It is useful to understand how log4j runs without a configuration file, but let's see how to set it up according to our needs.

Configuration file location

We can provide a configuration file for log4j in a specific location through the log4j.configurationFile system property. This is the first place it will look for configuration files.

If log4j cannot find the system properties, it will look for the file in the classpath. Since log4j version 2 supports four different file formats and two different file naming conventions, the rules for locating files are complicated. After we introduce the different options, we will discuss them.

Configuration file format

Log4j will load Java properties and YAML, JSON and XML configuration files. It recognizes the file format by checking the file extension.

• Java properties — .properties
• YAML — .yaml or .yml
• JSON — .json or .jsn
• XML — .xml

log4j.configurationFile system attribute must have one of these file extensions, but can have any base name. Log4j will parse it according to the format indicated by the extension.

When log4j scans the classpath of a file, it scans each format in the order listed above and stops when it finds a match.

For example, if it finds a YAML configuration, it will stop searching and load it. If there is no YAML file but it finds JSON, it will stop searching and use it.

Configuration file name

When log4j scans the classpath, it looks for one of two file names: log4j2- test .[extension] or log4j2.[extension].

It first loads the test file and provides a convenient mechanism for developers to force the application to record at the debug or trace level without changing the standard configuration.

Scan configuration

When we put together the rules of file format and name, we can see log4j's self-configuration algorithm.

If any of the following steps are successful, log4j will stop and load the generated configuration file.

1. Check the log4j.configurationFile system property and load the specified file when found.
2. Search log4j2-test.properties in the classpath.
3. Scan the classpath of log4j2-test.yaml or log4j2-test.yml
4. Check log4j2-test.json or log4j2-test.jsn
5. Search log4j2-test.xml
6. Look for log4j2.properties
7. Search log4j2.yaml or log4j2.yml
8. Scan the classpath of log4j2.json or log4j2.jsn
9. Check log4j2.xml
10. Use the default configuration.

Practice correct profile hygiene

There are 12 potential configuration file names for log4j. If the application logs unnecessary messages in the production environment, loading the wrong information may cause log information loss or performance degradation.

Before deploying the code, make sure that your application has only one configuration file and you know where it is. If you insist on loading configuration from the classpath, scan for fake files before posting the code.

basic configuration

Now that we know how to provide configuration for log4j, let's create one and use it to customize our application.

Revisit the default configuration of Log4j

Let's start with the default configuration and then modify the behavior of our application from there. We will get prompts from log4j configuration rules and use YAML.

The default configuration is as follows:

Configuration:
  status: warn
  Appenders:
    Console:
      name: Console
      target: SYSTEM_OUT
      PatternLayout:
        Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
  Loggers:
    Root:
      level: error
      AppenderRef:
        ref: Console

Use these contents to create a file name log4j2.yaml and set log4j.configurationFile to point to its location.

Next, run the application. You will see the same output as before.

09:38:14.114 [main] ERROR com.company.Main - Hello there!
09:38:14.119 [main] FATAL com.company.Main - Hello there!

We have controlled the logging configuration of the application. Let us improve it now.

Log file location

The first step is to take our log out of the console and put it in a file. For this, we need to understand appender.

Appenders put log messages where they belong. The default configuration provides a console add-on program. As the name suggests, it appends the message to the console.

Appenders:
  Console:
    name: Console
    target: SYSTEM_OUT
    PatternLayout:
      Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

We want a file attachment program. Let's replace our console appender.

Appenders:
  File:
    name: File_Appender
    fileName: logfile.log
    PatternLayout:
      Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

The file appender has a name, just like the console appender. But they have a fileName instead of a target.

Similar to the console appender, they also have a PatternLayout, which we will introduce below.

The name is not just for display. If we want to replace the console appender with a file appender, we need to let our logger know where to put our log messages.

Therefore, change the ref value in the recorder to the name of the file attachment program.

Loggers:
  Root:
    level: error
    AppenderRef:
      ref: File_Appender

Now, re-run the application. It does not log to the console, but puts the message in a file named logfile.log in the working directory. We have moved the log to the file!

Log level

In our log file, we still only saw two of the six log messages. Let's talk about loggers and how they manage log messages.

Our basic configuration defines a recorder.

Loggers:
  Root:
    level: error
    AppenderRef:
      ref: File_Appender

It has an "error" level, so it only prints errors or fatal messages.

When the logger receives a log message, it will pass or filter it according to its configured level. This table shows the relationship between logger configuration and log message level.

Therefore, if we change the level of the logger, we will see more messages. Set it to "Debug".

Loggers:
  Root:
    level: debug
    AppenderRef:
      ref: File_Appender

Next, re-run the program. The application logs all messages of debug level or higher.

Logger hierarchy
Log4j arranges loggers in a hierarchical structure. This makes it possible to specify different configurations for each class.

Let's change our application and see its actual effect.

public class Main {

  private static final Logger logger = LogManager.getLogger(Main.class);

  public static void main(String[] args) {
    String message = "Hello there!";
    System.out.println(message);
    logger.debug(message);
    logger.info(message);
    logger.error(message);

    LoggerChild.log();
  }

  private static class LoggerChild {
    private static final Logger childLogger = LogManager.getLogger(LoggerChild.class);

    static void log() {
        childLogger.debug("Hi Mom!");
    }
  }
}

We added an inner class to create a logger and use it to log a message.

After that, it mainly records, it calls LoggerChild.

If we run it with the current configuration, we will see the new message, and it is logged from a different class.

12:29:23.325 [main] DEBUG com.company.Main - Hello there!
12:29:23.331 [main] INFO  com.company.Main - Hello there!
12:29:23.332 [main] ERROR com.company.Main - Hello there!
12:29:23.333 [main] DEBUG com.company.Main.LoggerChild - Hi Mom!

The logger has a class hierarchy similar to Java. All loggers are descendants of the root logger we have been using so far.

Loggers that lack any specific configuration inherit the root configuration.

So when Main and LoggerChild use their class names to create loggers, these loggers inherit the configuration of Root, that is, send debug-level and higher-level messages to File_Appender.

We can override the specified configuration of these two loggers.

Loggers:
  logger:
    -
      name: com.company.Main
      level: error
      additivity: false
      AppenderRef:
        ref: File_Appender
    -
      name: com.company.Main.LoggerChild
      level: debug
      additivity: false
      AppenderRef:
        ref: File_Appender
  Root:
    level: debug
    AppenderRef:
      ref: File_Appender

The recorder is named in the recorder section. Since we listed two, we use YAML array syntax.

We set the record of com.company.Main to "error" and the record of com.company.Main.LoggerChild to "debug".

This add setting controls whether log4j will send messages from the ancestor of the recorder to the descendants.

If set to true, both loggers will process the same message. Some systems want to add the same message to two different logs. We don't want this behavior, so we override the default value and specify false.

Now run the program again:

12:33:11.062 [main] ERROR com.company.Main - Hello there!
12:33:11.073 [main] DEBUG com.company.Main.LoggerChild - Hi Mom!

We only see error messages from Main, but we still see debug messages from LoggerChild!

More than one Appender

Just as we can have multiple loggers, we can also have multiple appenders.

Let's make some changes to our configuration.

Add a second file attachment program. To do this, create a list with the original appender, and create a second list with a different name and file. Your Appenders section should look like this:

  Appenders:
    File:
      -
        name: File_Appender
        fileName: logfile.log
        PatternLayout:
          Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
      -
        name: Child_Appender
        fileName: childlogfile.log
        PatternLayout:
          Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

Next, point the LoggerChild logger to the new appender. Your logger section will look like the following.

Loggers:
   logger:
     -
      name: com.company.Main
      level: error
      additivity: false
      AppenderRef:
        ref: File_Appender
     -
      name: com.company.Main.LoggerChild
      level: debug
      additivity: false
      AppenderRef:
        ref: Child_Appender
   Root:
     level: debug
     AppenderRef:
      ref: File_Appender

Now run the application, and you will see two different log files, each containing messages from its associated class.

Log message format
Each of our appenders has a PatternLayout.

PatternLayout:
Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

PatternLayout is an instance of the Log4j layout class. Log4j has built-in layouts for logging messages in CSV, JSON, Syslog, and various different formats.

PatternLayout has a set of operators for formatting messages, and its operation is similar to C's sprintf function. By specifying the mode, we can control the format of the log message written by the appender.

Our layout string looks like this:

"%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

Each% corresponds to a field in the log message.

PatternLayout has many additional operators.

Variable substitution

As appenders and loggers increase, configuration files may become duplicated. Log4j supports variable substitution to help reduce duplication and make it easier to maintain. Let's use Properties to optimize our configuration.

Configuration:
  status: warn
  Properties:
    property:
      -
        name: "LogDir"
        value: "logs"
      -
        name: "DefaultPattern"
        value: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
  Appenders:
    File:
      -
        name: File_Appender
        fileName: ${LogDir}/logfile.log
        PatternLayout:
          Pattern: ${DefaultPattern}
      -
        name: Child_Appender
        fileName: ${LogDir}/childlogfile.log
        PatternLayout:
          Pattern: ${DefaultPattern}
  Loggers:
    logger:
      -
        name: com.company.Main
        level: error
        additivity: false
        AppenderRef:
          ref: File_Appender
     -
        name: com.company.Main.LoggerChild
        level: debug
        additivity: false
        AppenderRef:
          ref: Child_Appender
    Root:
      level: debug
      AppenderRef:
        ref: File_Appender

At the top of the file, we declared two properties, one named LogDir and the other named DefaultPattern.

After the attribute is declared, you can use braces and dollar signs to use it in the configuration: ${LogDir} or ${DefaultPattern}

LogDir is the name of the subdirectory we added to the names of the two log files. When we run the application, log4j will create this directory and place the log files there.

We specified DefaultPattern as the pattern layout for our two log files and moved the definition to one place. If we want to modify our log file format, we now only need to worry about changing it once.

Log4j can also import properties from the environment. You can find details here.

For example, if we want to import the log file directory from the Java system property, we specify it as ${sys: LogDir} in the log4j configuration and set the LogDir system property to the desired directory.

Automatic reconfiguration

Log4j can periodically reload its configuration, allowing us to change the application's logging configuration without restarting the application.

Add the monitorInterval setting to the Configuration section of the file, and log4j will scan the file at the specified time interval.

Configuration:
  monitorInterval: 30

The interval is specified in seconds.

in conclusion

Log4j is a powerful logging framework that allows us to configure the application to log in in a variety of different ways and fine-tune how different components use log files.

This tutorial covers the basic aspects of configuring log4j, but there is still a lot to learn. You can learn about log4j configuration on the project website.

This article was written by Eric Goebelbecker. Eric has worked in the financial markets of New York City for 25 years, developing infrastructure for the market data and financial information exchange (FIX) protocol network. He likes to talk about what makes the team effective (or not so effective!)