How to track the request response of the Spring MVC interface

Certain business requirements need to track our interface access, that is, to record the request and response. The basic record dimensions include request input parameters (path query parameters, request body), request path (uri), request method (method), request headers (headers), response status, response headers, and even sensitive response bodies, etc. Wait. Today I have summarized several methods, you can choose according to your needs.

Implementation of request tracking

Gateway layer

Many gateway facilities have the function of httptrace , which can help us to centrally record the request traffic. Orange, Kong, Apache Apisix of which have the ability gateway based on Nginx, even Nginx itself also provides a record httptrace ability to log.

advantage is centralized log management httptrace, development-free; drawback is technically demanding, it requires supporting the distribution, storage, query facilities.

Spring Boot Actuator

In Spring Boot , a simple tracking function is actually provided. You only need to integrate:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Turn on /actuator/httptrace :

management:
  endpoints:
    web:
      exposure:
        include: 'httptrace'

You can get the latest Http request information http://server:port/actuator/httptrace

However, in the latest version, you may need to explicitly declare the storage method of these tracking information, that is, implement the HttpTraceRepository interface and inject Spring IoC .

For example, put it in the memory and limit to the most recent 100 ( not recommended for production use ):

@Bean
public HttpTraceRepository httpTraceRepository(){
    return new InMemoryHttpTraceRepository();
}

The trace log is json format:

There are not many dimensions recorded, of course, you can try if you have enough.

advantage that integrate simple, almost exempted development; drawback few records that dimension, but also need to build these facilities consume buffer log information.

CommonsRequestLoggingFilter

Spring Web module also provides a filter CommonsRequestLoggingFilter , which can log the details of the request. The configuration is also relatively simple:

@Bean
CommonsRequestLoggingFilter  loggingFilter(){
    CommonsRequestLoggingFilter loggingFilter = new CommonsRequestLoggingFilter();
    // 记录 客户端 IP信息
    loggingFilter.setIncludeClientInfo(true);
    // 记录请求头
    loggingFilter.setIncludeHeaders(true);
    // 如果记录请求头的话，可以指定哪些记录，哪些不记录
    // loggingFilter.setHeaderPredicate();
    // 记录 请求体  特别是POST请求的body参数
    loggingFilter.setIncludePayload(true);
    // 请求体的大小限制 默认50
    loggingFilter.setMaxPayloadLength(10000);
    //记录请求路径中的query参数 
    loggingFilter.setIncludeQueryString(true);
    return loggingFilter;
}

And it must be open to CommonsRequestLoggingFilter of Debug log:

logging:
  level:
    org:
      springframework:
        web:
          filter:
            CommonsRequestLoggingFilter: debug

A request will output two logs, once before passing the filter for the first time; once after completing the filter chain.

One more thing here can actually be transformed into output json format.

advantage is flexible configuration, but also to track the overall dimensions of the request, drawback only record request without recording a response.

ResponseBodyAdvice

The Spring Boot unified return body can actually be recorded, and it needs to be implemented by itself. Here we use the CommonsRequestLoggingFilter parse the request. The response body can also be obtained, but the response header and status are not clear because of the life cycle, and it is not clear whether it is appropriate to obtain it here, but this is an idea.

/**
 * @author felord.cn
 * @since 1.0.8.RELEASE
 */
@Slf4j
@RestControllerAdvice(basePackages = {"cn.felord.logging"})
public class RestBodyAdvice implements ResponseBodyAdvice<Object> {
    private static final int DEFAULT_MAX_PAYLOAD_LENGTH = 10000;
    public static final String REQUEST_MESSAGE_PREFIX = "Request [";
    public static final String REQUEST_MESSAGE_SUFFIX = "]";
    private ObjectMapper objectMapper = new ObjectMapper();

    @Override
    public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
        return true;
    }

    @SneakyThrows
    @Override
    public Object beforeBodyWrite(Object body,
                                  MethodParameter returnType,
                                  MediaType selectedContentType,
                                  Class<? extends HttpMessageConverter<?>> selectedConverterType,
                                  ServerHttpRequest request,
                                  ServerHttpResponse response) {

        ServletServerHttpRequest servletServerHttpRequest = (ServletServerHttpRequest) request;

        log.debug(createRequestMessage(servletServerHttpRequest.getServletRequest(), REQUEST_MESSAGE_PREFIX, REQUEST_MESSAGE_SUFFIX));
        Rest<Object> objectRest;
        if (body == null) {
            objectRest = RestBody.okData(Collections.emptyMap());
        } else if (Rest.class.isAssignableFrom(body.getClass())) {
            objectRest = (Rest<Object>) body;
        }
        else if (checkPrimitive(body)) {
            return RestBody.okData(Collections.singletonMap("result", body));
        }else {
            objectRest = RestBody.okData(body);
        }
        log.debug("Response Body ["+ objectMapper.writeValueAsString(objectRest) +"]");
        return objectRest;
    }


    private boolean checkPrimitive(Object body) {
        Class<?> clazz = body.getClass();
        return clazz.isPrimitive()
                || clazz.isArray()
                || Collection.class.isAssignableFrom(clazz)
                || body instanceof Number
                || body instanceof Boolean
                || body instanceof Character
                || body instanceof String;
    }


    protected String createRequestMessage(HttpServletRequest request, String prefix, String suffix) {
        StringBuilder msg = new StringBuilder();
        msg.append(prefix);
        msg.append(request.getMethod()).append(" ");
        msg.append(request.getRequestURI());


        String queryString = request.getQueryString();
        if (queryString != null) {
            msg.append('?').append(queryString);
        }


        String client = request.getRemoteAddr();
        if (StringUtils.hasLength(client)) {
            msg.append(", client=").append(client);
        }
        HttpSession session = request.getSession(false);
        if (session != null) {
            msg.append(", session=").append(session.getId());
        }
        String user = request.getRemoteUser();
        if (user != null) {
            msg.append(", user=").append(user);
        }

        HttpHeaders headers = new ServletServerHttpRequest(request).getHeaders();
        msg.append(", headers=").append(headers);

        String payload = getMessagePayload(request);
        if (payload != null) {
            msg.append(", payload=").append(payload);
        }

        msg.append(suffix);
        return msg.toString();
    }

    protected String getMessagePayload(HttpServletRequest request) {
        ContentCachingRequestWrapper wrapper =
                WebUtils.getNativeRequest(request, ContentCachingRequestWrapper.class);
        if (wrapper != null) {
            byte[] buf = wrapper.getContentAsByteArray();
            if (buf.length > 0) {
                int length = Math.min(buf.length, DEFAULT_MAX_PAYLOAD_LENGTH);
                try {
                    return new String(buf, 0, length, wrapper.getCharacterEncoding());
                } catch (UnsupportedEncodingException ex) {
                    return "[unknown]";
                }
            }
        }
        return null;
    }
}

Do not forget to configure ResponseBodyAdvice the logging level DEBUG .

logstash-logback-encoder

This is the logback encoder of logstash, which can output httptrace as json in a structured manner. Introduce:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>6.6</version>
</dependency>

ConsoleAppender to LogstashEncoder in the logback configuration:

<configuration>
    <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
    </appender>
    <root level=" INFO">
        <appender-ref ref="jsonConsoleAppender"/>
    </root>
</configuration>

Then implement a parsed Filter :

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.UUID;

/**
 * @author felord.cn
 * @since 1.0.8.RELEASE
 */
@Order(1)
@Component
public class MDCFilter implements Filter {

    private final Logger LOGGER = LoggerFactory.getLogger(MDCFilter.class);
    private final String X_REQUEST_ID = "X-Request-ID";

    @Override
    public void doFilter(ServletRequest request,
                         ServletResponse response,
                         FilterChain chain) throws IOException, ServletException {
        HttpServletRequest req = (HttpServletRequest) request;
        HttpServletResponse res = (HttpServletResponse) response;
        try {
            addXRequestId(req);
            LOGGER.info("path: {}, method: {}, query {}",
                    req.getRequestURI(), req.getMethod(), req.getQueryString());
            res.setHeader(X_REQUEST_ID, MDC.get(X_REQUEST_ID));
            chain.doFilter(request, response);
        } finally {
            LOGGER.info("statusCode {}, path: {}, method: {}, query {}",
                    res.getStatus(), req.getRequestURI(), req.getMethod(), req.getQueryString());
            MDC.clear();
        }
    }

    private void addXRequestId(HttpServletRequest request) {
        String xRequestId = request.getHeader(X_REQUEST_ID);
        if (xRequestId == null) {
            MDC.put(X_REQUEST_ID, UUID.randomUUID().toString());
        } else {
            MDC.put(X_REQUEST_ID, xRequestId);
        }
    }

}

The analysis method here can actually be more refined.

Not only can the interface request log be recorded, but it can also be structured as json:

{"@timestamp":"2021-08-10T23:48:51.322+08:00","@version":"1","message":"statusCode 200, path: /log/get, method: GET, query foo=xxx&bar=ooo","logger_name":"cn.felord.logging.MDCFilter","thread_name":"http-nio-8080-exec-1","level":"INFO","level_value":20000,"X-Request-ID":"7c0db56c-b1f2-4d85-ad9a-7ead67660f96"}

Summarize

Today, I introduced a lot of methods for recording and tracking interface request responses. They are relatively simple. If your project becomes bigger, you may need to use link tracking. You can make up this pit if you have the opportunity in the future. Of course you may have a better way, please leave a message to share.

Follow the public account: Felordcn for more information

personal blog: https://felord.cn