The mystery of the mysterious disappearance of inputStream in springboot

Preface

Recently, Xiao Ming took over the code of his former colleague, and unexpectedly and logically encountered a pit.

In order to avoid falling into the same pit twice, Xiao Ming decided to write down the pit and set up a big sign in front of the pit to prevent other friends from falling into it.

HTTPClient mock call

In order to explain this problem, we first start with the simplest http call.

Set body

Server

The server code is as follows:

@Controller
@RequestMapping("/")
public class ReqController {

    @PostMapping(value = "/body")
    @ResponseBody
    public String body(HttpServletRequest httpServletRequest) {
        try {
            String body = StreamUtil.toString(httpServletRequest.getInputStream());
            System.out.println("请求的 body: " + body);

            // 从参数中获取
            return body;
        } catch (IOException e) {
            e.printStackTrace();
            return e.getMessage();
        }
    }

}

How does the java client request so that the server can read the passed body?

Client

This problem must not be difficult for you, there are many ways to achieve it.

Let's take apache httpclient as an example:

//post请求，带集合参数
public static String post(String url, String body) {
    try {
        // 通过HttpPost来发送post请求
        HttpPost httpPost = new HttpPost(url);
        StringEntity stringEntity = new StringEntity(body);
        // 通过setEntity 将我们的entity对象传递过去
        httpPost.setEntity(stringEntity);
        return execute(httpPost);
    } catch (UnsupportedEncodingException e) {
        throw new RuntimeException(e);
    }
}

//执行请求返回响应数据
private static String execute(HttpRequestBase http) {
    try {
        CloseableHttpClient client = HttpClients.createDefault();
        // 通过client调用execute方法
        CloseableHttpResponse Response = client.execute(http);
        //获取响应数据
        HttpEntity entity = Response.getEntity();
        //将数据转换成字符串
        String str = EntityUtils.toString(entity, "UTF-8");
        //关闭
        Response.close();
        return str;
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

It can be found that httpclient is very convenient after encapsulation.

We can set setEntity to the StringEntity corresponding to the input parameter.

test

In order to verify the correctness, Xiao Ming implemented a verification method locally.

@Test
public void bodyTest() {
    String url = "http://localhost:8080/body";
    String body = buildBody();
    String result = HttpClientUtils.post(url, body);

    Assert.assertEquals("body", result);
}

private String buildBody() {
    return "body";
}

Very relaxed, Xiao Ming leaked the smile of the Dragon King.

Set parameter

Server

Xiao Ming saw that there is a server-side code implementation as follows:

@PostMapping(value = "/param")
@ResponseBody
public String param(HttpServletRequest httpServletRequest) {
    // 从参数中获取
    String param = httpServletRequest.getParameter("id");
    System.out.println("param: " + param);
    return param;
}

private Map<String,String> buildParamMap() {
    Map<String,String> map = new HashMap<>();
    map.put("id", "123456");

    return map;
}

All parameters are obtained through the getParameter method, how should it be implemented?

Client

This is not difficult, Xiaoming thought.

Because a lot of code was implemented in this way before, so ctrl+CV got the following code:

//post请求，带集合参数
public static String post(String url, Map<String, String> paramMap) {
    List<NameValuePair> nameValuePairs = new ArrayList<>();
    for (Map.Entry<String, String> entry : paramMap.entrySet()) {
        NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry.getValue());
        nameValuePairs.add(pair);
    }
    return post(url, nameValuePairs);
}

//post请求，带集合参数
private static String post(String url, List<NameValuePair> list) {
    try {
        // 通过HttpPost来发送post请求
        HttpPost httpPost = new HttpPost(url);
        // 我们发现Entity是一个接口，所以只能找实现类，发现实现类又需要一个集合，集合的泛型是NameValuePair类型
        UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(list);
        // 通过setEntity 将我们的entity对象传递过去
        httpPost.setEntity(formEntity);
        return execute(httpPost);
    } catch (Exception exception) {
        throw new RuntimeException(exception);
    }
}

This is the most commonly used paramMap, which is easy to construct; it is separated from the specific implementation method, and it is also easy to expand later.

servlet standard

UrlEncodedFormEntity looks unremarkable, indicating that this is a post form request.

It also involves a standard of servlet 3.1. The following standards must be met before the parameter collection of the post form is available.

1. 请求是 http 或 https

2. 请求的方法是 POST

3. content type 为： application/x-www-form-urlencoded

4. servlet 已经在 request 对象上调用了相关的 getParameter 方法。

When the above conditions are not met, the data of the POST form will not be set in the parameter collection, but it can still be obtained through the inputstream of the request object.

When the above conditions are met, the data of the POST form will no longer be available in the inputstream of the request object.

This is a very important agreement, which has caused many small partners to become trapped.

test

Therefore, Xiao Ming also wrote the corresponding test case:

@Test
public void paramTest() {
    String url = "http://localhost:8080/param";

    Map<String,String> map = buildParamMap();
    String result = HttpClientUtils.post(url, map);

    Assert.assertEquals("123456", result);
}

It would be great if falling in love could be like programming.

Xiao Ming thought about it, but he couldn't help but frowned, and realized that things were not simple.

Set parameter and body

Server

One request has a relatively large input parameter, so it is placed in the body, and other parameters are still placed in the paramter.

@PostMapping(value = "/paramAndBody")
@ResponseBody
public String paramAndBody(HttpServletRequest httpServletRequest) {
    try {
        // 从参数中获取
        String param = httpServletRequest.getParameter("id");
        System.out.println("param: " + param);
        String body = StreamUtil.toString(httpServletRequest.getInputStream());
        System.out.println("请求的 body: " + body);
        // 从参数中获取
        return param+"-"+body;
    } catch (IOException e) {
        e.printStackTrace();
        return e.getMessage();
    }
}

Among them, StreamUtil#toString is a tool class for simple processing of streams.

/**
 * 转换为字符串
 * @param inputStream 流
 * @return 结果
 * @since 1.0.0
 */
public static String toString(final InputStream inputStream)  {
    if (inputStream == null) {
        return null;
    }
    try {
        int length = inputStream.available();
        final Reader reader = new InputStreamReader(inputStream, StandardCharsets.UTF_8);
        final CharArrayBuffer buffer = new CharArrayBuffer(length);
        final char[] tmp = new char[1024];
        int l;
        while((l = reader.read(tmp)) != -1) {
            buffer.append(tmp, 0, l);
        }
        return buffer.toString();
    } catch (Exception exception) {
        throw new RuntimeException(exception);
    }
}

Client

So the question is, how to set parameter and body in HttpClient at the same time?

The witty friends can try it for themselves.

Xiao Ming tried a variety of methods and found a cruel reality-httpPost can only set one Entity, and also tried various sub-categories, but LUAN.

Just when Xiao Ming wanted to give up, Xiao Ming suddenly thought that paramter can be achieved by splicing URLs.

means that we set the parameter and url as a new URL, and the body is set in the same way as before.

The implementation code is as follows:

//post请求，带集合参数
public static String post(String url, Map<String, String> paramMap,
                          String body) {
    try {
        List<NameValuePair> nameValuePairs = new ArrayList<>();
        for (Map.Entry<String, String> entry : paramMap.entrySet()) {
            NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry.getValue());
            nameValuePairs.add(pair);
        }

        // 构建 url
        //构造请求路径，并添加参数
        URI uri = new URIBuilder(url).addParameters(nameValuePairs).build();

        //构造HttpClient
        CloseableHttpClient httpClient = HttpClients.createDefault();
        // 通过HttpPost来发送post请求
        HttpPost httpPost = new HttpPost(uri);
        httpPost.setEntity(new StringEntity(body));

        // 获取响应
        // 通过client调用execute方法
        CloseableHttpResponse Response = httpClient.execute(httpPost);
        //获取响应数据
        HttpEntity entity = Response.getEntity();
        //将数据转换成字符串
        String str = EntityUtils.toString(entity, "UTF-8");
        //关闭
        Response.close();
        return str;
    } catch (URISyntaxException | IOException | ParseException e) {
        throw new RuntimeException(e);
    }
}

Here we use new URIBuilder(url).addParameters(nameValuePairs).build() construct a new URL. Of course, you can use &key=value to splice it yourself.

Test code

@Test
public void paramAndBodyTest() {
    String url = "http://localhost:8080/paramAndBody";
    Map<String,String> map = buildParamMap();
    String body = buildBody();
    String result = HttpClientUtils.post(url, map, body);

    Assert.assertEquals("123456-body", result);
}

The test passed and it was perfect.

New journey

Of course, the general article should end here.

But the above is not the focus of this article, our story has just begun.

Log requirements

The wild goose flies by, and the sky will definitely leave his traces.

The procedure should be even more so.

In order to track the problem conveniently, we generally log the incoming parameters of the call to leave traces.

In order to facilitate code expansion and maintainability, Xiaoming of course adopts the interceptor approach.

Log interceptor

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.util.StreamUtils;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.nio.charset.StandardCharsets;
import java.util.Enumeration;

/**
 * 日志拦截器

 * @author 老马啸西风
 * @since 1.0.0
 */
@Component
public class LogHandlerInterceptor implements HandlerInterceptor {

    private Logger logger = LoggerFactory.getLogger(LogHandlerInterceptor.class);

    @Override
    public boolean preHandle(HttpServletRequest httpServletRequest,
                             HttpServletResponse httpServletResponse, Object o) throws Exception {
        // 获取参数信息
        Enumeration<String> enumeration = httpServletRequest.getParameterNames();
        while (enumeration.hasMoreElements()) {
            String paraName = enumeration.nextElement();
            logger.info("Param name: {}, value: {}", paraName, httpServletRequest.getParameter(paraName));
        }

        // 获取 body 信息
        String body = StreamUtils.copyToString(httpServletRequest.getInputStream(), StandardCharsets.UTF_8);
        logger.info("body: {}", body);

        return true;
    }

    @Override
    public void postHandle(HttpServletRequest httpServletRequest,
                           HttpServletResponse httpServletResponse, Object o, ModelAndView modelAndView) throws Exception {

    }

    @Override
    public void afterCompletion(HttpServletRequest httpServletRequest,
                                HttpServletResponse httpServletResponse, Object o, Exception e) throws Exception {

    }

}

Very simple and easy to understand, the parameter parameter and body information in the input and output parameters.

Then specify the effective range:

@Configuration
public class SpringMvcConfig extends WebMvcConfigurerAdapter {

    @Autowired
    private LogHandlerInterceptor logHandlerInterceptor;

    @Override
    public void addInterceptors(InterceptorRegistry registry) {
        registry.addInterceptor(logHandlerInterceptor)
                .addPathPatterns("/**");

        super.addInterceptors(registry);
    }

}

All requests will take effect.

What about my inputStream?

Do you think there is a problem with the log interceptor just now?

If so, how should it be solved?

After Xiaoming finished writing, he thought that everything went well. As soon as he ran the test case, his whole person was split.

httpServletRequest.getInputStream() in all Controller methods becomes empty.

who is it? Stole my inputStream?

After thinking about it, Xiao Ming discovered the problem.

There must be a problem with the log interceptor I just added, because stream can only be read once as a stream. After reading the log once, it will not be read later.

But the log must be output, so what should I do?

Undecided

In case of indecision, the technology asks google, gossip to the scarf.

So Xiao Ming checked it out, and the solution was relatively straightforward and rewritten.

Override HttpServletRequestWrapper

First rewrite the HttpServletRequestWrapper to save the stream information read each time for repeated reading.

/**
 * @author binbin.hou
 * @since 1.0.0
 */
public class MyHttpServletRequestWrapper extends HttpServletRequestWrapper {

    private byte[] requestBody = null;//用于将流保存下来

    public MyHttpServletRequestWrapper(HttpServletRequest request) throws IOException {
        super(request);
        requestBody = StreamUtils.copyToByteArray(request.getInputStream());
    }


    @Override
    public ServletInputStream getInputStream() {
        final ByteArrayInputStream bais = new ByteArrayInputStream(requestBody);
        return new ServletInputStream() {
            @Override
            public int read() {
                return bais.read();  // 读取 requestBody 中的数据
            }

            @Override
            public boolean isFinished() {
                return false;
            }

            @Override
            public boolean isReady() {
                return false;
            }

            @Override
            public void setReadListener(ReadListener readListener) {
            }
        };
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(getInputStream()));
    }

}

Implement Filter

When will the MyHttpServletRequestWrapper rewritten above take effect?

We can implement a Filter ourselves to replace the original request:

import org.springframework.stereotype.Component;

import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;

/**
 * @author binbin.hou
 * @since 1.0.0
 */
@Component
public class HttpServletRequestReplacedFilter implements Filter {

    @Override
    public void destroy() {}

    @Override
    public void doFilter(ServletRequest request, ServletResponse response,
                         FilterChain chain) throws IOException, ServletException {
        ServletRequest requestWrapper = null;

        // 进行替换
        if(request instanceof HttpServletRequest) {
            requestWrapper = new MyHttpServletRequestWrapper((HttpServletRequest) request);
        }

        if(requestWrapper == null) {
            chain.doFilter(request, response);
        } else {
            chain.doFilter(requestWrapper, response);
        }
    }
    @Override
    public void init(FilterConfig arg0) throws ServletException {}

}

Then you can find that everything is okay, and the smile of the Dragon King leaks from the corner of Xiao Ming's mouth.

summary

The original problem is simplified in the article. When this problem is actually encountered, it is directly an interceptor + parameter and body request.

Therefore, it is a waste of time to troubleshoot the whole problem.

But a waste of time if there is no reflection, it is really a waste of .

The two core points are:

(1) Understanding of servlet standards.

(2) Understanding of stream reading and some related knowledge of spring.

I am an old horse, and I look forward to seeing you again next time.