The little thing about Request reuse.

Hello, I am crooked.

Didn't I publish this article before: "Don't pass Request to an asynchronous thread! There is a pit! 》

It is said that since Request is multiplexed in tomcat, if a related method is called in an asynchronous thread after the life cycle of a Request is completed, it will cause the Request to be polluted, and then some unbelievable observations will be observed in the next request. scene.

But there was a question in the comment area of the article, and it stopped me:

Since the focus of my article is on the operation of passing the Request to the asynchronous thread, I didn't pay any particular attention to how the Request is reused.

I just observed this phenomenon of reuse by printing the log:

After starting the project, visit testRequest and testRequest1 respectively. From the console output, the Request object is indeed an object.

But judging from the previous thread names, these are two completely different threads in the thread pool.

So, although I haven't analyzed anything yet, at least the answer to this question can be seen based on the log:

Are multiplexed requests bound to threads?

No, there is no binding relationship.

If it is not bound to the thread, then the problem will follow:

How to decide which thread to reuse which request each time?

This is a good question and I don't know the answer, so I decided to give it a try.

But before we set it up, let's think about a question: Assuming that Request and the request thread are bound together, is this a reasonable design?

Certainly not.

A thread should be a pure thread, and it should not be "bound" to a Request. This binding makes the thread not simple, and the thread and the request are coupled together.

A better design should be to put the Request in a "pool", and a thread will fetch the available Request from the pool.

This can achieve the effect of decoupling between threads and requests.

Of course, this is just an assumption I made before exploring. I will put it here first, and finally see if this conjecture is correct.

You don't need to know much about Tomcat to read this article. You can use it. Many things can be inferred based on the source code.

By the way, let's talk about the Tomcat source code version: 9.0.58.

first breakpoint

To find the answer to the question, you must go to the source code, but where to start?

Or another question: where is the first breakpoint?

When I encounter this problem, my first reaction is to see if I can find relevant clues from the log, so as to find the location of the first breakpoint.

However, I adjusted the log to DEBUG level and TRACE level respectively, and found no valuable information, so I feel that the log path cannot work, what should I do?

Don't panic, it's time to calm down and analyze it.

Quietly ask yourself: Can I put a breakpoint at the method entry?

Of course, this is also a very conventional method that can be thought of:

But if you hit the breakpoint here, it is equivalent to pushing the source code backward from the first line of the business code, and detouring the road a little further.

So where can the breakpoint be hit?

Didn't I output the full class name of the Request object here:

http-nio-8080-exec-2: testRequest1=org.apache.catalina.connector.RequestFacade@5db48dd3

RequestFacade, if this class can be used, there must be a place to new it, and to new it, its constructor must be called.

Then do I just need to make a breakpoint on its corresponding constructor, when the program creates this class, isn't it the source I'm looking for?

So, I hit the first breakpoint on the constructor of RequestFacade.

Starting with the construction method, this is also one of my debugging tips. I'll give it to you, you're welcome.

Some friends will ask: what if a class has multiple constructors?

It's very simple. Make a miracle. Every construction method has a breakpoint, and there must be a place to trigger it.

Debug source code

Find the location of the first breakpoint, the next step is to restart the project and initiate the call.

I made two calls in a row, and I knew from the performance of the program that the breakpoint was hit.

I'll give you a moving picture first, and you'll know why I said that:

After the project is started, the first call stops at the breakpoint, and then the second call does not stop at the breakpoint.

It shows that the RequestFacade object is not created for the second time, but the RequestFacade object generated in the first call is reused.

After verifying that the position of the breakpoint is correct, you can start debugging slowly.

First, let's focus on where this RequestFacade object is created:

There are two if judgments.

The first is to judge whether the facade is null, and if it is not null, it is new.

The second is to assign the facade to the applicationRequest object, and then return the applicationRequest object.

The second if is actually very interesting. You think, you can directly return to the facade here. Why use applicationRequest to undertake it?

This is a good question.

The key to these two ifs is whether facade and applicationRequest are empty.

It was definitely empty on the first visit. So when will it be empty again?

That is, when a request ends and the recycle method is executed:

org.apache.catalina.connector.Request#recycle

You can see from the source code that applicationRequest is directly set to null.

But there is a premise that this facade is set to null, the getDiscardFacades method returns true.

What is this?

You can see at a glance:

It means that the RECYCLE_FACADES parameter controls whether to cycle the facade object. If it is set to true, it will improve security, and this parameter defaults to false.

That is to say, if I change this parameter to true in this place, the facade object will be recycled after each call is completed.

It can be configured through the startup parameter JAVA_OPTS:

-Dorg.apache.catalina.connector.RECYCLE_FACADES=true

As you can see from the previous source code, by default, applicationRequest will be set to null after each request is completed, and the facade will remain.

Therefore, when the next request comes, the facede is not empty, and the facade is directly reused. Assign facade to applicationRequest .

So what we observed in the log is that the facade object output by the two requests is the same.

Next, we move on to the call stack.

See who is calling the getRequest request to create the facade:

It is found that a Request object is calling the getRequest method.

So the next thing to look for is the method from which the Request object was first passed as an input parameter.

Going down the call stack, you can find the following place:

org.apache.coyote.http11.Http11Processor#service

This is where the Request object is initially passed as an argument.

So how does this Request object come about?

I do not know either.

Therefore, to know the answer to this question, the position of the second breakpoint is ready to appear:

Restart the project, initiate a request, and find that Debug stops at the constructor of the AbstractProcessor class. This is where the request is first generated, and we also gain a call stack:

org.apache.coyote.AbstractProcessor#AbstractProcessor(org.apache.coyote.Adapter, org.apache.coyote.Request, org.apache.coyote.Response)

How did this Request come from?

new out:

Why do you want to execute this new method?

Because this place is in createProcessor:

And the answer to the question we are looking for is hidden in the screenshot above.

To be precise, it is hidden in the screenshot above, where the five-pointed star is marked:

processor = recycledProcessors.pop();

From the code snippet, if the processor object popped from recycledProcessors is not empty, the createProcessor method will not be called.

From a debugging point of view, the RequestFacade object will not be created without calling the createProcessor method.

Therefore, recycledProcessors, this thing is a Chinese point and a real breakthrough.

In this section, I mainly share the process of finding this breakthrough. Two key breakpoints are set based on the above considerations.

In fact, if you think about it, this is a very natural thing. It is a relatively simple thing to debug the source code with problems.

Don't be shy, just flip.

recycledProcessors

If you look at the name of this object, recycled + Processors, you can see that there is a story in it, a story about object reuse.

org.apache.coyote.AbstractProtocol.RecycledProcessors

The methods of this class are also very simple. There are three methods: push, pop, and clear.

Inheriting to the SynchronizedStack object is a standard stack structure, but using Synchronized to modify the corresponding method:

It is mentioned in the annotation of the SynchronizedStack class that this is an object pool. This object pool does not need to be shrunk. The purpose is to reduce garbage objects and release GC pressure.

Now we have found the object pool, and also the place to call the object pool pop.

So when to push to this object pool?

I do not know either.

So the third breakpoint is here, and it can be hit on the push method:

Then make a call and find that when the request processing is completed and the current processor is released, put the processor into recycledProcessors and wait for the next request to use it:

At this point we have mastered such a closed loop:

When the request comes, first see if there is any processor available in the stack structure of recycledProcessors, if not, call the createProcessor method to create a new one, and then put it into the stack structure after the request ends.

When the createProcessor method is called, a new Request object will be constructed, and finally the Request object will be encapsulated as a RequestFacade object.

So I now want to verify that there is such a correspondence between Processor, Request and RequestFacade.

How to verify it?

Print log.

Note, here is another debugging trick.

I want to add a line of output statement after the processor is selected:

How to add it?

Create a package path that is the same as the source code in your own project, and then paste the corresponding class directly:

Because it is in your own project, you can change it however you want:

For example, I add this output statement to print out the processor and the request inside.

After making the request, you will find that it does take effect, but the output of reuqest is like this:

why?

Because in the source code, the toString method of this class is overridden:

How to do?

Change the source code, I just taught you:

After modifying the call, you can see the corresponding expected output in the console:

You see, there is a request in the processor. Now what I'm looking for is the relationship between request and RequestFacade.

Very simple, also output a line here in the getRequest method:

After initiating the call, I found that it was finished:

These two Requests are not the same thing at all:

org.apache.coyote.Request@667cbb30
org.apache.catalina.connector.Request@9ffc697

Don't panic, calm down and talk about it. Although these are two different Requests, they must be inextricably linked.

Let's take a look at how org.apache.catalina.connector.Request came from, the old rules, the breakpoint on the construction method:

Based on this call stack, looking forward a little bit, you can see something worth noting:

org.apache.catalina.connector.CoyoteAdapter#service

In this method in the screenshot above, there is a line of code like this:

request.setCoyoteRequest(req);

Where request is the org.apache.catalina.connector.Request object.

And req is the org.apache.coyote.Request object.

That is to say, my output statement here should be like this:

After the modification, the call is made again, and the output log is as follows:

If you haven't seen something, let me process it for you:

It means that Processor and RequestFacade are indeed one-to-one correspondence.

Going back to the screenshot at the beginning of the article, why do I make two requests and the RequestFacade object is the same?

Because the same Processor is used for the two requests.

You see, I make two more requests, both of which are being processed by Http11Processor@26807016:

Therefore, on the surface, it is the same RequestFacade, but in fact the same Processor is used.

In other words: if two requests use different processors, there will be no reuse.

How to verify it?

I thought of the following verification method:

I can request sleepTenSeconds first, then testRequest within 10s. This way, I can observe two different Processors:

In order to see this phenomenon more intuitively.

I decided to output the contents of recycledProcessors before operating the pop method of recycledProcessors and after the push method:

org.apache.coyote.AbstractProtocol.RecycledProcessors

But when you write it like this, you will find that the parent class of RecycledProcessors, that is, the SynchronizedStack class, does not provide a print method. What should I do?

It's very simple, I can get the source code, and adding a method, isn't it a matter of hand?

Then, I still initiate the request in the order of accessing sleepTenSeconds and then accessing the testRequest method. The log is as follows:

Take it out separately, after the entire request of testRequest is completed, the corresponding log is like this,

 ========pop之前【开始】打印当前所有Processor========
========pop之前【结束】打印当前所有Processor========
1.processor=org.apache.coyote.http11.Http11Processor@6720055f,request=org.apache.coyote.Request@69e7f7cb
2.coyoteRequest=org.apache.coyote.Request@69e7f7cb,facade=org.apache.catalina.connector.RequestFacade@6dd86e2f
3.http-nio-8080-exec-1：testRequest = org.apache.catalina.connector.RequestFacade@6dd86e2f
========push之后【开始】打印当前所有Processor========
org.apache.coyote.http11.Http11Processor@6720055f
========push之后【结束】打印当前所有Processor========

After the entire request of sleepTenSeconds is completed, the corresponding log is as follows:

 ========pop之前【开始】打印当前所有Processor========
========pop之前【结束】打印当前所有Processor========
1.processor=org.apache.coyote.http11.Http11Processor@7ba33829,request=org.apache.coyote.Request@1334fe58
2.coyoteRequest=org.apache.coyote.Request@1334fe58,facade=org.apache.catalina.connector.RequestFacade@2a0231eb
3.http-nio-8080-exec-2：sleepTenSeconds = org.apache.catalina.connector.RequestFacade@2a0231eb
========push之后【开始】打印当前所有Processor========
org.apache.coyote.http11.Http11Processor@6720055f
org.apache.coyote.http11.Http11Processor@7ba33829
========push之后【结束】打印当前所有Processor========

That is to say, there are two Processors in recycledProcessors at this time:

 ========push之后【开始】打印当前所有Processor========
org.apache.coyote.http11.Http11Processor@6720055f
org.apache.coyote.http11.Http11Processor@7ba33829
========push之后【结束】打印当前所有Processor========

Then the question arises: You said that I will initiate a request again next, which Processor will undertake this request?

Although I haven't initiated the request yet, I know that it must be Http11Processor@7ba33829 to handle it.

Because I know it will be the next Processor object to be popped.

If you don't believe me, just watch this animation:

In the above animation, I first testRequest this request.

What if I visit sleepTenSeconds first and then testRequest?

Although I haven't made a request yet, I know that there must be such a correspondence to handle these two requests:

sleepTenSeconds->Http11Processor@7ba33829
testRequest->Http11Processor@6720055f

Because when the sleepTenSeconds request comes, the object Processor@7ba33829 will be popped in the recycledProcessors to process the request.

Therefore, within 10 seconds, that is, when the sleepTenSeconds request is not completed, the testRequest request is accessed, and the object Http11Processor@6720055f is then popped out of the recycledProcessors.

If you don't believe me, watch this animation again:

So, now have we found the answer to this question:

How to decide which thread reuses that request each time?

There is no relationship between the request thread and the request. Which request is used for each request depends on which Processor is used. Which Processor is used for each request depends on which Processors are cached in the recycledProcessors class. When the request comes, whichever pops out is the one.

Since recycledProcessors is a cache, its size determines the performance of the project to a certain extent.

And its default value is 200:

Why 200?

Because the maximum number of threads in the tomcat thread pool is 200 by default:

Can you understand this?

Although there is no binding relationship between a thread and a Processor, logically a thread corresponds to a Processor. Therefore, it is a good idea to keep the number of threads and the number of Processors the same.

If I change the processorCache parameter to 1:

server.tomcat.processor-cache=1

What happens when you say high concurrency?

Many requests for push will not push in, so they go to the logic of handler.unregister(processor):

And this unregister method corresponds to a register method, let me show you:

They are holding the same synchronized lock, which means there is competition between them.

We know that the push method of RecycledProcessors will be called after a request ends, and the unregister method will be called during push.

Then the question arises: when is register called?

In fact, it has already appeared before:

A request comes, after the processor is created.

Therefore, when I set the processorCache to 1, in the case of high concurrency, I keep calling register and unregister, and the lock competition is frequent and the performance is degraded.

This conclusion is the conclusion I came to by reading the source code, rather than a ready-made conclusion obtained in some other book or video.

This is the joy and meaning of reading the source code.

pay back

When I wrote this, I couldn't help thinking of what I wrote in "Don't pass Request to an asynchronous thread! There is a pit! " The pit stepped on in this article.

Take a look at this animation again, focusing on the output corresponding to the console when it is called twice:

It is because it is used outside the life cycle of Request, which leads to problems in reuse.

At that time, the correct solution I gave was to use the asynchronous programming of Request, that is, the set of startAsync and AsyncContext.complete methods.

But after writing this article, I thought of two more tricks.

The first method is hidden in the RECYCLE_FACADES configuration I mentioned earlier.

From the description on the official documentation, if this parameter is set to true, it will improve security, but it defaults to false.

How does it improve security?

That is, the RequestFacade is recycled every time.

Then I'll change it to true and try it out to see what works:

-Dorg.apache.catalina.connector.RECYCLE_FACADES=true

Start the project and make a call:

An exception was thrown.

When I saw this exception, I immediately understood the meaning of "security" in the official document: your usage is wrong, I will throw an exception for you, remind you, here needs to be modified and improved safety.

And the second one is this:

server.tomcat.processor-cache=0

You see what I mean?

I will not let you reuse it, and use a new one every time to bypass the "pit" of reuse:

Don't worry about whether it works well or not, and whether there are performance problems, you can say that after you thoroughly understand the underlying logic, this operation is not good.