The idea of this cohort is really good, and now it's a highlight on my resume.

Hello, I am crooked.

A few days ago, I saw such a pr in the github of an open source project:

Just looking at the name, with a MemorySafe in it, got me a little bit stuck.

Let me show you this first:

This must be familiar, right? I took the picture from the Alibaba development specification.

Why are FixedThreadPool and SingleThreadPool not recommended?

Because the queue is too long, requests will pile up, and once the requests pile up, it is easy to cause OOM.

Then the question comes again: what is the queue used for the thread pool mentioned above?

A LinkedBlockingQueue with no specified length is used.

No length is specified, the default length is Integer.MAX_VALUE, which can be understood as an unbounded queue:

So, in my cognition, using LinkedBlockingQueue may lead to OOM.

If you want to avoid this OOM, you need to specify a reasonable value during initialization.

"Reasonable value" sounds like an understatement, but what exactly is this value? Are you accurate?

Basically can't tell.

So, when I saw the name MemorySafeLinkedBlockingQueue on pr, I got stuck.

Add the qualifier MemorySafe in front of LinkedBlockingQueue.

Indicates that this is a memory-safe LinkedBlockingQueue.

So, I wanted to study how to achieve "safety", so I clicked in with a snap, very soon.

MemorySafeLBQ

In this pr, let's take a look at what it mainly wants to do:

https://github.com/apache/dubbo/pull/10021

The guy who provided the code described its function like this:

It can completely solve the OOM problem caused by LinkedBlockingQueue, and does not rely on instrumentation, which is better than MemoryLimitedLinkedBlockingQueue.

Then you can see that this commit involves 7 files.

In fact, the real core code is these two:

But don't panic, first familiarize yourself with these two categories, and then I will press the table first. First trace back to the source, from the source.

The names of these two classes are too long, so let's make an appointment first. In this article, I use MemoryLimitedLBQ instead of MemoryLimitedLinkedBlockingQueue. Use MemorySafeLBQ instead of MemorySafeLinkedBlockingQueue.

As you can see, it also mentioned "better than MemoryLimitedLBQ" in pr.

That is, it is used to replace the MemoryLimitedLBQ class.

This class can also be seen from the naming, and it is also a LinkedBlockingQueue, but its qualifier is MemoryLimited, which can limit memory.

I looked for it, and the pr corresponding to this class is this:

https://github.com/apache/dubbo/pull/9722

In this pr, a big guy asked him:

What is the point or purpose of your new queue implementation? Can you name the queue in the current repository that needs to be replaced by this queue? This way we can decide whether to use this queue or not.

That is to say, he just submitted a new queue, but did not say what the application scenario is, so the official did not know whether to accept this PR.

So he added a reply:

It's the example of the FixedThreadPool taken.

In this, the LinkedBlockingQueue without parameters is used, so there is a risk of OOM.

Then you can use MemoryLimitedLBQ to replace this queue.

For example, I can limit the maximum memory that this queue can use to 100M, and avoid OOM by limiting the memory.

Okay, let me sort it out for you first.

First of all, there should be a queue called MemoryLimitedLBQ, which can limit the maximum memory that this queue can occupy.

Then, for some reason, another queue called MemorySafeLBQ appeared, which was claimed to be better than it, so it replaced it.

So, next I will sort out three questions:

• What is the implementation principle of MemoryLimitedLBQ?
• What is the implementation principle of MemorySafeLBQ?
• Why is MemorySafeLBQ better than MemoryLimitedLBQ?

MemoryLimitedLBQ

Don't look at this thing, I saw it in Dubbo's pr, but it is essentially a queue implementation.

Therefore, it can exist completely out of the frame.

That is to say, you open the following link, and then paste the two related classes directly, you can run it for your use:

https://github.com/apache/dubbo/pull/9722/files

Let me first show you the class MemoryLimitedLBQ, which inherits from LinkedBlockingQueue, and then rewrites several of its core methods.

Just customize a memoryLimiter object, and then operate the memoryLimiter object in each core method:

So the real secret is in the memoryLimiter object.

For example, let me show you this put method:

This calls the acquireInterruptibly method of the memoryLimiter object.

Before interpreting the acquireInterruptibly method, let's focus on several of its member variables:

• memoryLimit is the maximum size that this queue can hold.
• memory is of type LongAdder, indicating the currently used size.
• acquireLock, notLimited, releaseLock, and notEmpty are lock-related parameters. As can be seen from the name, both placing elements in the queue and releasing elements in the queue need to acquire the corresponding locks.
• inst This parameter is of type Instrumentation.

The first few parameters are at least familiar to me, but this inst is a bit strange.

This thing is basically useless in daily development, but if you use it, this is a black technology. Many tools are based on this thing, such as the famous Arthas.

It can be more convenient to do bytecode enhancement operations, allowing us to modify the classes that have been loaded or even have not been loaded, and achieve functions similar to performance monitoring.

It can be said that Instrumentation is the key point of memoryLimiter:

For example, in the acquireInterruptibly method of memoryLimiter, it is used like this:

You can see the name of the method, get the size of the object, this object is the input parameter of the method, that is, the element to be put into the queue.

To prove that I'm not talking nonsense, I'll show you the annotations on this method:

an implementation-specific approximation of the amount of storage consumed by the specified object

Note the word: approximation.

This is a serious fourth-level vocabulary, and it still starts with a. If you are not familiar with it, you will be punished.

The whole sentence translates as: Returns an implementation-specific approximation of the amount of storage consumed by the specified object.

To put it more bluntly, it is how long the object you pass in occupies in the memory. This length is not a very precise value.

So, after understanding the inst.getObjectSize(e) line of code, let's take a closer look at what acquireInterruptibly looks like:

First of all, the two places marked with ① indicate that the operation of this method is to be locked, and the method in the whole try is thread-safe.

Then what is going on inside labeled ②?

It is to calculate whether the sum value of the LongAdder type of memory plus the current value of the object is greater than or equal to memoryLimit.

If the calculated value really exceeds the memoryLimit, it means that you need to block and call the notLimited.await() method.

If the memoryLimit is not exceeded, it means that there is still something to put in the queue, then the value of memory is updated.

Then we arrived at the place marked ③.

When you come here, check again whether the currently used value does not exceed the memoryLimit. If so, call the notLimited.signal() method to wake up the object that could not be put in due to the memoryLimit parameter limit.

The whole logic is very clear.

The core logic of the whole logic is to call the getObjectSize method of the Instrumentation type to obtain a size of the currently placed object, and to determine whether the currently used value plus this size is greater than the maximum value we set.

Therefore, you can guess it with your toes. In the release method, the size of the current object must also be calculated, and then subtracted from the memory:

To put it bluntly, it's such a big deal.

Then, look again at the logic in the try code block of the acquireInterruptibly method, and have you found any bugs:

If you haven't reflected it, then let me remind you again: Do you seriously analyze the local variable sum, is it a bit wrong?

If you haven't responded yet, I'll give you a code directly. There is a commit later, which is to modify the sum to memory.sum():

Why change it this way?

Let me tell you a scenario, assuming that our memoryLimit is 1000, the currently used memory is 800, that is, the sum is 800. At this time, the calculated size of the element I want to put is 300, that is, the objectSize is 300.

sum+objectSize=1100, which is larger than the value of memoryLimit, is it blocked during this while judgment:

After that, suppose another object with a size of 600 is released in the queue.

At this time, the memory.add(-objectSize) method is executed, and the memory becomes 200:

Then the signalNotLimited method will be called to wake up the intercepted buddy:

As soon as this buddy wakes up, look at the code:

 while (sum + objectSize >= memoryLimit) {
    notLimited.await();
}

I thought to myself: my sum here is 800, the objectSize is 300, or is it larger than the memoryLimit, what are you doing to wake me up, are you stupid?

So you say, who is it scolding?

The code in this place must be like this, just check the latest memory value every time:

 while (memory.sum() + objectSize >= memoryLimit) {
    notLimited.await();
}

Therefore, this place is a BUG, or an infinite loop of BUG.

There is also a link in the previous code screenshot, that is, this BUG:

https://github.com/apache/incubator-shenyu/pull/3335

Also, you can see the project name in the link is incubator-shenyu, which is an open source API gateway:

Both MemoryLimitedLBQ and MemorySafeLBQ in this article were originally from this open source project.

MemorySafeLBQ

The basic principle of MemoryLimitedLBQ has been understood earlier.

Next, I will show you the MemorySafeLBQ thing.

Its source code can be obtained directly through this link:

https://github.com/apache/dubbo/pull/10021/files

It is also the kind that you can put in your own project to run, and change the author of the file to your own name.

Let's go back to where we started:

This pr said that I made MemorySafeLBQ to replace MemoryLimitedLBQ, because I am better than it, and I do not rely on Instrumentation.

But after reading the source code, you will find that the ideas are almost the same. It's just that MemorySafeLBQ does the opposite.

What kind of "opposite" law?

Take a look at the source code:

MemorySafeLBQ is still inherited from LinkedBlockingQueue, but there is a custom member variable called maxFreeMemory, the initial value is 256 1024 1024.

The name of this variable is very worth noting, you will look at it carefully. maxFreeMemory, the maximum remaining memory, the default is 256M.

The MemoryLimitedLBQ mentioned in the previous section limits how much space the queue can use at most, from the perspective of the queue.

The MemorySafeLBQ limit is the remaining space in the JVM. For example, the default is that when there is only 256M of available memory left in the entire JVM, I will not let you add elements to the queue.

Because the entire memory is relatively tight, the queue cannot continue to be added without limit. From this perspective, the risk of OOM is avoided.

Such a one does the opposite.

Also, it says it doesn't depend on Instrumentation anymore, so how does it detect memory usage?

The MemoryMXBean in ManagementFactory is used.

You are no stranger to this MemoryMXBean.

Have you used JConsole?

Have you entered the interface below?

This information is taken from the ManagementFactory:

So, it's true that it doesn't use Instrumentation, but it does use ManagementFactory.

The purpose is to obtain the running status of the memory.

So how can it be seen that it is better to use than MemoryLimitedLBQ?

I saw that the key method is this hasRemainedMemory, which must be called before the put and offer methods are called:

And you see that MemorySafeLBQ just overrides the put and offer methods of putting elements, and does not focus on removing elements.

why?

Because its design concept is to only care about the remaining space when adding an element, it doesn't even care about the current size of the element.

And remember the MemoryLimitedLBQ mentioned earlier? It also calculates the size of each element, and then makes a variable to accumulate.

There is only one line of code in the hasRemainedMemory method of MemoryLimitedLBQ, where maxFreeMemory is specified when the class is initialized. Then the key code is MemoryLimitCalculator.maxAvailable().

So let's look at the source code of MemoryLimitCalculator.

The source code of this class is very simple to write. I only have this little bit of content after I cut it, and it all adds up to more than 20 lines of code:

The core of the whole method is the static code block I framed, which contains three lines of code.

The first line is to call the refresh method, that is, to reassign the parameter maxAvilable, which means the JVM memory that can be used currently.

The second line injects a timed task that runs every 50ms. When the point is reached, trigger the refresh method to ensure the quasi-real-time performance of the maxAvilable parameter.

The third line is the ShutdownHook added to the JVM. When the service is stopped, the scheduled task needs to be stopped to achieve the purpose of graceful shutdown.

The core logic is just that.

From my point of view, it is indeed simpler and better to use than MemoryLimitedLBQ.

Finally, let's look at the MemorySafeLBQ test case provided by the author. I added a little comment, which is very easy to understand. I will try it myself, and I won't say more:

it's yours

The MemoryLimitedLBQ and MemorySafeLBQ mentioned in the article, as I said, these two things are completely independent of the framework, and the code can be used directly by sticking it.

There are only a few lines of code. Whether using Instrumentation or ManagementFactory, the core idea is to limit memory.

Let’s expand the idea. For example, if we use Map as a local cache in some projects, we will put a lot of elements in it, and there is also the risk of OOM. Then, through the ideas mentioned above, have we found a solution to the problem?

Therefore, the idea is very important. If you master this idea, you can also talk a few words during the interview.

For another example, when I saw this thing, I thought of the dynamic adjustment of thread pool parameters that I wrote before.

Take the MemorySafeLBQ queue as an example, can the parameter maxFreeMemory in it be dynamically adjusted?

It is nothing more than the adjustment of the previous queue length to adjust the memory space occupied by the queue. It is only a parameter change, and the implementation scheme can be directly applied.

These are what I see from open source projects, but the moment I see it, it's mine.

Now, I write it out, share it with you, and it's yours.

You're welcome, just come for three.