Changed the data structure and accidentally saved 591 machines!

Hello, I am crooked.

Some time ago, I saw a technical video on station B titled "Some Solutions for Concurrent Scenarios with High Air Ticket Quotes".

up is the base of Qunar technology, which is also known as "where to go".

Video link is here:

https://www.bilibili.com/video/BV1eX4y1F7zJ?p=2

At that time, I was actually attracted by this picture of him (the 12 qps in it should be 12k qps):

He introduced that the two core systems save the server resources of 204C and 2160C respectively after a "data compression" operation.

The total is 2364C of server resources.

If you follow the standard 4C8G server, my dear, this is a saving of 591 machines. Think about how much you can save in one year.

The video introduces several data compression schemes, one of which is to use high-performance collections:

Because their system design uses a lot of "local cache", and most of the local cache is to use HashMap to help.

So they replaced the HashMap with the better performing IntObjectHashMap, a class from Netty.

Why does it save so many resources after changing a class?

In other words, what makes IntObjectHashMap perform better?

I didn't know either, so I went to research it.

Pull the source code

The first step in research must be to find the corresponding source code.

You can find a Netty dependency and find the IntObjectHashMap inside.

I just have the Netty source code that I pulled down before, and I just need to synchronize the latest code.

But when I looked for this class in the 4.1 branch, I didn't find it, and only saw a related Benchmark class:

Click in and see that there is indeed no IntObjectHashMap class:

I'm very puzzled, I don't understand why anyway, but I just don't care anymore. Anyway, I just want to find an IntObjectHashMap class.

If there is no 4.1 branch, then take a look on 4.0:

So I cut to the 4.0 branch to look for it, and I found the corresponding class and test class very smoothly:

Being able to see the test class is actually the reason why I like to pull down the project source code. If you find the corresponding class by importing Netty's Maven dependency, you won't see the test class.

Sometimes I look at the source code in conjunction with the test class, and get twice the result with half the effort. I will give you a little trick to look at the source code.

And one of the most important purposes for me to pull the source code is actually this:

You can see the submission records of this class and observe the evolution of this class, which is very important.

Because in most cases, a submission corresponds to a bug modification or performance optimization, which is where we should pay attention.

For example, we can see that this guy has submitted three times for the hashIndex method:

Before formally studying the source code of IntObjectHashMap, let's take a look at the local method that only focuses on hashIndex.

First, the code for this place now looks like this:

I know that this method is to get the subscript of the key of type int in the keys array, which supports the case that the key is a negative number.

So why is this line of code submitted three times?

Let's look at the first commit first:

Very clear, the left is the most original code, if the key is negative, then the returned index is a negative number, which is obviously illogical.

So someone submitted the code on the right. When the hash value is negative, add the length of the array, and finally get a positive number.

Soon, the buddies who submitted the code found a better way of writing and made an optimized submission:

The judgment of less than zero is removed. Regardless of whether the value calculated by key%length is positive or negative, the result is added to the length of the array and then % is performed on the length of the array again.

This ensures that the calculated index must be a positive number.

The code submitted for the third time is easy to understand. Substitute the variable:

So, the final code looks like this:

return (key % keys.length + keys.length) % keys.length;

Isn't this way of writing more elegant and better performance than judging less than zero? And this is also a regular optimization scheme.

If you can't see the commit log, you can't see the evolution of the method. What I want to express is that there is a lot more valuable information than the source code can be mined in the code submission record.

Another little trick for you.

IntObjectHashMap

Next, let's explore the mystery of IntObjectHashMap together.

Regarding this Map, there are actually two related classes:

Where IntObjectMap is an interface.

They do not depend on anything other than JDK, so after you understand the principle, if you find a suitable scene in your business scenario, you can paste these two classes into your own project without changing a single line of code , just use it.

After studying the official test cases and code submission records, I chose to paste these two classes first and write a code to debug it myself. The advantage of this is that the source code can be modified at any time so that we can conduct research.

Before arranging the source code of IntObjectHashMap, let's pay attention to these few sentences in its javadoc:

The first sentence is very critical. Here is the solution to IntObjectHashMap for key conflicts:

It uses the open addressing strategy for the key, that is, the open addressing strategy.

Why use open addressing instead of using a linked list like HashMap?

This question is also answered here: To minimize the memory footprint, that is, to minimize the memory footprint.

How to reduce memory usage?

This question will be said when looking at the source code below, but here is a sentence: if you use a linked list, do you have to have at least one next pointer, and does it take up space to maintain this thing?

Without further ado, back to open addressing.

Open addressing is a strategy, which is also divided into many implementation schemes, such as:

• Linear Probing
• Quadratic probing
• Double hashing

As you can see from the last sentence of the underlined part above, IntObjectHashMap uses linear probing, that is, linear detection.

Now we basically understand the solution used by IntObjectHashMap for hash conflicts.

Next, let's do a test case practice. The code is very simple, just one initialization and one put method:

Just a few lines of code, at first glance, it seems to be no different from HashMap. But after thinking about it carefully, I still found a clue.

If we use HashMap, the initialization should be like this:

HashMap<Integer,Object> hashMap = new HashMap<>(10);

Take a look at what the class definition of IntObjectHashMap looks like?

There is only one Object:

This Object represents the value contained in the map.

So what is the key and where did it go? Did the first question arise?

After looking at the put method, I found that the key turned out to be a value of type int:

That is, this class has restricted the value of key to be of type int, so the generic type of key cannot be specified during initialization.

This class has also clearly stated this from the name: I am IntObjectHashMap, the key is int, and the value is the HashMap of Object.

So why did I use "even"?

Because you look at what the key of HashMap is:

is an Object type.

That is, if we want to initialize HashMap like this, it is not possible:

ide will remind you: Brother, don't make trouble, you can't put basic types here, you have to create a package type.

And when we usually code, we can put the int type in this way because the "boxing" operation is hidden:

That's why there is an ancient eight-legged text asking: Can the keys of HashMap use basic types?

Don't even think about it, you can't!

key, from a wrapper type to a basic type, this is a performance optimization point. Because basic types are known to take up less space than wrapper types.

Next, let's start with its construction method, focusing on the part I framed:

The first thing to come in is two if judgments, which check the validity of the parameters.

Next, look at the place labeled ①. From the method name, it is necessary to adjust the capacity:

As can be seen from the comments on the code and method, here is to adjust the capacity to an odd number, for example if I give in 8, it will adjust it to 9 for me:

As for why the capacity cannot be an even number, an explanation is given from the comments:

Even capacities can break probing.

This means that probing will be destroyed when the capacity is even, that is, the linear probing we mentioned earlier.

Forehead...

I haven't figured out why an even-numbered capacity would break linear detection, but it doesn't matter, just be suspicious, and then go down to the main process.

It can be seen from the place labeled ② that this is doing data initialization. We got a capacity of 9 earlier, here are the initial two arrays, namely key[] and values[], and the capacities of these two arrays are the same, both are 9:

After the two arrays are initialized in the constructor, they look like this:

In the construction method, we mainly focus on the change of capacity and the two arrays of key[] and values[].

The construction method has paved the way for you, and then we look at the put method, and it will be smoother:

There are only a few lines of code for the put method, and the analysis is very clear.

The first is the place labeled ①, the hashIndex method is to obtain the subscript in the key[] array corresponding to the key of this put.

This method has been analyzed at the beginning of the article, and we even know the evolution of this method, so I won't say more.

Then there is a for(;;) loop.

First look at the place labeled ②, you should pay attention to it, the judgment condition at this time is value[index] == null , which is to judge whether the subscript corresponding to the value[] array corresponding to the calculated index has a value.

Earlier, I specifically emphasized a sentence and drew a picture for you:

The two arrays key[] and values[] have the same capacity.

Why not first determine whether the index exists in key[]?

It can be, but if you think about it, if the value in the corresponding subscript of value[] is null, it means that nothing has been maintained in this position. The positions of key and value are in one-to-one correspondence, so there is no need to care whether the key exists or not.

If value[index] == null is true, it means that the key has not been maintained before, and the corresponding value is directly maintained, and the key[] and values[] arrays need to be maintained separately.

Assuming that my demo code is used as an example, after the fourth loop, it looks like this:

After the maintenance is completed, determine whether the current capacity needs to be expanded:

The code for growSize looks like this:

In this method, we can see that the expansion mechanism of IntObjectHashMap is to expand twice at a time.

One more thing: this place is a bit low, and the double expansion in the source code must be performed by upper-level operations. It is just right to use length << 1.

But a condition needs to be met before expansion: size > maxSize

size, we know that it means that there are several values in the current map.

So what is maxSize?

This value is initialized in the constructor. For example in my example code maxSize is equal to 4:

That is to say, if I insert another data, it will expand. For example, after I insert the fifth element, the length of the array becomes 19:

Earlier we discussed the case where value[index] == null is true. So what if it is false?

It came to the place marked ③.

Determine whether the value at the index subscript of the key[] array is the current key.

If yes, describe to be overwritten. First take out the value at the original position, then do an overwrite operation directly, and return the original value, the logic is very simple.

But what if it wasn't for this key?

Explain what?

Does it mean that the index position where this key wants to be placed has already been occupied by other keys?

Is this situation a hash conflict?

What should I do if there is a hash conflict?

Then I came to the place labeled ③, and looked at the comments of this place:

Conflict, keep probing...
Conflict, keep probing...

Continuing to detect is to see what is the next position of the currently conflicting index.

If you let me write it, it's very simple, the next position, I can knock it out with my feet closed, it's index+1.

But let's see how the source code is written:

It does see index+1, but there is one more prerequisite, which is index != values.length -1 .

If the above expression holds, it is very simple to use index+1.

If the above expression does not hold, it means that the current index is the last position of the values[] array, then 0 is returned, that is, the first index of the array is returned.

To trigger this scene is to engage in a hash conflict scene. I write a code to show you:

The above code will only put something into the IntObjectHashMap when the calculated subscript is 8, so there will be a hash conflict at the subscript 8 position.

For example, within 100, the numbers with subscript 8 are these:

After the first loop it looks like this:

In the second loop, the key is 17, and it will find that the subscript 8 has been occupied:

So, came to this judgment:

returns index=0, so it falls here:

It looks like a ring, right?

Yes, it is a ring.

But take a closer look at this judgment:

After each index is calculated, it is also necessary to judge whether it is equal to the startIndex of this loop. If it is equal, it means that it has run a circle and has not found an empty seat, then throws an "Unable to insert" exception.

Some friends immediately jumped out: No, won't it expand by 2 times after using half of the space? It should have been expanded long before the capacity was full, right?

This friend, you are very witty. Your question is the same as the question I had when I first saw this place. We are all thoughtful and good children.

But pay attention, where the exception is thrown, a comment is given in the source code:

Can only happen if the map was full at MAX_ARRAY_SIZE and couldn't grow.
This can only happen if the map was full at MAX_ARRAY_SIZE and couldn't grow.

For expansion, there must be an upper limit. Let’s look at the source code when expanding:

The maximum capacity is Integer.MAX_VALUE - 8, indicating that there is an upper limit.

But wait, Integer.MAX_VALUE I see, what's the case with minus 8?

Hey, I know it anyway, but we just don't talk about it, it's not the focus of this article. If you are interested, go and explore by yourself, and I will take a picture for you to finish:

What if I want to verify "Unable to insert"?

Isn't that simple? The source code is in my hands.

Two solutions, one is to modify the source code of the growSize() method, and modify the longest length limit to a specified value, such as 8.

The second solution is to directly prohibit expansion, and annotate this line of code to it:

Then run the test case:

You will find that when inserting the 10th value, an "Unable to insert" exception is thrown.

The 10th value, 89, is like this. After a circle, it goes back to startIndex:

This condition is met, so an exception is thrown:

(index = probeNext(index)) == startIndex

At this point, the put method is finished. You also learned about its data structure and how it works.

So do you remember what the question I was looking for in writing this article?

What is the reason for the better performance of IntObjectHashMap?

One point mentioned earlier is that the key can use the native int type instead of the wrapped Integer type.

Now I'm going to reveal the second point: there is nothing messy about value, value is a pure value. What you put in is what it is.

Think about the structure of HashMap, which has a Node that encapsulates the four attributes of Hash, key, value, and next:

This part of things is also saved by IntObjectHashMap, and this part of the savings is where the bulk is.

You don't look down on a little memory usage. In the face of a huge base, any small optimization can be magnified countless times.

I don't know if you still remember this case in the book "In-depth Understanding of Java Virtual Machine":

.png)

Improper data structures lead to excessive memory usage. This problem can be completely solved by using Netty's LongObjectHashMap data structure, just by changing the class, you can save a lot of resources.

The truth is the same.

one extra point

Finally, let me add to you an additional windfall when I looked at the source code.

Deletions implement compaction, so cost of remove can approach O(N) for full maps, which makes a small
recommended. , so we recommend using a smaller loadFactor.

There are two words in it, compaction and loadFactor.

Let's talk about the loadFactor property, which is initialized in the constructor:

Why does loadFactor have to be a number between (0,1]?

First look at when loadFactor is used:

It is only used when calculating maxSize, which is to multiply the current capacity by this factor.

If this coefficient is greater than 1, then the final calculated value, that is, maxSize, will be greater than capacity.

Assuming that our loadFactor is set to 1.5 and the capacity is set to 21, then the calculated maxSize is 31, which has already exceeded the capacity, which is meaningless.

In short: loadFactor is used to calculate maxSize, and as mentioned earlier, maxSize is used to control the expansion conditions. That is to say, the smaller the loadFactor, the smaller the maxSize, and the easier it is to trigger the expansion. Conversely, the larger the loadFactor, the harder it is to expand. The default value for loadFactor is 0.5.

Next, I will explain that there is a word compaction in the previous comment, which is called this thing when translated:

It can be understood as a kind of "compression", but the sentence "deletion achieves compression" is very abstract.

Don't worry, I'll tell you.

Let's take a look at the delete method first:

The logic of the delete method is a bit complicated, and it's a bit puzzling if you want to rely on my description to make it clear to you.

Therefore, I decided to only show you the results, and you can use the results to push back the source code.

First of all, the previous comment said: buddy, I recommend that you use the smaller loadFactor.

Then I won't listen to it and will directly fill the loadFactor to 1 for you.

That is to say, when the map is full, adding something in it will trigger the expansion.

For example, I initialize like this:

new IntObjectHashMap<>(8,1);

Does it mean that the current initial capacity of this map can hold 9 elements, and the expansion operation will only be triggered when you put the 10th element.

Hey, coincidentally, I just want to put 9 elements, I don't trigger the expansion. And my 9 elements all have hash conflicts.

code show as below:

These values should have been placed at the subscript 8, but after linear detection, the array in the map should be like this:

At this point we remove the key 8, which normally should look like this:

But it actually looks like this:

It will move back all the values that have been displaced because of the hash conflict.

This process, I understand, is the "compaction" mentioned in the comments.

The actual output of the above program is this:

It matches the picture I drew earlier.

However, I'll note that my code was slightly tweaked:

Without any modification, the output should look like this:

key=8 is not the last one, because the operation of rehash is involved in this process. If reHash is added when explaining "compaction", it will be complicated and will affect your understanding of "compaction".

In addition, this thing is mentioned in the comments of the removeAt method:

This algorithm is actually the "compaction" I explained earlier.

I searched for keywords globally and found that both IdentityHashMap and ThreadLocal are mentioned:

But, you pay attention to this but ah.

In ThreadLocal, "unlike" is used.

ThreadLocal also uses linear detection for hash conflicts, but the details are still a bit different.

Without going into details, interested students can explore it by themselves. I just want to express that this part can be compared and studied.

The title of this section is called "One Extra Point". Because I didn't plan to have this part of the content, but I saw this when I was looking through the commit record:

https://github.com/netty/netty/issues/2659

There are many discussions in this issue. Based on this discussion, it is equivalent to a big transformation of IntObjectHashMap.

For example, I can know from this submission record that IntObjectHashMap used the "Double hashing" strategy for hash conflicts before, and then changed it to linear detection.

Including the suggestion of using a smaller loadFactor and the algorithm used in removeAt are all based on this transformation:

Quoting a conversation in this issue:

This dude said: I've got carried away, I made a major improvement to this code.

In my opinion, this is not considered a "major improvement", it is already a rewrite.

Also, what does "I've got carried away" mean?

English teaching, late but late:

Remember this phrase, it may be taken during the TOEFL speaking test.

Netty 4.1

At the beginning of the article, I said that in Netty 4.1, I didn't find the IntObjectHashMap thing.

In fact, I lied to you, I found it, but it was hidden a little deep.

In fact, I only wrote int in this article, but in fact, basic types can be transformed based on this idea, and their codes should be similar.

So in 4.1, a saucy operation is used, which is encapsulated once based on groovy:

To compile this template after:

Only then will we see what we are looking for in the target directory:

However, if you look closely at the compiled IntObjectHashMap, you will find a little different.

For example, the method of adjusting capacity in the construction method becomes like this:

From the method name we also know that here is to find the nearest multiple of 2 of the current value.

Wait, a multiple of 2, isn't it an even number?

In the code of the 4.0 branch, the adjustment capacity has to be an odd number:

Remember a question I mentioned earlier: I didn't think about understanding why even capacity would break linear probing?

But it can be seen from here that even-numbered capacities are also possible.

This confuses me.

If in the code of the 4.0 branch, the adjustCapacity method does not have this line of specially written comments:

Adjusts the given capacity value to ensure that it's odd. Even capacities can break probing.

I would have no hesitation in thinking that this place can be even odd. But he deliberately emphasized the need for "odd numbers", which made me a little bit unbearable.

Forget it, I can’t learn anymore, I’m in doubt!

The article was first published on the public account [why technology], everyone is welcome to pay attention and see the latest article as soon as possible.