ArrayPool source code interpretation of byte[] can also be pooled?

One: background

1. Tell a story

Recently, in the process of analyzing a dump, it was found that there are many large-size frees on gen2 and LOH. Upon closer inspection, most of these frees were byte[] arrays of html fragments generated by the template engine. Of course, this one I'm not here to analyze dump, but to talk about how to make memory utilization more efficient when there are many byte[] arrays with a larger length in the managed heap, and how to make gc old man less stressful.

I don't know if you have found that a lot of pooled objects have been added to .netcore, such as: ArrayPool, ObjectPool, etc., which are really useful in some scenarios, so it is necessary to have a deeper understanding of them.

Two: ArrayPool source code analysis

1. A picture is worth a thousand words

After I spent nearly an hour reading the source code, I drew a pooling diagram of a picture in hand, I have the world.

With this picture, let's talk about a few concepts and match the corresponding source code, I think it should be almost the same.

2. What is the architectural hierarchy of pooling?

ArrayPool is composed of several Buckets, and Bucket is composed of several buffer[] arrays. With this concept, we need to configure the code.

public abstract class ArrayPool<T>
{
    public static ArrayPool<T> Create()
    {
        return new ConfigurableArrayPool<T>();
    }
}

internal sealed class ConfigurableArrayPool<T> : ArrayPool<T>
{
    private sealed class Bucket
    {
        internal readonly int _bufferLength;
        private readonly T[][] _buffers;
        private int _index;
    }

    private readonly Bucket[] _buckets;     //bucket数组
}

3. Why there are 50 buffers in every bucket[]

This question is easy to answer. The maxArraysPerBucket=50 setting was made during initialization. Of course, you can also customize it. For details, refer to the following code:

internal sealed class ConfigurableArrayPool<T> : ArrayPool<T>
{
    internal ConfigurableArrayPool() : this(1048576, 50)
    {
    }

    internal ConfigurableArrayPool(int maxArrayLength, int maxArraysPerBucket)
    {
        int num = Utilities.SelectBucketIndex(maxArrayLength);
        Bucket[] array = new Bucket[num + 1];
        for (int i = 0; i < array.Length; i++)
        {
            array[i] = new Bucket(Utilities.GetMaxSizeForBucket(i), maxArraysPerBucket, id);
        }
        _buckets = array;
    }
}

4. Why buffer[].length in the bucket is 16, 32, 64...

Frame default assumption made, the first in the bucket buffer[].length=16 , the subsequent bucket buffer[].length are accumulated x2 for the code is GetMaxSizeForBucket() method, refer to the following:

internal ConfigurableArrayPool(int maxArrayLength, int maxArraysPerBucket)
{
    Bucket[] array = new Bucket[num + 1];
    for (int i = 0; i < array.Length; i++)
    {
        array[i] = new Bucket(Utilities.GetMaxSizeForBucket(i), maxArraysPerBucket, id);
    }
}

internal static int GetMaxSizeForBucket(int binIndex)
{
    return 16 << binIndex;
}

5. How many buckets are there during initialization?

In fact, in the picture above, I didn't show how many buckets there are. How many are there? 😓😓😓, after I read the source code, this algorithm is quite interesting.

Let me talk about the results first. There are 17 buckets by default. You will definitely be curious how to calculate it? Let me talk about the next two variables:

• maxArrayLength=1048576 = 2 to the 20th power
• buffer.length = 16 = 2 to the 4th power

The final algorithm is to take the difference of the power: bucket[].length= 20 - 4 + 1 = 17 , in other words buffer[].length=1048576 under the last bucket, please refer to the SelectBucketIndex() method for the detailed code.

internal sealed class ConfigurableArrayPool<T> : ArrayPool<T>
{
    internal ConfigurableArrayPool(): this(1048576, 50)
    { }

    internal ConfigurableArrayPool(int maxArrayLength, int maxArraysPerBucket)
    {
        int num = Utilities.SelectBucketIndex(maxArrayLength);
        Bucket[] array = new Bucket[num + 1];
        for (int i = 0; i < array.Length; i++)
        {
            array[i] = new Bucket(Utilities.GetMaxSizeForBucket(i), maxArraysPerBucket, id);
        }
        _buckets = array;
    }

    internal static int SelectBucketIndex(int bufferSize)
    {
        return BitOperations.Log2((uint)(bufferSize - 1) | 0xFu) - 3;
    }
}

At this point, I believe you have understood the idea of ArrayPool's pooling architecture. Next, let's look at how to apply for and return buffer[].

Three: How to apply and return

Since the buffer[] is granular, it should be borrowed and returned. The code is Rent() and Return() methods. In order to facilitate understanding, the code speaks:

    class Program
    {
        static void Main(string[] args)
        {
            var arrayPool = ArrayPool<int>.Create();

            var bytes = arrayPool.Rent(10);

            for (int i = 0; i < bytes.Length; i++) bytes[i] = 10;

            arrayPool.Return(bytes);

            Console.ReadLine();
        }
    }




Once you have the code and diagram, let's touch the process a little bit.

1. byte[10] size 06135843dc3342 from ArrayPool. In order to save memory, we do not stock up first, temporarily generate an byte[].size=16 . The simplified code is as follows, refer to if (flag) :

    internal T[] Rent()
    {
        T[][] buffers = _buffers;
        T[] array = null;
        bool lockTaken = false;
        bool flag = false;
        try
        {
            if (_index < buffers.Length)
            {
                array = buffers[_index];
                buffers[_index++] = null;
                flag = array == null;
            }
        }
        if (flag)
        {
            array = new T[_bufferLength];
        }
        return array;
    }

There is a pit here, that is, you thought you borrowed byte[10] , but the reality is byte[16] , so pay attention to it here.

2. When using ArrayPool.Return to return byte[16] , it is obvious that it falls on the first buffer[] of the first bucket. Refer to the following simplified code:

    internal void Return(T[] array)
    {
        if (_index != 0)
        {
            _buffers[--_index] = array;
        }
    }

There is also a notable pit, that is byte[16] that is still returned will not be cleared by default. It can be seen from the above code. If you want to clean up, you need to specify clearArray=true in the Return method. Refer to the following Code:

    public override void Return(T[] array, bool clearArray = false)
    {
        int num = Utilities.SelectBucketIndex(array.Length);

        if (num < _buckets.Length)
        {
            if (clearArray)
            {
                Array.Clear(array, 0, array.Length);
            }
            _buckets[num].Return(array);
        }
    }

Four: Summary

Studying the pooling architecture ideas can still provide some inspiration for normal project development. Secondly, for those byte[] is used for one-time use, pooling is a very good method, which is also proposed after my friend dump analysis. An optimization idea.

More high-quality dry goods: see my GitHub: dotnetfly