Commons-pool2 pooling technology exploration

I. Introduction

We often come into contact with various pooling technologies or concepts, including object pools, connection pools, thread pools, etc. The biggest advantage of pooling technology is to realize the reuse of objects, especially the creation and use of large objects or valuable resources (HTTP connection Objects, MySQL connection objects), etc., can greatly save system overhead, and it is also crucial to improve the overall performance of the system.

Under concurrent requests, if you need to create/close MySQL connections for hundreds of query operations at the same time or create a processing thread for each HTTP request or create a parsing object for each image or XML parsing without using pooling technology , Will bring great load challenges to the system.

This article mainly analyzes the implementation of commons-pool2 pooling technology, and hopes that through this article, readers can have a more comprehensive understanding of the implementation principle of commons-pool2.

2. Analysis of commons-pool2 pooling technology

More and more frameworks are choosing to use apache commons-pool2 for pool management, such as jedis-cluster, the logic of commons-pool2 work is shown in the figure below:

2.1 Core three elements

2.1.1 ObjectPool

The object pool is responsible for the life cycle management of objects and provides the function of counting active and idle objects in the object pool.

2.1.2 PooledObjectFactory

The object factory class is responsible for the creation and initialization of specific objects, the destruction and verification of object states. The commons-pool2 framework itself provides a default abstract implementation BasePooledObjectFactory. The business side only needs to inherit this class when using it, and then implement the warp and create methods.

2.1.3 PooledObject

The pooled object is a wrapper class that needs to be placed in the ObjectPool object. Added some additional information, such as status information, creation time, activation time, etc. commons-pool2 provides two implementations, DefaultPooledObject and PoolSoftedObject. Among them, PoolSoftedObject inherits from DefaultPooledObject. The difference is that the soft reference of the object is realized by using SoftReference. The use of obtaining objects is also obtained through SoftReference.

2.2 Logic Analysis of Object Pool

2.2.1 Description of object pool interface

1) When we use commons-pool2, the operation of the application to acquire or release objects is based on the object pool. The core interfaces of the object pool mainly include the following:

/**
*向对象池中增加对象实例
*/
void addObject() throws Exception, IllegalStateException,
      UnsupportedOperationException;
/**
* 从对象池中获取对象
*/
T borrowObject() throws Exception, NoSuchElementException,
      IllegalStateException;
/**
* 失效非法的对象
*/
void invalidateObject(T obj) throws Exception;
/**
* 释放对象至对象池
*/
void returnObject(T obj) throws Exception;

In addition to the interface itself, the object pool also supports setting the maximum number of objects, retention time, and so on. The core parameter items of the object pool include maxTotal, maxIdle, minIdle, maxWaitMillis, testOnBorrow, etc.

2.2.2 Object creation decoupling

The object factory is the core link used to generate objects in the commons-pool2 framework. The business side needs to implement the corresponding object factory implementation class by itself in the process of use. Through the factory pattern, the generation of object pools and objects and the details of the implementation process are realized. Decoupling, each object pool should have member variables of the object factory, so that the object pool itself and the generation logic of the object are decoupled.

We can further verify our ideas through the code:

public GenericObjectPool(final PooledObjectFactory<T> factory) {
      this(factory, new GenericObjectPoolConfig<T>());
  }
  
  public GenericObjectPool(final PooledObjectFactory<T> factory,
                            final GenericObjectPoolConfig<T> config) {
​
      super(config, ONAME_BASE, config.getJmxNamePrefix());
​
      if (factory == null) {
          jmxUnregister(); // tidy up
          throw new IllegalArgumentException("factory may not be null");
      }
      this.factory = factory;
​
      idleObjects = new LinkedBlockingDeque<>(config.getFairness());
      setConfig(config);
  }
​
  public GenericObjectPool(final PooledObjectFactory<T> factory,
                            final GenericObjectPoolConfig<T> config, final AbandonedConfig abandonedConfig) {
      this(factory, config);
      setAbandonedConfig(abandonedConfig);
  }

It can be seen that the construction method of the object pool depends on the object construction factory PooledObjectFactory. When the object is generated, it is generated based on the parameters defined in the object pool and the object construction factory.

/**
* 向对象池中增加对象，一般在预加载的时候会使用该功能
*/
@Override
public void addObject() throws Exception {
  assertOpen();
  if (factory == null) {
      throw new IllegalStateException(
              "Cannot add objects without a factory.");
  }
  final PooledObject<T> p = create();
  addIdleObject(p);
}

The create() method is based on the objects generated by the object factory, and continues to follow up the code to confirm the logic;

final PooledObject<T> p;
try {
  p = factory.makeObject();
  if (getTestOnCreate() && !factory.validateObject(p)) {
      createCount.decrementAndGet();
      return null;
  }
} catch (final Throwable e) {
  createCount.decrementAndGet();
  throw e;
} finally {
  synchronized (makeObjectCountLock) {
      makeObjectCount--;
      makeObjectCountLock.notifyAll();
  }
}

The operation of factory.makeObject() is confirmed here, and the above speculation is also confirmed, and the corresponding object is generated based on the object factory.

In order to better realize the use of objects in the object pool and track the status of objects, the commons-pool2 framework uses the concept of pooled objects PooledObject. PooledObject itself is a generic class and provides a getObject() method to obtain actual objects. .

2.2.3 Analysis of Object Pool Source Code

After the above analysis, we know that the object pool carries the life cycle management of the object, including the logic of controlling the number of objects in the entire object pool. Next, we use the source code of GenericObjectPool to analyze how it is implemented.

In the object pool, a double-ended queue LinkedBlockingDeque is used to store objects. LinkedBlockingDeque supports FIFO and FILO strategies for columns, and realizes the coordination of queue operations based on AQS.

LinkedBlockingDeque provides the operations of inserting and removing elements at the end of the queue and the head of the queue, and the related operations are performed to add the lock operation of the reentrant lock. The notFull and notEmpty state variables are set in the queue of the reentrant lock. When the queue is operated on the elements Will trigger corresponding operations such as await and notify.

/**
* 第一个节点
* Invariant: (first == null && last == null) ||
*           (first.prev == null && first.item != null)
*/
private transient Node<E> first; // @GuardedBy("lock")
​
/**
* 最后一个节点
* Invariant: (first == null && last == null) ||
*           (last.next == null && last.item != null)
*/
private transient Node<E> last; // @GuardedBy("lock")
​
/** 当前队列长度 */
private transient int count; // @GuardedBy("lock")
​
/** 队列最大容量 */
private final int capacity;
​
/** 主锁 */
private final InterruptibleReentrantLock lock;
​
/** 队列是否为空状态锁 */
private final Condition notEmpty;
​
/** 队列是否满状态锁 */
private final Condition notFull;

core point of the

1. All moving in, moving out, and initializing construction elements in the queue are all locked based on the main lock.

2. The offer and pull of the queue support the setting of timeout parameters, which are mainly used for coordinated operation through two states. For example, during the offer operation, if the operation is unsuccessful, wait based on the notFull state object.

public boolean offerFirst(final E e, final long timeout, final TimeUnit unit)
  throws InterruptedException {
  Objects.requireNonNull(e, "e");
  long nanos = unit.toNanos(timeout);
  lock.lockInterruptibly();
  try {
      while (!linkFirst(e)) {
          if (nanos <= 0) {
              return false;
          }
          nanos = notFull.awaitNanos(nanos);
      }
      return true;
  } finally {
      lock.unlock();
  }
}

For example, during the pull operation, if the operation is unsuccessful, wait for notEmpty.

public E takeFirst() throws InterruptedException {
  lock.lock();
  try {
      E x;
      while ( (x = unlinkFirst()) == null) {
          notEmpty.await();
      }
      return x;
  } finally {
      lock.unlock();
  }
}

On the contrary, when the operation is successful, the wake-up operation is performed, as shown below:

private boolean linkLast(final E e) {
  // assert lock.isHeldByCurrentThread();
  if (count >= capacity) {
      return false;
  }
  final Node<E> l = last;
  final Node<E> x = new Node<>(e, l, null);
  last = x;
  if (first == null) {
      first = x;
  } else {
      l.next = x;
  }
  ++count;
  notEmpty.signal();
  return true;
}

2.3 Core business process

2.3.1 Pooled object state change

The figure above is the state machine diagram of PooledObject, blue indicates state, red indicates methods related to ObjectPool. The state of PooledObject is: IDLE, ALLOCATED, RETURNING, ABANDONED, INVALID, EVICTION, EVICTION\_RETURN\_TO_HEAD

All states are defined in the PooledObjectState class, and some of them are temporarily unused, so I won’t repeat them here.

2.3.2 Object pool browObject process

The first step is to determine whether to call the removeAbandoned method for label deletion according to the configuration.

The second step is to try to obtain or create an object. The source code process is as follows:

//1、尝试从双端队列中获取对象，pollFirst方法是非阻塞方法
p = idleObjects.pollFirst();
if (p == null) {
    p = create();
    if (p != null) {
        create = true;
    }
}
if (blockWhenExhausted) {
    if (p == null) {
        if (borrowMaxWaitMillis < 0) {
            //2、没有设置最大阻塞等待时间，则无限等待
            p = idleObjects.takeFirst();
        } else {
            //3、设置最大等待时间了，则阻塞等待指定的时间
            p = idleObjects.pollFirst(borrowMaxWaitMillis,
                    TimeUnit.MILLISECONDS);
        }
    }
}

The schematic diagram is as follows:

The third step is to call allocate to change the state to the ALLOCATED state.

The fourth step is to call the factory's activateObject to initialize the object. If an error occurs, call the destroy method to destroy the object, such as the six steps in the source code.

The fifth step is to call the validateObject of TestFactory to analyze the availability of the object based on the TestOnBorrow configuration. If it is not available, call the destroy method to destroy the object. The source code process of steps 3-7 is as follows:

//修改对象状态
if (!p.allocate()) {
    p = null;
}
if (p != null) {
    try {
        //初始化对象
        factory.activateObject(p);
    } catch (final Exception e) {
        try {
            destroy(p, DestroyMode.NORMAL);
        } catch (final Exception e1) {
        }
 
}
    if (p != null && getTestOnBorrow()) {
        boolean validate = false;
        Throwable validationThrowable = null;
        try {
            //验证对象的可用性状态
            validate = factory.validateObject(p);
        } catch (final Throwable t) {
            PoolUtils.checkRethrow(t);
            validationThrowable = t;
        }
        //对象不可用，验证失败，则进行destroy
        if (!validate) {
            try {
                destroy(p, DestroyMode.NORMAL);
               destroyedByBorrowValidationCount.incrementAndGet();
            } catch (final Exception e) {
                // Ignore - validation failure is more important
            }
 
        }
    }
}

2.3.3 The process execution logic of the object pool returnObject

The first step is to call the markReturningState method to change the state to RETURNING.

The second step is to call the validateObject method of PooledObjectFactory based on the testOnReturn configuration to check the availability. If the check fails, call destroy to consume the object, and then make sure to call idle to ensure that there is an IDLE state object available in the pool, if not, call the create method to create a new object.

The third step is to call the passivateObject method of PooledObjectFactory to perform de-initialization.

The fourth step is to call deallocate to change the status to IDLE.

The fifth step is to detect whether the maximum number of free objects has been exceeded, and if it exceeds, destroy the current objects.

The sixth step is to place the object at the beginning or end of the queue according to the LIFO (last in, first out) configuration.

2.4 Expand and think

2.4.1 Another implementation of LinkedBlockingDeque

The above analysis shows that commons-pool2 uses double-ended queues and conditions in java to realize the management of objects in the queue and the coordination between object acquisition and release operations by different threads. Is there any other solution that can achieve similar effects? ? The answer is yes.

Using double-ended queues to operate, in fact, we want to isolate idle objects from active objects. In essence, we use two queues to store the idle queue and the currently active objects separately, and then use an object lock uniformly, which can also achieve the same The general idea of the goal is as follows:

1. The double-ended queue is changed to two one-way queues for storing idle and active objects respectively. The synchronization and coordination between the queues can be completed by wait and notify of the object lock.

public  class PoolState {
 
protected final List<PooledObject> idleObjects = new ArrayList<>();
protected final List<PooledObject> activeObjects = new ArrayList<>();
 
 
//...
 
}

2. When acquiring objects, the original LIFO or FIFO of the deque becomes the acquisition of objects from the idle queue idleObjects, and then after the acquisition is successful and the object status is legal, the object is added to the active object collection activeObjects, if the object is acquired Need to wait, the PoolState object lock should enter the waiting state through the wait operation.

3. When releasing the object, first delete the element from the active object collection activeObjects, and after the deletion is complete, add the object to the idle object collection idleObjects. It should be noted that the state of the object needs to be verified during the release of the object. . When the object status is illegal, the object should be destroyed and should not be added to idleObjects. After the release is successful, PoolState wakes up the waiting acquisition operation through notify or notifyAll.

4. In order to ensure the thread safety of the operation on the active queue and the idle queue, the acquisition and release of the object need to be locked, which is consistent with the commons2-pool.

2.4.2 Self-protection mechanism of object pool

When we use commons-pool2 to obtain objects, we will block from the deque waiting to obtain elements (or create new objects), but if it is an application exception, when returnObject or invalidObject has not been called, it may be There will be objects in the object pool rising all the time, and when you call borrowObject after reaching the set line, you will be waiting or waiting for a timeout and unable to obtain the object.

In order to avoid the above analysis problems, commons-pool2 provides two self-protection mechanisms:

2.4.2.1 Threshold-based detection

When obtaining objects from the object pool, it will verify the proportion of the number of active objects and free objects in the current object pool. When there are very few free exclusives and very many active objects, the recovery of free objects will be triggered. Specific verification rules This is: if there are less than 2 idle objects in the current object pool or when the number of active objects is greater than the maximum number of objects-3, start leak cleanup when borrowing objects. Turn it on by setting AbandonedConfig.setRemoveAbandonedOnBorrow to true.

//根据配置确定是否要为标签删除调用removeAbandoned方法
final AbandonedConfig ac = this.abandonedConfig;
if (ac != null && ac.getRemoveAbandonedOnBorrow() && (getNumIdle() < 2) && (getNumActive() > getMaxTotal() - 3) ) {
    removeAbandoned(ac);
}

2.4.2.2 Asynchronous scheduling thread detection

After AbandonedConfig.setRemoveAbandonedOnMaintenance is set to true, the leaked objects will be cleaned up when the maintenance task is running. Set the time interval for the maintenance task execution by setting setTimeBetweenEvictionRunsMillis.

detection and recovery:

The startEvictor method is called at the end of the internal logic of the construction method. The function of this method is to start the collector to monitor the recovery of idle objects after constructing the object pool. startEvictor is defined in the BaseGenericObjectPool (abstract) class, the parent class of GenericObjectPool. Let's take a look at the source code of this method first.

The following setting parameters will be executed in the constructor;

public final void setTimeBetweenEvictionRunsMillis(
      final long timeBetweenEvictionRunsMillis) {
  this.timeBetweenEvictionRunsMillis = timeBetweenEvictionRunsMillis;
  startEvictor(timeBetweenEvictionRunsMillis);
}

The timed cleanup task will be started only if the timeBetweenEvictionRunsMillis parameter is set.

final void startEvictor(final long delay) {
  synchronized (evictionLock) {
      EvictionTimer.cancel(evictor, evictorShutdownTimeoutMillis, TimeUnit.MILLISECONDS);
      evictor = null;
      evictionIterator = null;
      //如果delay<=0则不会开启定时清理任务
      if (delay > 0) {
          evictor = new Evictor();
          EvictionTimer.schedule(evictor, delay, delay);
      }
  }
}

Continuing to follow up the code, we can find that the implementation logic of the cleanup method set in the scheduler is actually defined in the object pool, which is implemented by GenericObjectPool or GenericKeyedObjectPool. Next, we continue to explore how the object pool performs object recycling.

a), core parameters:

minEvictableIdleTimeMillis: Specify the maximum retention time of idle objects, and those that exceed this time will be recycled. If it is not configured, it will not be recycled.

softMinEvictableIdleTimeMillis: A millisecond value used to specify that when the number of idle objects exceeds the minIdle setting, and an idle object exceeds this idle time, it can be recycled.

minIdle: The minimum amount of space objects to be reserved in the object pool.

b), recycling logic

And an object recycling policy interface EvictionPolicy, it can be expected that the recycling of the object pool will be associated with the above-mentioned parameter items and the interface EvictionPolicy. If you continue to follow up the code, you will find the following content. You can see that when it is judged that the object pool can be recycled, Call destroy directly for recycling.

boolean evict;
try {
  evict = evictionPolicy.evict(evictionConfig, underTest,
  idleObjects.size());
} catch (final Throwable t) {
  // Slightly convoluted as SwallowedExceptionListener
  // uses Exception rather than Throwable
    PoolUtils.checkRethrow(t);
    swallowException(new Exception(t));
    // Don't evict on error conditions
    evict = false;
}
if (evict) {
    // 如果可以被回收则直接调用destroy进行回收
    destroy(underTest);
    destroyedByEvictorCount.incrementAndGet();
}

In order to improve the efficiency of recycling, when the recycling strategy determines that the state of the object is not evict, further state determination and processing will be performed. The specific logic is as follows:

1. Try to activate the object. If the activation fails, the object is considered no longer alive, and destroy it directly by calling destroy.

2. In the case of successful activation of the object, the state of the verification object will be obtained through the validateObject method. If the verification fails, the object is not available and needs to be destroyed.

boolean active = false;
try {
  // 调用activateObject激活该空闲对象，本质上不是为了激活，
  // 而是通过这个方法可以判定是否还存活，这一步里面可能会有一些资源的开辟行为。
  factory.activateObject(underTest);
  active = true;
} catch (final Exception e) {
  // 如果激活的时候，发生了异常，就说明该空闲对象已经失联了。
  // 调用destroy方法销毁underTest
  destroy(underTest);
  destroyedByEvictorCount.incrementAndGet();
}
if (active) {
  // 再通过进行validateObject校验有效性
  if (!factory.validateObject(underTest)) {
      // 如果校验失败，说明对象已经不可用了
      destroy(underTest);
      destroyedByEvictorCount.incrementAndGet();
  } else {
      try {
          /*
            *因为校验还激活了空闲对象，分配了额外的资源，那么就通过passivateObject把在activateObject中开辟的资源释放掉。
          */
          factory.passivateObject(underTest);
      } catch (final Exception e) {
          // 如果passivateObject失败，也可以说明underTest这个空闲对象不可用了
          destroy(underTest);
          destroyedByEvictorCount.incrementAndGet();
      }
  }
}

Three, write at the end

Connection pooling can bring some convenience to program developers. In the preface, we analyzed the benefits and necessity of using pooling technology, but we can also see that the commons-pool2 framework locks the creation and acquisition of objects. The operation of this will affect the performance of the application to a certain extent in the concurrent scenario. Secondly, the number of objects in the object pool of the pooled object also needs to be set reasonably, otherwise it will be difficult to achieve the purpose of using the object pool. This also brings us certain challenges.

Author: vivo Internet server team -Huang Xiaoqun