Python ThreadPoolExecutor limits the size of _work_queue

Use python's futures.ThreadPoolExecutor is, if you call submit to submit the task
ThreadPoolExecutor will be executed

self._work_queue.put(w)

in

self._work_queue = queue.SimpleQueue()

SimpleQueue does not limit the size of the queue. If too many tasks are submitted and the processing is not timely, it will occupy too much memory

Can be replaced to the implementation of queue.Queue(maxsize=maxsize)

class ThreadPoolExecutorWithQueueSizeLimit(futures.ThreadPoolExecutor):
    def __init__(self, maxsize=50, *args, **kwargs):
        super(ThreadPoolExecutorWithQueueSizeLimit, self).__init__(*args, **kwargs)
        self._work_queue = queue.Queue(maxsize=maxsize)

links:

• https://stackoverflow.com/questions/48263704/threadpoolexecutor-how-to-limit-the-queue-maxsize