Deep Demystifying the Asynchronous Task Capability of Alibaba Cloud Function Computing

Author: gradually

In the previous article , "Task Status and Life Cycle Management of Decryption Function Computing Asynchronous Task Capability" , we introduced the status management of the task system, and introduced how users should query the task status information in real time according to their needs. and so on. In this article, we will go further into function computing asynchronous tasks, introduce the scheduling scheme of asynchronous tasks and the functions supported by the system in terms of observability.

task scheduling

Task scheduling refers to the related operations that the system puts different tasks into appropriate computing resources to perform according to the current load situation. A perfect scheduling system often needs to balance the two requirements of isolation between tasks with different characteristics and optimal efficiency. Function Compute asynchronous tasks adopt an independent queue model and an automatic load balancing strategy, and have the ability to perform multi-tenancy isolation without affecting processing performance.

Serverless Task task scheduling model

When a user submits a task, the system will convert the task into a message and put it into the internal queue by asynchronously delivering it. The processing flow of a message is shown in the following figure:

figure 1

The multi-tenancy isolation and message backlog control of the entire system in task scheduling mainly depend on the scheduler's consumption and control of the queue. We will divide an account-level queue for each user in advance, and the asynchronous calls (including task calls) of all functions of the user will share the queue.

Such a model structure will ensure that each user's asynchronous execution request (including task invocation) will not be affected by the invocation of other users. However, in some large-scale application scenarios, such as a user has many functions, and the call volume of each function is large, all asynchronous messages share a queue, which will inevitably cause mutual influence between calls. Some long-tail calls may consume too much queue resources and cause starvation in the execution of other functions.

To prevent this from affecting the execution of important functions, Function Compute provides a finer-grained queue - a function-level queue. You can set a separate queue for each different function to ensure that the consumption of high-priority functions will not be affected by the execution of other functions under the same account. The relationship between the queues is shown in the following figure:

figure 2

Typical application scenarios

Suppose a user A has 2 different task functions. Due to the limitation of downstream services, one of the tasks A needs to be executed one message at a time; the other task B is a large concurrent task and hopes to be executed as soon as possible. In the default mode, tasks A and B share the same user queue; at this time, the following scenario occurs: Due to the concurrency limit of task A, the function computing side will control the dequeue rate of the entire task queue. This causes the task of task B to be delayed from being dequeued. When task A is executed, task B gets the opportunity to be dequeued. At this time, the degree of concurrency increases, and the message of task B preempts the resource pool for execution. Task A becomes difficult to dequeue again, and it cannot be started once for a long time. implement. The result is that both A and B are seriously disturbed by the other party's business.

When the queue is adjusted, tasks A and B have exclusive queues respectively. In this case, the consumption speeds of tasks A and B are not affected by each other, and both can achieve their own demands.

At present, Serverless Task provides a task backlog dashboard. You can obtain the number of tasks that have been backlogged on the task interface, and comprehensively analyze whether you need to open the exclusive queue of the function.

Serverless Task Task Queue Load Balancing Model

The above describes how to avoid the "Noisy Neighbour" problem with function-level queues. However, in some scenarios, if the concurrency of the task is too large, even if a single queue is divided for the task, it will lead to a backlog of tasks. The solution to this problem requires the introduction of a serverless task load balancing strategy.

The task processing module of Function Compute has the concept of Partition. Each user belongs to a Partition by default, and the Scheduler responsible for the Partition will monitor the task queue corresponding to the user. When there is a serious backlog, we will allocate multiple Partitions to users according to the load situation, and assign them to different Schedulers for consumption to improve the overall consumption speed of the task.

image 3

It can be seen that Alibaba Cloud Function Compute has multi-tenancy and isolation capabilities by default in task queue management, which can be applied to most scenarios. For some heavy-load, long-execution, and large-concurrency scenarios, Function Compute also supports horizontal expansion to speed up consumption. In terms of task isolation, Function Compute supports separate isolation of functions with different priorities to avoid the Noisy Neighbour problem.

observability

The observability of the mission is one of the essential capabilities of the mission system. Strong observability will help business parties reduce the amount of extra work required at various stages of task operation.

1. Development stage: The online debugging ability of tasks and the debugging ability of running results will directly affect the progress of business online;
2. Routine business operation stage: various monitoring, traffic statistics and runtime logs will help users quickly understand business development and changes, and quickly locate & deal with faults;
3. Staged audit: The storage and retention of historical records of tasks will provide users with good traceability, and follow-up business planning can be carried out based on historical information.

ServerlessTask observability support - development test phase

The main requirement in the development phase of the business is to quickly debug and locate problems. In support of this stage, ServerlessTask provides the ability to log in to instances and real-time logs. After the code is developed and uploaded, the process of testing-debug-modifying the code-testing again can be completed in the console, which greatly improves the efficiency of research and development. If there is a need for performance debugging, third-party Binary debugging (such as FFmpeg debugging in the field of audio and video processing), etc., it can be done with the help of the login instance function. The operation flow is as follows:

Select the task for which you want to log in to the instance, and click the instance link.

You will enter the instance monitoring page, click the login instance function in the upper right corner to log in to the corresponding instance.

ServerlessTask observability support - running phase after business goes live

After the service goes online, it is often prone to failures due to insufficient capacity estimation, which prevents the downstream system from being able to bear the pressure. Therefore, ServerlessTask provides runtime metrics, that is, the number of task submissions, completions, and execution over a period of time. Users can quickly understand the current business load based on this indicator graph. When the downstream consumption of user tasks is slow, it may cause a backlog of tasks. This situation is also easily reflected in the indicator graph, so that a corresponding response can be made quickly. The relevant indicators currently provided by ServerlessTask are as follows:

The task monitoring dashboard provides the following task monitoring data:

In terms of quickly locating problems, Function Compute supports real-time viewing of function logs and instance metrics. You can go to the task list page to find the actual failed task, and go to the log page and instance page to locate the problem:

ServerlessTask observability support - staged auditing

When an online task runs for a period of time, it is often necessary to perform a series of periodic audits, such as the total number of tasks executed in the previous week, the number of tasks that failed to be executed, and the time of failed execution. Currently, in addition to the console, Function Compute provides rich API capabilities for task auditing. It mainly includes the following abilities:

1. Filter according to the state, and only query the execution of a certain state;
2. Filter according to the trigger time, such as querying tasks initiated within a certain period of time in the past;
3. Query by task name. If your task has TraceIDs upstream and downstream of the business, you can specify a meaningful task ID when triggering the task. Subsequent range queries can be performed based on the ID prefix;

The above filtering methods can be combined to achieve more convenient needs. The filter conditions supported by the console are shown in the following figure:

For more parameters, please refer to:

https://help.aliyun.com/document_detail/256588.html

ServerlessTask observability support - dead letter queue and business compensation

In the message field, there is a very important concept - dead letter queue. When some messages cannot be consumed, these messages often need to be stored in a place for subsequent human intervention and processing to avoid business losses due to unprocessing. Serverless Task also supports such a type of functionality. You can set a target function for a Serverless Task; when a task fails to execute, Function Compute supports automatically pushing the context information of the failed execution to message services such as message queues for subsequent processing. If your processing logic supports automation, Function Compute also supports pushing context information of failed tasks back to Function Compute to execute a piece of your custom business logic to achieve business compensation.

You can configure success and failure goals on the asynchronous call configuration page.

For more configuration content, please refer to:

https://help.aliyun.com/document_detail/415755.htm .

To sum up, the observability provided by Serverless Task can effectively support the monitoring requirements of the entire task life cycle. All console capabilities can be custom developed using open APIs to meet more needs. In addition to compensating for task failure, the target function of Serverless Task can also be used as a data source in Event-Driven mode to automatically deliver processed events to downstream services.

Past articles

1. The asynchronous task processing system, how to solve the problem of long time-consuming and high concurrency ?

2. Introduction to Function Compute Asynchronous Task Capability - Task Triggering and Deduplication

3. The ability of decryption function to calculate asynchronous tasks - task status and life cycle management

---

Recommended recent popular activities of Servereless

The serverless function computing evaluation essay activity is coming . From June 28th to July 31st, participate in product evaluation and publish articles, you will have the opportunity to get Beats headphones, mechanical keyboard, thousand yuan Tmall supermarket card, Youku membership season card, many good things Gifts are waiting for you to win! The submission direction can refer to (but not limited to):

• Your experience and suggestions on the capabilities of Function Compute FC products can help other users choose serverless services.
• Scenario evaluation of using Function Compute FC to create applications, such as building cloud blogs based on Function Compute FC, building elastic and highly available serverless web applications, and building elastic and highly available video processing systems based on serverless architecture.

For more event details, please scan the QR code to visit the official website:

https://developer.aliyun.com/topic/serverless2022

Click here to contribute!