Serverless engineering practice | Serverless application optimization and debugging tips

Author｜Liu Yu

Preface: This article will take Alibaba Cloud Function Computing as an example to provide various application optimization and debugging solutions such as online debugging and local debugging.

Serverless application debugging tips

In the application development process, or when the application development is completed, the execution result does not meet the expectations, we need to carry out certain debugging work. But under the serverless architecture, debugging is often greatly restricted by the environment, and the developed applications can run healthily and in line with expectations locally, but some unpredictable problems occur on the FaaS platform. Moreover, in some special environments, there is no way to simulate the online environment locally, which makes it difficult to develop and debug projects.

The debugging of serverless applications has always been criticized, but various cloud vendors have not given up on in-depth exploration in the debugging direction. Taking Alibaba Cloud Function Computing as an example, it provides various debugging solutions such as online debugging and local debugging.

Online debugging

1. Simple debugging

The so-called simple debugging is to debug on the console. Take Alibaba Cloud Function Computing as an example, it can perform basic debugging through the "Execute" button on the console, as shown in the figure.

Simple function online debugging page

When necessary, we can also simulate some events by setting Event, as shown in the figure.

Simulate events by setting Event

The advantage of online debugging is that you can use some online environments for code testing. When the online environment has resources such as VPC, it is difficult to debug in the local environment. For example, the database needs to be accessed through the VPC, or there are business logics such as object storage triggers.

2. Breakpoint debugging

In addition to simple debugging, some cloud vendors also support breakpoint debugging, such as remote debugging of Alibaba Cloud Function Computing, and remote debugging of Tencent Cloud Cloud Functions. Take the remote debugging of Alibaba Cloud Function Computing as an example, which can perform online debugging of functions through the console. After the function is created, the user can select remote debugging and click the "Open debugging" button, as shown in the figure.

Function online breakpoint debugging page (1)

After starting the debugging, wait a while, the system will enter the remote debugging interface, as shown in the figure.

Function online breakpoint debugging page (2)

At this point, you can perform some breakpoint debugging, as shown in the figure.

Function online breakpoint debugging page (3)

Local debugging

1. Command line tools

For now, most FaaS platforms will provide users with relatively complete command-line tools, including AWS's SAM CLI, Alibaba Cloud's Funcraft, and some open source projects such as Serverless Framework, Serverless Devs and other support for multi-cloud vendors. The method of code debugging through command line tools is very simple. Take Serverless Devs as an example to debug Alibaba Cloud function computing locally.

First, make sure that you have a function calculation project locally, as shown in the figure.

Local function calculation project

Then execute the debugging instructions under the project, such as debugging in Docker, as shown in the figure.

Command line tool to debug function calculation

2. Editor plugin

Take the VScode plug-in as an example. After downloading the VSCode plug-in for Alibaba Cloud Function Computing and configuring the account information, you can create a new function locally, and you can perform breakpoint debugging after clicking, as shown in the figure.

VSCode plug-in debugging function calculation

When the function debugging is completed, perform operations such as deployment.

Other debugging options

1. Local debugging of Web framework

To develop a traditional web framework on the Alibaba Cloud FaaS platform, take the Bottle framework written in Python as an example, you can add the following code:

app = bottle.default_app()
并且对run方法进行条件限制 (if __name__ == '__main__')：
if __name__ == '__main__':
    bottle.run(host='localhost', port=8080, debug=True)
例如：
# index.py
import bottle

@bottle.route('/hello/<name>')
def index(name):
    return "Hello world"

app = bottle.default_app()

if __name__ == '__main__':
    bottle.run(host='localhost', port=8080, debug=True)

When deploying an application online, you only need to fill in ndex.app at the entry method to achieve smooth deployment.

2. Local analog event debugging

For non-web frameworks, we can build a method locally, for example, to debug the object storage trigger:

import json
def handler(event, context):
    print(event)
def test():
    event = {
        "events": [
            {
                "eventName": "ObjectCreated:PutObject",
                "eventSource": "acs:oss",
                "eventTime": "2017-04-21T12:46:37.000Z",
                "eventVersion": "1.0",
                "oss": {
                    "bucket": {
                        "arn": "acs:oss:cn-shanghai:123456789:bucketname",
                        "name": "testbucket",
                        "ownerIdentity": "123456789",
                        "virtualBucket": ""
                    },
                    "object": {
                        "deltaSize": 122539,
                        "eTag": "688A7BF4F233DC9C88A80BF985AB7329",
                        "key": "image/a.jpg",
                        "size": 122539
                    },
                    "ossSchemaVersion": "1.0",
                    "ruleId": "9adac8e253828f4f7c0466d941fa3db81161****"
                },
                "region": "cn-shanghai",
                "requestParameters": {
                    "sourceIPAddress": "140.205.***.***"
                },
                "responseElements": {
                    "requestId": "58F9FF2D3DF792092E12044C"
                },
                "userIdentity": {
                    "principalId": "123456789"
                }
            }
        ]
    }
    handler(json.dumps(event), None)
if __name__ == "__main__":
    print(test())

In this way, by constructing an event object, the simulated event trigger can be realized.

Serverless application optimization

Resource assessment is still important

Although the serverless architecture is paid for by volume, it does not mean that it is necessarily lower than the traditional server rental cost. If the evaluation of your own projects is inaccurate, and some indicators are set unreasonably, the costs incurred by the serverless architecture may be huge.

In general, the fees of the FaaS platform are directly related to three indicators, namely the configured function specifications (such as memory specifications, etc.), the time consumed by the program, and the traffic costs generated. Under normal circumstances, the time consumed by the program may be related to the memory specifications and the business logic handled by the program itself. The traffic cost is related to the size of the data packet that the program itself interacts with the client. Therefore, among the three common indicators, the memory specification may cause a relatively large deviation in billing due to non-standard configuration. Take Alibaba Cloud Function Computing as an example. Suppose there is a Hello World program that is executed 10,000 times a day. The costs (excluding network costs) incurred by different specifications of memory are shown in the table.

It can be seen from the table that when the program can be executed normally in the 128MB memory, if the memory specification is set to 3072MB by mistake, the monthly cost may skyrocket 25 times! Therefore, before the serverless application is launched, resources must be evaluated in order to further reduce costs with a more reasonable configuration.

Reasonable code package specifications

Each cloud vendor's FaaS platform has restrictions on the code package size. Abandoning the restrictions of cloud vendors on code packages, simply speaking of the impact that the specifications of the code package may have can be seen through the cold start process of the function, as shown in the figure.

Function cold start process diagram

In the process of function cold start, when the uploaded code package is too large, or the decompression speed is too slow due to too many files, it will make the process of loading code longer and further lead to longer cold start time.

Imagine that there are two compression packages, one is a code compression package of only 100KB, and the other is a code compression package of 200MB. Both are idealized under gigabit intranet bandwidth (that is, regardless of disk storage speed, etc. ) For downloading, even if the maximum speed can reach 125MB/s, the download time of the former is less than 0.01 seconds, and the download time of the latter is 1.6 seconds. In addition to the download time, plus the file decompression time, the cold start time of the two may differ by 2 seconds. In general, for the traditional web interface, if a response time of more than 2 seconds is required, it is actually unacceptable for many businesses, so when packaging the code, it is necessary to reduce the size of the compressed package as much as possible. Taking the Node.js project as an example, when packaging code packages, we can use methods such as Webpack to reduce the size of dependent packages, further reduce the overall code package specifications, and improve the cold start efficiency of functions.

Reasonable reuse of examples

In order to better solve the problem of cold start and make more reasonable use of resources, there are instances of instance reuse in the FaaS platforms of various cloud vendors. The so-called instance reuse means that when an instance completes a request, it will not be released, but will enter a silent state. Within a certain time frame, if a new request is allocated, the corresponding method will be called directly, without the need to initialize various resources, etc., which greatly reduces the occurrence of cold start of the function. For verification, we can create two functions:

函数1：
# -*- coding: utf-8 -*-

def handler(event, context):
    print("Test")
    return 'hello world'
函数2：
# -*- coding: utf-8 -*-
print("Test")

def handler(event, context):
    return 'hello world'

Click the "Test" button on the console to test the above two functions and determine whether they output "Test" in the log. The statistical results are shown in the table.

Function reuse record

As you can see, in fact, instance reuse exists. Further thinking, if the print("Test") statement is an initial database connection, or function 1 and function 2 load a deep learning model, is it function 1 or every request will be executed, and function 2 can reuse existing Object?

So in actual projects, some initialization operations can be implemented according to function 2, for example:
 

• In the machine learning scenario, load the model during initialization to avoid loading the model every time the function is triggered.
• The link object is established during initialization to avoid creating the link object every time a request is made.
• Some other files that need to be downloaded and loaded at the first load are implemented at the time of initialization, which improves the efficiency of instance reuse.

Good use of function features

FaaS platforms of various cloud vendors have some characteristics. The so-called platform features mean that these functions may not be the capabilities or described in CNCF WG-Serverless Whitepaper v1.0, but are merely functions that are excavated and implemented from the perspective of users based on the development and demands of the cloud platform. , May just be a function owned by a certain cloud platform or a few cloud platforms. Under normal circumstances, if this type of function is used properly, business performance will be qualitatively improved.

1.Pre-freeze & Pre-stop

Taking Alibaba Cloud Function Computing as an example, during the development of the platform, user pain points (especially hindering the smooth migration of traditional applications to the serverless architecture) are as follows.

• Asynchronous background indicator data is delayed or lost: If it is not sent successfully during the request, it may be delayed until the next request, or the data point may be discarded.
• Synchronous sending indicators increase latency: If the Flush interface is called after each request, it will not only increase the latency of each request, but also cause unnecessary pressure on the back-end service.
• The function goes offline gracefully: When the instance is closed, the application has requirements for cleaning up the connection, closing the process, and reporting the status. When the instance goes offline in the function calculation, the developer cannot grasp it, and there is also a lack of Webhook notification of the offline event of the function instance.

Based on these pain points, Alibaba Cloud released the Runtime Extensions function. This function is extended on the existing HTTP service programming model, adding PreFreeze and PreStop Webhook to the existing HTTP server model. The extension developer is responsible for implementing the HTTP handler and monitoring the life cycle events of the function instance, as shown in the figure.

A sketch of the work content handled by the extended programming model and the existing programming model

• PreFreeze: Each time the function computing service decides to freeze the current function instance, the function computing service will call the HTTP GET/prefreeze path. The extension developer is responsible for implementing the corresponding logic to ensure that the necessary operations before the instance is frozen, such as waiting for the indicator to be sent successfully, etc. as the picture shows. The time when the function calls InvokeFunction does not include the execution time of PreFreeze Hook.

PreFreeze timing diagram

• PreStop: Before each function calculation decides to stop the current function instance, the function calculation service will call the HTTP GET/prestop path. The extension developer is responsible for implementing the corresponding logic to ensure the completion of the necessary operations before the instance is released, such as waiting for the database link to be closed and reporting , Update status, etc., as shown in the figure.

PreStope timing diagram

2. Single instance multiple concurrency

As we all know, the function computing of various cloud vendors is usually request-level isolation, that is, when the client initiates 3 requests to the function computing at the same time, theoretically 3 instances will be generated to respond, this time may involve cold start and state between requests Related issues. Therefore, some cloud vendors provide single-instance multi-concurrency capabilities (for example, Alibaba Cloud Function Computing). This capability allows users to set an instance concurrency for a function (InstanceConcurrency), that is, a single function instance can handle multiple requests at the same time, as shown in the figure.

Single instance multiple concurrency effect diagram

As shown in the figure above, assuming that there are 3 requests to be processed at the same time, when the instance concurrency is set to 1, function computing needs to create 3 instances to handle these 3 requests, and each instance processes 1 request; when the instance concurrency is When it is set to 10 (that is, one instance can handle 10 requests at the same time), function computing only needs to create one instance to handle these 3 requests.

The advantages of single instance and multiple concurrency are as follows.

• Reduce execution time and save costs. For example, partial I/O functions can process requests concurrently in one instance, reducing the number of instances, thereby reducing the total execution time.
• State can be shared between requests. Multiple requests can share the database connection pool within an instance, thereby reducing the number of connections to the database.
• Reduce the probability of cold start. Since multiple requests can be processed in one instance, the number of times to create a new instance will be reduced and the probability of a cold start will decrease.

The application scenarios of single instance and multiple concurrency are relatively wide. For example, the scenario where there is a lot of time in the function to wait for the response of the downstream service is more suitable to use this function. Single-instance multi-concurrency also has scenarios that are not suitable for applications. For example, when the function has shared state and cannot be accessed concurrently, the execution of a single request consumes a lot of CPU and memory resources. At this time, it is not suitable to use the single-instance multi-concurrency function.

*About the author:
Liu Yu (Jiang Yu) is a Ph.D candidate majoring in electronic information at National University of Defense Technology, Alibaba Cloud Serverless product manager, Alibaba Cloud Serverless cloud evangelist, and a distinguished lecturer of CIO Academy. *