Introduction to Serverless architecture also includes changes in thinking, especially in the development process. Some people say that to regard the Serverless architecture as a natural distributed architecture, it is necessary to develop Serverless applications with the idea of a distributed architecture. Admittedly, this statement is correct. But in some cases, Serverless still has some features, so it is necessary to change the development concept.
Foreword: In the Serverless architecture, although more energy is focused on business code, in fact, some configurations and costs also need to be paid attention to, and when necessary, serverless applications need to be configured and code optimized according to the configuration and cost. .
The change of serverless application development concept
In addition to a new architecture and a new programming paradigm, Serverless architecture also includes changes in thinking, especially some changes in the development process. Some people say that to regard the Serverless architecture as a natural distributed architecture, it is necessary to develop Serverless applications with the idea of a distributed architecture. Admittedly, this statement is correct. But in some cases, Serverless still has some features, so it is necessary to change the development concept.
1. File upload method
In traditional web frameworks, uploading files is very simple and convenient, such as Python's Flask framework:
f = request.files['file']
f.save('my_file_path')But under the serverless architecture, files cannot be uploaded directly for the following reasons:
- Under normal circumstances, some cloud platform API gateway triggers will convert binary files into strings, which is inconvenient to obtain and store directly;
- In general, the data packets transmitted between the API gateway and the FaaS platform have a size limit, and many platforms limit the data packet size to less than 6MB;
- FaaS platforms are mostly stateless. Even if they are stored in the current instance, files will be lost as the instance is released.
Therefore, the file upload scheme commonly used in traditional web frameworks is not suitable for direct use in the serverless architecture. In the Serverless architecture, there are usually two methods for uploading files: one is to convert to Base64 format and upload, and persist the files to object storage or NAS, but the data packets transmitted between the API gateway and the FaaS platform have a size limit , So this method is usually suitable for business scenarios where small files such as avatars are uploaded.
Another upload method is to upload through platforms such as object storage. Because the client directly transfers the file to the object storage through the key, etc., there is a certain risk, so usually the client initiates the upload request, and the function calculation is based on the requested content. Perform the pre-signature operation and return the pre-signed address to the client. The client uploads using the specified method. After the upload is completed, the upload result is updated through the object storage trigger, etc., as shown in the following figure.
File upload example under Serverless architecture
Taking Alibaba Cloud Function Computing as an example, the above two common upload methods are implemented through Bottle. In function calculation, first initialize the object storage related objects, etc.:
Initialize object storage related objects, etc.:
AccessKey = {
"id": '',
"secret": ''
}
OSSConf = {
'endPoint': 'oss-cn-hangzhou.aliyuncs.com',
'bucketName': 'bucketName',
'objectSignUrlTimeOut': 60
}
#获取/上传文件到OSS的临时地址
auth = oss2.Auth(AccessKey['id'], AccessKey['secret'])
bucket = oss2.Bucket(auth, OSSConf['endPoint'], OSSConf['bucketName'])
#对象存储操作
getUrl = lambda object, method: bucket.sign_url(method, object, OSSConf['object
SignUrlTimeOut'])
getSignUrl = lambda object: getUrl(object, "GET")
putSignUrl = lambda object: getUrl(object, "PUT")
#获取随机字符串
randomStr = lambda len: "".join(random.sample('abcdefghijklqrstuvwxyz123456789
ABCDEFGZSA' * 100, len))The first upload method, after uploading via Base64, persist the file to the object storage:
#文件上传
# URI: /file/upload
# Method: POST
@bottle.route('/file/upload', "POST")
def postFileUpload():
try:
pictureBase64 = bottle.request.GET.get('picture', '').split("base64,")[1]
object = randomStr(100)
with open('/tmp/%s' % object, 'wb') as f:
f.write(base64.b64decode(pictureBase64))
bucket.put_object_from_file(object, '/tmp/%s' % object)
return response({
"status": 'ok',
})
except Exception as e:
print("Error: ", e)
return response(ERROR['SystemError'], 'SystemError')The second upload method is to obtain the pre-signed object storage address, and then initiate an upload request on the client side, and transmit it directly to the object storage:
#获取文件上传地址
# URI: /file/upload/url
# Method: GET
@bottle.route('/file/upload/url', "GET")
def getFileUploadUrl():
try:
object = randomStr(100)
return response({
"upload": putSignUrl(object),
"download": 'https://download.xshu.cn/%s' % (object)
})
except Exception as e:
print("Error: ", e)
return response(ERROR['SystemError'], 'SystemError')HTML part:
<div >
<div >
<h3>Web端上传文件</h3>
</div>
<hr>
<div>
<p>
方案1：上传到函数计算进行处理再转存到对象存储，这种方法比较直观，问题是 FaaS 平台与 API 网关处有数据包大小上限，而且对二进制文件处理并不好。
</p>
<input type="file" name="file" id="fileFc"/>
<input type="button" onclick="UpladFileFC()" value="上传"/>
</div>
<hr>
<div>
<p>
方案2：直接上传到对象存储。流程是先从函数计算获得临时地址并进行数据存储（例如将文件信息存到 Redis 等），然后再从客户端将文件上传到对象存储，之后通过对象存储触发器触发函数，从存储系统（例如已经存储到Redis）读取到信息，再对图像进行处理。
</p>
<input type="file" name="file" id="fileOss"/>
<input type="button" onclick="UpladFileOSS()" value="上传"/>
</div>
</div>Implementation of client-side JavaScript uploaded by Base64:
function UpladFileFC() {
const oFReader = new FileReader();
oFReader.readAsDataURL(document.getElementById("fileFc").files[0]);
oFReader.onload = function (oFREvent) {
const xmlhttp = window.XMLHttpRequest ? (new XMLHttpRequest()) : (new
ActiveXObject("Microsoft.XMLHTTP"))
xmlhttp.onreadystatechange = function () {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
alert(xmlhttp.responseText)
}
}
const url = "https://domain.com/file/upload"
xmlhttp.open("POST", url, true);
xmlhttp.setRequestHeader("Content-type", "application/json");
xmlhttp.send(JSON.stringify({
picture: oFREvent.target.result
}));
}
}The client uses the pre-signed address to transmit directly to the client-side JavaScript implementation of object storage:
function doUpload(bodyUrl) {
const xmlhttp = window.XMLHttpRequest ? (new XMLHttpRequest()) : (new Active
XObject("Microsoft.XMLHTTP"));
xmlhttp.open("PUT", bodyUrl, true);
xmlhttp.onload = function () {
alert(xmlhttp.responseText)
};
xmlhttp.send(document.getElementById("fileOss").files[0]);
}
function UpladFileOSS() {
const xmlhttp = window.XMLHttpRequest ? (new XMLHttpRequest()) : (new Active
XObject("Microsoft.XMLHTTP"))
xmlhttp.onreadystatechange = function () {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
const body = JSON.parse(xmlhttp.responseText)
if (body['url']) {
doUpload(body['url'])
}
}
}
const getUploadUrl = 'https://domain.com/file/upload/url'
xmlhttp.open("POST", getUploadUrl, true);
xmlhttp.setRequestHeader("Content-type", "application/json");
xmlhttp.send();
}The overall effect is shown in the figure.
File upload experiment web effect under serverless architecture
At this point, we can experiment with different types of file upload schemes on the current page.
2. File read and write and persistence method
During the execution of the application, file read and write operations may be involved, or some file persistence operations may be involved. In the traditional cloud host mode, you can directly read and write files, or persist files in a directory, but this is not the case under the serverless architecture.
Since the FaaS platform is stateless and will be destroyed after use, files cannot be directly persisted in the instance, but can be persisted to other services, such as object storage, NAS, etc.
At the same time, without configuring NAS, the FaaS platform usually only has the writable permission of the /tmp directory, so some temporary files can be cached in the /tmp folder.
3. Use the features of some web frameworks with caution
(1) Asynchronous
Function calculation is the isolation of the request level, so it can be considered that the request is over, and the instance may enter a silent state. In function computing, API gateway triggers are usually called synchronously (take Alibaba Cloud Function Computing as an example, usually only in several cases such as timing triggers, OSS event triggers, MNS topic triggers, and IoT triggers. trigger).
This means that when the API gateway returns the result to the client, the entire function will enter a silent state or be destroyed instead of continuing to execute the asynchronous method. Therefore, it is usually difficult for frameworks such as Tornado to play its asynchronous role under the serverless architecture. Of course, if users need asynchronous capabilities, they can refer to the asynchronous methods provided by cloud vendors.
Taking Alibaba Cloud Function Computing as an example, Alibaba Cloud Function Computing provides users with an asynchronous call capability. When the asynchronous call of the function is triggered, the function calculation will put the trigger event into the internal queue and return the request ID instead of returning the specific call situation and function execution status. If the user wants to obtain the result of the asynchronous call, it can be achieved by configuring the asynchronous call target, as shown in the figure.
Function schematic diagram of asynchronous function
(2) Timed task
Under the serverless architecture, once the application completes the current request, it will enter a silent state, and even the instance will be destroyed, which results in some frameworks with self-timed tasks that cannot execute the timed tasks normally. Function calculations are usually triggered by events and do not start automatically at regular intervals. For example, a timing task is set in the Egg project, but in the actual function calculation, if the function is not triggered by a trigger, the function will not be triggered, nor will it be automatically started from the inside to execute the timing task. You can use it at this time Timing triggers replace timing tasks by triggering a specified method by timing triggers.
4. Pay attention to the application composition structure
(1) Static resources and business logic
Under the serverless architecture, static resources should provide external services with the support of object storage and CDN, otherwise all resources are in functions. External exposure through function calculations will not only reduce the concurrency of the business logic of the function, but also cause more costs. Especially when migrating some existing programs to a serverless architecture, such as Wordpress, we must pay more attention to splitting static resources and business logic, otherwise performance and cost will be severely tested in the case of high concurrency.
(2) Split of business logic
In many cloud vendors, the charging standards for functions are based on running time, configured memory, and generated traffic. If the memory setting of a function is unreasonable, the cost will increase exponentially. Want to ensure reasonable memory settings, but also to ensure the reliability of the business logic structure.
Taking Alibaba Cloud Function Computing as an example, an application has two external interfaces, one of which consumes less than 128MB of memory, and the other has a stable memory consumption of around 3000MB. These two interfaces are triggered 10,000 times a day on average, and the time consumption is 100 milliseconds. If two interfaces are written into one function, then this function may need to set the memory at 3072MB, and at the same time, the user requests an interface with less memory consumption and it is difficult to get better performance in the case of cold start; if the two interfaces are written to the function separately , Then the two function memories can be set to 128MB and 3072MB respectively, as shown in the table.
From the above table, it can be clearly seen that a reasonable and appropriate split of the business will save costs to a certain extent. The cost savings of the above example is nearly 50%.
Electronic Information, National University of Defense Technology, Alibaba Cloud Serverless Product Manager, Alibaba Cloud Serverless Cloud Evangelist, Special Lecturer of CIO Academy.
New book recommendation
This book will introduce to readers what serverless architecture is, how to get started with serverless architecture, the application of serverless architecture in different fields, and how to develop a serverless from scratch through multiple open source projects, multiple cloud products from multiple cloud vendors, and multiple channels. Application etc. This book can help readers integrate the Serverless architecture into their own field, put the Serverless project on the ground, and obtain the technical dividends brought by the Serverless architecture.
Serverless Engineering Practice Series
Serverless Engineering Practice | From Cloud Computing to Serverless
Serverless Engineering Practice |
Serverless Engineering Practice | Traditional Web Framework Migration
Click on the link below to buy now!
Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。