Continued innovation in serverless sleds

Article author: Huang Shuai, senior developer evangelist of Amazon Cloud Technology

Introduction

Since 2014, Amazon Cloud Technology has been providing serverless capabilities to users through the Lambda service. With Lambda, users don't have to worry about managing servers or adjusting capacity to respond to changing market demands, all of which are automatically provisioned by Amazon Cloud Technology, and users only pay for the resources they use.

Datadog's 2021 State of Serverless report shows that developers are accelerating the adoption of serverless architectures to address new, more advanced business challenges. We're excited to see more and more organizations benefit from the agility, resiliency, and cost efficiencies that come with adopting serverless technologies like Lambda, supporting this growing and diverse developer community. -- Ajay Nair, General Manager of Amazon Cloud Technology Lambda Service Experience

Behind this powerful serverless service lies the inside story of the continuous innovation of many bases. Today we will talk about the continuous innovation of Amazon Cloud Technology's Serverless base.

Base Innovation

Serverless is an architecture in the field of cloud computing. Users no longer need to purchase or manage a fixed number of servers, but the cloud dynamically manages server resources according to the user's business characteristics, and provides flexible scheduling, automatic expansion and contraction, and self-healing of faults. Capability, users do not need to be billed for the server, but pay for the resources that are actually consumed.

In a broad sense, Serverless usually consists of FaaS (Function as a Service) and BaaS (Backend as a Service). Users do not need to care about the management and maintenance of the infrastructure, but only need to care about the business code to run their own server business in the FaaS environment. Logic, and can use a variety of BaaS services provided by the cloud, and do not need to care about operation and maintenance and server expansion and contraction.

Therefore, apart from the diversity of BaaS, the foundation of the entire Serverless essentially depends on the performance and security of Serverless Computing Services (FaaS). That is to say, the innovation of the serverless base we are discussing here is aimed at the Lambda service. At the same time, we can also think about the performance and security of FaaS, which depends on which aspects. In short, the performance and security of FaaS depend heavily on the base chip and virtualization capabilities. Therefore, the serverless base we are going to discuss today continues to innovate, mainly focusing on FaaS chips and virtualization.

Chip innovation

Graviton is an ARM-based server processor released by Amazon Cloud Technology in 2018 for users of EC2 virtual machine instances. The first-generation Graviton processors feature custom silicon and 64-bit Neoverse cores, and EC2 A1 instances support ARM-based applications such as web servers, cache queues, distributed big data processing, and containerized microservices. Using the open ARM architecture means saving costs by not creating new chips from scratch, instead taking the existing ARM architecture, customizing it for how EC2 instances work, hoping to provide users with EC2 instance selection More options to provide high availability and security for ARM-based applications, while reducing virtualization costs, providing users with good server performance and lower prices.

The Graviton2 processor represents a significant leap in performance and functionality compared to the first-generation Graviton processor. Graviton2-based instances offer the best price/performance for workloads in EC2 instances. Graviton2-based instances support a broad range of general-purpose, burst, compute-optimized, memory-optimized, storage-optimized, and accelerated computing workloads, including application servers, microservices, high-performance computing, machine learning inference, video encoding, EDA, gaming, databases, and memory cache. Many serverless services, including Lambda and Fargate, etc., also support Graviton2-based instances to provide a fully managed experience with cost-effectiveness, improved performance, and cost savings.

The Graviton3 processor is the latest in the Graviton processor family. Compared to the Graviton2 processor, the compute performance is increased by 25%, the floating-point performance is increased by 2x, and the performance of cryptographic workloads is increased by 2x. Compared to Graviton2 processors for ML workloads, Graviton3 processors offer 3x better performance, including support for bfloat16, and also support for DDR5 memory, which provides 50% more memory bandwidth compared to DDR4, and will also be supported in the future Serverless services such as Lambda.

The Lambda service currently supports running on ARM-based Graviton2 processors. Get up to 34% better price/performance with this processor architecture option. Julian Wood's article, Migrating AWS Lambda functions to Arm-based AWS Graviton2 processors, highlights how to migrate from x86 to ARM64, and what to consider during the migration. The performance and security built into the Graviton2 processor are designed to provide up to 19% performance improvement for compute-intensive workloads, workloads that use multithreading and multiprocessing or perform I/O operations, can experience shorter call times and Cost reduction, with a 20% reduction in duration charges billed in milliseconds compared to x86 pricing. The change from the x86 architecture to the ARM architecture does not affect the way Lambda functions are invoked, and integrations with APIs, services, applications or tools are unaffected by the new architecture and continue to work as before.

Many functions can be migrated seamlessly with configuration changes, other functions need to be rebuilt to use the ARM64 package, we can also use the Lambda PowerTuning tool to test the performance of the migrated Lambda function, which can be accessed here:

• https://github.com/alexcasalboni/aws-lambda-power-tuning

  Virtualization Innovations

When initially building the Lambda service, we had two paths:

1. Containerized, fast and resource efficient, but does not provide strong isolation between users
2. Running code in a virtual machine provides greater security at the cost of computational overhead.

Security is always a top priority at AWS, so we built Lambda in the first place using traditional VMs.

Users are asking us for faster scaling, lower latency, and advanced features like provisioned concurrency. We know that this functional requirement cannot be built on a traditional VM.

As users increasingly adopt serverless technologies, we realize that current virtualization technologies are not optimized for ephemeral workloads. We believe that we need to build a virtualization technology especially for serverless computing, which needs to provide a security boundary based on hardware virtualization, while maintaining the lightweight and agility of containers and functions.

So we built Firecracker and released the virtualization platform as open source in November 2018. Firecracker is an open source hypervisor (VMM) using Linux Kernel-based Virtual Machine (KVM) technology. Firecracker allows the creation of tiny virtual machines, known as microVMs. Firecracker adheres to a minimalist design principle and includes only the components needed to run a secure, lightweight virtual machine. Firecracker is licensed under Apache 2.0. Please visit Firecracker GitHub repository to learn more and contribute to Firecracker.

• https://github.com/firecracker-microvm/firecracker
  Firecracker also satisfies

1. Security of Virtual Machines Based on Hardware Virtualization
2. Resource efficiency and fast startup times for containers

At the USENIX Network System Design and Implementation Symposium 2020, we published a paper explaining in detail how Firecracker works.

• https://assets.amazon.science/96/c6/302e527240a3b1f86c86c3e8fc3d/firecracker-lightweight-virtualization-for-serverless-applications.pdf

Among them, containers (left) give code direct access to some core functions of the operating system ("kernel"), enhancing security by denying access to others (x in the "sandbox" layer); virtual machine (right) ) provides workloads with their own user kernel, isolated using hardware virtualization capabilities.

The core part of virtualization is the virtual machine monitor (or VMM), which is used to set up virtualization, manage memory, and handle I/O (such as network connections and disk storage). A traditional VMM is almost as complex as a full operating system, for example: QEMU is a hypervisor, often used in conjunction with the Linux Kernel Virtual Machine (KVM), with over 1.4 million lines of code (and a correspondingly broad and powerful set of features) ).

The reason Firecracker is so much more efficient than QEMU is its stripped-down VMM, which is only 50,000 lines of code—96% less than QEMU. This allows the creation of a single microVM for each user program, a simple but powerful security model. Also, these 50,000 lines of code were written in Rust language, and in the fall of 2017 we decided to write Firecracker in Rust language, a very advanced programming language that guarantees thread and memory safety, prevents buffer overflows and may cause Many other types of memory safety issues for security breaches, and has a built-in REST control API to launch instances, get or set VM configurations, manage snapshots, and more. A single server can create up to 150 Firecracker microVMs per second and run thousands of microVMs simultaneously.

Of course, drastically reducing the size of the VMM also drastically reduces its functionality. Firecracker does not implement traditional devices like BIOS or PCI bus, but communicates with the user kernel through an optimized virtio interface. Importantly, serverless workloads do not require hardware features such as USB, monitors, speakers, and microphones, so there is no need to implement support for these features at all.

Firecracker powers the Lambda service, and a 2020 figure is that Firecracker handles trillions of requests per month for hundreds of thousands of users. At present, the stable 1.0 version has been released. Currently, a Firecracker VM can be fully started in less than 125 milliseconds, and it takes less than 7 milliseconds to fully create a microVM. The memory footprint of each microVM is less than 5MB.

Firecracker hosting platform for a plurality of containers, such as appfleet , Fly.io , Koyeb managing isochronous operation may use a container, comprising a container of by firecracker-containerd, Weave Kubes and Kata Containers, such that It can integrate with Kubernetes.

Summary

This article shares the continuous innovation of Amazon Cloud Technology in the serverless base. We will continue to invest heavily in three levels of serverless computing: application layer, virtualization layer and hardware layer, to provide users with excellent computing power, while ensuring security and scalability. performance and performance. Investing heavily in the research and development of basic technologies is one of the keys to continuous innovation, not only for tomorrow, but also for the next decade and beyond. In the process of this innovation, we also work closely with the community and share. By open-sourcing Firecracker, we not only welcome you to delve into the future foundational technologies for building serverless computing, but also hope that you will work with us to enhance and improve Firecracker. See the Firecracker issue list and Firecracker roadmap for more information.

Click to understand the " start Linux virtual machine " tutorial

For more tutorials, here !