From 0 to 1, build a new generation of open source function computing platform

This article is based on the content shared by Huo Bingjie at the 2021 Rare Earth Developers Conference.

Author: Huo Bingjie, the cloud native FaaS project OpenFunction Founder; the initiator of FluentBit Operator; he is also the initiator of several observable open source projects, such as Kube-Events, Notification Manager, etc.; he loves cloud native and open source technologies, and is Prometheus Operator Contributors to, Thanos, Loki, Falco.

Serverless computing , which is commonly referred to as Serverless, has become a hot term in the current cloud-native field, and is the next wave of cloud computing development after IaaS and PaaS. Serverless emphasizes an architectural idea and service model, so that developers do not need to care about infrastructure (servers, etc.), but instead focus on application business logic. The University of California, Berkeley, in the paper A Berkeley View on Serverless Computing gave two core views on Serverless:

• Serviced computing will not disappear, but as serverless matures, the importance of serviced computing will gradually decrease.
• Serverless will eventually become the computing paradigm in the cloud era. It can largely replace the serviced computing model and put an end to the Client-Server era.

So what is Serverless?

Serverless introduction

Regarding what serverless is, the University of California, Berkeley also gave a clear definition in the aforementioned paper: Serverless computing = FaaS + BaaS . According to the degree of abstraction, the traditional classification of cloud services from the bottom to the top is hardware, basic components of the cloud platform, PaaS, and applications. However, the ideal state of the PaaS layer is to have serverless capabilities, so here we replace the PaaS layer with Serverless, as shown in the figure below In the yellow part.

Serverless consists of two components BaaS and FaaS , of which the basic support services on the cloud such as object storage, relational database and MQ belong to BaaS (back-end as a service), which are essential basic services for every cloud , FaaS (Function as a Service) is the core of Serverless.

Analysis of existing open source serverless platforms

The KubeSphere community will conduct in-depth research on the serverless field from the second half of 2020. After a period of research, we found that:

• Most of the existing open source FaaS projects started early, and most of them existed before Knative appeared;
• Knative is a very outstanding serverless platform, but Knative Serving can only run applications, not functions, and cannot be called a FaaS platform;
• Knative Eventing is also a very good event management framework, but the design is a bit too complicated, and users have a certain threshold to use;
• OpenFaaS is a relatively popular FaaS project, but the technology stack is a bit old, relying on Prometheus and Alertmanager for Autoscaling, which is not the most professional and agile approach in the cloud native field;
• In recent years, cloud native Serverless related fields gradually emerged a lot of good open source projects such as KEDA , DAPR , Cloud Native Buildpacks (CNB) , Tekton , Shipwright , etc., to create a new generation of open-source platform FaaS Laid the foundation.

In summary, the conclusion of our investigation is: existing open source Serverless or FaaS platform cannot meet the requirements for building a modern cloud-native FaaS platform, while the latest developments in the cloud-native serverless field provide the possibility to build a new generation of FaaS platform.

New generation FaaS platform framework design

If we are to redesign a more modern FaaS platform, what should its architecture look like? The ideal FaaS framework should be divided into several important parts according to the function life cycle: Function framework (Functions framework), function construction (Build), function service (Serving) and event-driven framework (Events Framework).

As a FaaS, you must first have a Function Spec to define how to write the function. After you have the function, you must convert it into an application. This conversion process is completed by the function framework ; if the application wants to run in a cloud native environment, You have to build a container image. The build process relies on the function to build to complete; after the image is built, the application can be deployed to the function service ; after being deployed to the runtime, this function can be accessed by the outside world.

Below we will focus on the architecture design of the function framework, function construction and function service.

Functions framework

In order to reduce the cost of learning function specifications in the development process, we need to add a mechanism to realize the conversion from function code to runnable applications. This mechanism needs to be implemented by making a general main function, which is used to process requests coming in through the serving url function. The main function specifically contains many steps, one of which is used to associate the code submitted by the user, and the rest are used to do some common tasks (such as processing context, processing event sources, processing exceptions, processing ports, etc.).

In the process of function construction, the builder will use the main function template to render user code, and then generate the main function in the application container image on this basis. Let's look directly at an example, suppose there is such a function.

package hello

import (
    "fmt"
    "net/http"
)

func HelloWorld(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, World!\n")
}

After the function framework is converted, the following application code will be generated:

package main

import (
    "context"
    "errors"
    "fmt"
    "github.com/OpenFunction/functions-framework-go/functionframeworks"
    ofctx "github.com/OpenFunction/functions-framework-go/openfunction-context"
    cloudevents "github.com/cloudevents/sdk-go/v2"
    "log"
    "main.go/userfunction"
    "net/http"
)

func register(fn interface{}) error {
    ctx := context.Background()
    if fnHTTP, ok := fn.(func(http.ResponseWriter, *http.Request)); ok {
        if err := functionframeworks.RegisterHTTPFunction(ctx, fnHTTP); err != nil {
            return fmt.Errorf("Function failed to register: %v\n", err)
        }
    } else if fnCloudEvent, ok := fn.(func(context.Context, cloudevents.Event) error); ok {
        if err := functionframeworks.RegisterCloudEventFunction(ctx, fnCloudEvent); err != nil {
            return fmt.Errorf("Function failed to register: %v\n", err)
        }
    } else if fnOpenFunction, ok := fn.(func(*ofctx.OpenFunctionContext, []byte) ofctx.RetValue); ok {
        if err := functionframeworks.RegisterOpenFunction(ctx, fnOpenFunction); err != nil {
            return fmt.Errorf("Function failed to register: %v\n", err)
        }
    } else {
        err := errors.New("unrecognized function")
        return fmt.Errorf("Function failed to register: %v\n", err)
    }
    return nil
}

func main() {
    if err := register(userfunction.HelloWorld); err != nil {
        log.Fatalf("Failed to register: %v\n", err)
    }

    if err := functionframeworks.Start(); err != nil {
        log.Fatalf("Failed to start: %v\n", err)
    }
}

The highlighted part is the function written by the user. Before starting the application, register the function first. You can register HTTP functions, cloudevents and OpenFunction functions. After the registration is completed, functionframeworks.Start will be called to start the application.

Function Build (Build)

After we have the application, we have to build the application into a container image. At present, Kubernetes has abandoned dockershim and no longer uses Docker as the default container runtime, so that it is impossible to build container images in the Kubernetes cluster in the way of Docker in Docker. Is there any other way to build an image? How to manage the build pipeline?

Tekton is an excellent assembly line tool, originally a sub-project of , later donated to 161b03be0ac6b9 CD Foundation (Continuous Delivery Foundation) . Tekton's pipeline logic is actually very simple. It can be divided into three steps: get the code, build the image, and push the image. Each step is a task in Tekton, and all tasks are connected in series to form a pipeline.

There are many options for container mirroring, such as Kaniko, Buildah, BuildKit, and Cloud Native Buildpacks (CNB). Among them, the first three rely on Dockerfile to make container images, and Cloud Native Buildpacks (CNB) is the latest emerging technology in the cloud native field. It was initiated by Pivotal and Heroku. It does not rely on Dockerfile, but can automatically detect requirements. Build the code and generate a container image that conforms to the OCI standard. This is a very amazing technology that has been adopted by companies such as Google Cloud, IBM Cloud, Heroku, Pivotal, etc. For example, many images on Google Cloud are built through Cloud Native Buildpacks (CNB).

Faced with so many options for mirroring construction tools, how to allow users to freely select and switch mirroring construction tools during the function construction process? This requires another project, Shipwright , which is an open source project by Red Hat and IBM. It is specifically used to build container images in a Kubernetes cluster. It is also donated to the CD Foundation. Using Shipwright, you can flexibly switch between the above four image building tools, because it provides a unified API interface and encapsulates different building methods in this API interface.

We can use an example to understand how Shipwright works. First, you need a configuration list of Build

apiVersion: shipwright.io/v1alpha1
kind: Build
metadata:
  name: buildpack-nodejs-build
spec:
  source:
    url: https://github.com/shipwright-io/sample-nodejs
    contextDir: source-build
  strategy:
    name: buildpacks-v3
    kind: ClusterBuildStrategy
  output:
    image: docker.io/${REGISTRY_ORG}/sample-nodejs:latest
    credentials:
      name: push-secret

This configuration list is divided into 3 parts:

• source indicates where to get the source code;
• output represents the mirror repository to which the image built by the source code should be pushed;
• The strategy specifies the tool to build the image.

The strategy is configured by the custom resource ClusterBuildStrategy , such as using buildpacks to build a mirror. The content of ClusterBuildStrategy is as follows:

There are two steps here, one is to prepare the environment, and the other is to build and push the image. Each step is a Task of Tekton, managed by the Tekton pipeline.

It can be seen that the significance of Shipwright is to abstract the ability of mirroring. Users can use a unified API to build mirrors. By writing different strategies, they can switch between different mirroring construction tools.

Function Service (Serving)

Function service (Serving) refers to how to run the function/application, and to give the function/application the ability to auto-scaling based on event-driven or traffic-driven (Autoscaling). The CNCF Serverless white paper defines four call types of function services:

We can simplify it, and it can be divided into two types:

• Synchronous function : The client must initiate an HTTP request, and then it must wait until the function is executed and the result of the function is obtained before returning.
• asynchronous function : return directly after initiating the request without waiting for the end of the function to run. The specific result is notified to the caller through events such as Callback or MQ notification, that is, Event Driven.

Synchronous functions and asynchronous functions have different runtimes to implement:

• In terms of synchronization functions, Knative Serving is a very good synchronization function runtime, with powerful auto-scaling capabilities. In addition to Knative Serving, you can also choose to implement synchronization function runtime http-add-on This combination method can get rid of the dependence on Knative Serving.
• Asynchronous functions can be implemented in combination with KEDA and Dapr . KEDA can automatically scale the number of copies of Deployment according to the monitoring indicators of the event source; Dapr provides the ability to access middleware such as MQ for functions.

Knative and KEDA have different capabilities in auto-scaling, and we will analyze them below.

Knative auto-scaling

Knative Serving has 3 main components: Autoscaler, Serverless and Activator. Autoscaler will get the metric of the workload (such as the amount of concurrency). If the current amount of concurrency is 0, the number of copies of the Deployment will be reduced to 0. But the function cannot be called after the number of copies is reduced to 0, so Knative will point the call entry of the function to Activator before the number of copies is reduced to 0.

When new traffic comes in, it will enter the Activator first. After receiving the traffic, the Activator will notify the Autoscaler, and then Autoscaler will expand the number of copies of the Deployment to 1, and finally the Activator will forward the traffic to the actual Pod to implement the service call. This process is also called cold start .

It can be seen that Knative can only rely on Restful HTTP traffic indicators for automatic scaling, but there are many other indicators that can be used as the basis for automatic scaling in real scenarios, such as the message backlog consumed by Kafka. If the message backlog is too large, it needs to be updated. Multiple copies to process the message. If you want to automatically scale according to more types of indicators, we can achieve it through KEDA.

KEDA auto-scaling

KEDA needs to cooperate with the HPA of Kubernetes to achieve more advanced auto-scaling capabilities. HPA can only achieve auto-scaling from 1 to N, while KEDA can auto-scaling from 0 to 1, combining KEDA and HPA The combination can realize automatic scaling from 0 to N.

KEDA can automatically scale according to many types of indicators. These indicators can be divided into these categories:

• Basic indicators of cloud services, such as related indicators of AWS and Azure;
• Linux system related indicators, such as CPU and memory;
• Indicators for specific protocols of open source components, such as Kafka, MySQL, Redis, Prometheus.

For example, to automatically scale based on Kafka's metrics, you need such a configuration list:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-scaledobject
  namespace: default
  labels:
    deploymentName: kafka-consumer-deployment # Required Name of the deployment we want to scale.
spec:
  scaleTargetRef:
    deploymentName: kafka-consumer-deployment # Required Name of the deployment we want to scale.
  pollingInterval: 15
  minReplicaCount: 0
  maxReplicaCount: 10 
  cooldownPeriod: 30
  triggers:
  - type: kafka
    metadata:
      topic: logs
      bootstrapServers: kafka-logs-receiver-kafka-brokers.default.svc.cluster.local
      consumerGroup: log-handler
      lagThreshold: "10"

The range of replica scaling is between 0 and 10. Metrics are checked every 15 seconds. After an expansion, you need to wait 30 seconds before deciding whether to scale.

At the same time, a trigger is defined, which is the "logs" topic of the Kafka server. The message accumulation threshold is 10, that is, when the number of messages exceeds 10, the number of instances of logs-handler will increase. If there is no message accumulation, the number of instances will be reduced to 0.

This method of auto-scaling based on component-specific protocol indicators is more reasonable and more flexible than the method of scaling based on HTTP traffic indicators.

Although KEDA does not support automatic scaling based on HTTP traffic indicators, it can be achieved with the help of KEDA’s http-add-on . The plug-in is currently in Beta status. We will continue to pay attention to the project and wait until it is mature enough. The runtime of the function replaces Knative Serving.

Dapr

Today's applications are basically distributed, and the capabilities of each application are different. In order to abstract the general capabilities of different applications, Microsoft has developed a distributed application runtime, namely Dapr (Distributed Application Runtime). Dapr abstracts the general capabilities of the application into a component . Different components responsible for different functions, such as calls between services, state management, resource binding for input and output, observability, and so on. These distributed components are all exposed to various programming languages for invocation using the same API.

Function computing is also a kind of distributed application, and various programming languages are used. Take Kafka as an example. If the function wants to communicate with Kafka, the Go language must use the Go SDK, and the Java language must use the Java SDK, etc. Wait. If you use several languages to access Kafka, you have to write several different implementations, which is very troublesome.

Assuming that you have to access many different MQ components besides Kafka, it will be more troublesome. Docking 10 MQ (Message Queue) in 5 languages requires 50 implementations of . After using Dapr, 10 MQs will be abstracted into a way, namely HTTP/GRPC docking, so that only 5 implementations , which greatly reduces the workload of developing distributed applications.

It can be seen that Dapr is very suitable for use in functional computing platforms.

A new generation of open source function computing platform OpenFunction

Combining all the technologies discussed above, OpenFunction was born, and its architecture is shown in the figure.

It mainly contains 4 components:

• Function : Convert function to application;
• Build : Choose different image building tools through Shipwright, and finally build the application as a container image;
• Serving : Deploy applications to different runtimes through Serving CRD, you can choose synchronous runtime or asynchronous runtime. Synchronous operation can be supported by Knative Serving or KEDA-HTTP, and asynchronous operation can be supported by Dapr+KEDA.

• Events : For event-driven functions, it is necessary to provide event management capabilities. Due to the complexity of Knative event management, we have developed a new event management driver called OpenFunction Events .

  

  OpenFunction Events borrows part of the design of Argo Events and introduces Dapr. The overall structure is divided into 3 parts:

  • EventSource : Used to dock a variety of event sources, implemented through asynchronous functions, and can be automatically scaled according to the indicators of the event source, making the consumption of events more flexible.
  • 161b03be0ad52d EventBus : EventBus uses the ability of Dapr to decouple the binding between EventBus and the underlying specific Message Broker, and you can EventSource consumption event. One is to directly call the synchronous function, and then wait for the synchronous function to return the result; the other is to write it to EventBus , and EventBus will directly trigger an asynchronous function after receiving the event.
  • Trigger : Trigger will EventBus through various expressions. After the filtering is completed, EventBus will be written to trigger another asynchronous function.

For the actual use case of OpenFunction, you can refer to this article: Using OpenFunction asynchronous functions to achieve log alarms in a serverless manner (click the picture below to jump to read).

OpenFunction Roadmap

The first version of OpenFunction was released in May of this year. Starting from v0.2.0, it supports asynchronous functions. Starting from v0.3.1, OpenFunction Events has been added, and Shipwright has been added, and v0.4.0 has added CLI.

follow-up, we will also introduce a visual interface, support more EventSource, support processing capabilities for edge loads, and use WebAssembly as a lighter runtime, combined with Rust functions to speed up the cold start.

Join the OpenFunction community

We look forward to interested developers joining OpenFunction community . You can put forward any questions, design proposals and cooperation proposals you have about OpenFunction.

You can also join our WeChat exchange group, add the group administrator WeChat: cloud-native-yang, and remarks into the OpenFunction exchange group.

You can find some typical use cases of OpenFunction here:

• implements Kubernetes log alarms in a
• OpenFunction Serverless Samples
• OpenFunction Events Samples
• OpenFunction official website