KubeVela + KEDA: Brings "inborn" elastic scalability to applications

Introduction to In this blog post, we will briefly explain the areas that need to be considered, how KEDA makes application auto-scaling easy, and why Alibaba Cloud Enterprise Distributed Application Service (EDAS) is fully standardized on KEDA.

Co-author|
Yan Xun, Senior Engineer of Alibaba Cloud EDAS Team
Andy Shi, Alibaba Cloud Developer Advocate
Tom Kerkhove, Codit Containerization Business Leader, Azure Architect, KEDA Maintainer, CNCF Ambassador
Source | Alibaba Cloud Native Public
​
When you are scaling Kubernetes, you will think of some areas, but if you are new to Kubernetes, you may find it difficult to handle.
​
In this blog post, we will briefly explain the areas that need to be considered, KEDA makes application auto-scaling easy, and why Cloud 160af589147dd5 Enterprise Distributed Application Service (EDAS) ​ is fully standardized on KEDA.
​

Scaling Kubernetes

When managing Kubernetes clusters and applications, you need to carefully monitor various things, such as:
​

• Cluster capacity-do we have enough available resources to run our workloads?
• Application workload-Does the application have enough resources available? Can it keep up with the work to be done? (Like queue depth)

In order to achieve automation, you usually set up alerts to get notifications, or even use auto-scaling. Kubernetes is a good platform, it can help you achieve this ready-to-use feature.
​
by using the Cluster Autoscaler 160af589147f48 component, which will monitor the cluster to find pods that cannot be scheduled due to resource shortages, and start adding/removing nodes accordingly.
​
Because Cluster Autoscaler only starts when the pod is over-scheduled, you may have a period of time during which your workload is not up and running.
​
Virtual Kubelet (a CNCF sandbox project) is a huge help, it allows you to add a "virtual node" to the Kubernetes cluster, pod can be scheduled on it.
​

By doing this, platform providers (such as Alibaba, Azure, HashiCorp, and others) allow you to spill the suspended pod outside the cluster until it provides the required cluster capacity to alleviate the problem.
​
In addition to scaling clusters, Kubernetes also allows you to easily scale applications:
​

• The Horizontal Pod Autoscaler ( HPA ) allows you to add/remove more Pods to your workload to scale in/out (add or remove copies).
• Vertical Pod Autoscaler ( VPA ) allows you to add/remove resources to your Pod to scale up/down (add or remove CPU or memory).

All of these provide a good starting point for scaling your application.
​

Limitations of HPA

Although HPA is a good starting point, it mainly focuses on the pod's own metrics, allowing you to scale it based on CPU and memory. In other words, you can fully configure how it should automatically scale, which makes it powerful.
​
Although this is ideal for certain workloads, you usually want to scale based on metrics elsewhere such as Prometheus, Kafka, cloud providers, or other events.
​
Thanks to external indicator support ​, users can install indicator adapters to provide various indicators from external services and automatically scale them by using indicator servers.
​
However, one thing to note is that you can only run one indicator server in the cluster, which means you must choose the source of custom indicators.
​

You can use Prometheus and tools, such as Promitor, to get your metrics from other providers and use them as a single source of truth to scale, but this requires a lot of plumbing and work to scale.
​
There must be an easier way...yes, use Kubernetes Event-Driven Autoscaling (KEDA)!
​

What is KEDA?

Kubernetes Event-Driven Autoscaling (KEDA) is a single-purpose event-driven autoscaling device for Kubernetes, which can be easily added to a Kubernetes cluster to scale applications.
​
Its goal is to make application auto-scaling very simple, and to optimize costs by supporting scale-to-zero.
​
KEDA removes all scaling infrastructure and manages everything for you, allowing you to scale on more than 30 systems or use your own scaler to scale.
​
Users only need to create ScaledObject or ScaledJob to define the objects you want to scale and the triggers you want to use; KEDA will take care of the rest!
​

You can scale anything; even if it is the CRD of another tool you are using, as long as it implements the /scale sub-resource.
​
So, has KEDA reinvented the wheel? Do not! Instead, it extends Kubernetes by using HPA at the bottom level. HPA uses our external metrics, which are provided by our own metrics adapter, which replaces all other adapters.
​

Last year, KEDA joined CNCF as a CNCF sandbox project, and plans to upgrade the proposal to the incubation stage later this year.
​

Alibaba's practice based on OAM/KubeVela and KEDA

Enterprise Distributed Application Service (EDAS), as the main enterprise PaaS product on Alibaba Cloud, has served countless developers on the public cloud on a huge scale for many years. From an architectural point of view, EDAS was together KubeVela project 160af5891483d8​. Its overall structure is shown in the figure below.
​

In production, EDAS integrates the ARMS monitoring service on Alibaba Cloud to provide fine-grained indicators for monitoring and application. The EDAS team added an ARMS Scaler to the KEDA project to perform automatic scaling. They also added some features and fixed some bugs in the KEDA v1 version. include:
​

• When there are multiple triggers, these values ​​will be summed instead of being left as separate values.
• When KEDA HPA is created, the length of the name will be limited to 63 characters to avoid triggering DNS complaints.
• The trigger cannot be disabled, which may cause trouble in production.

The EDAS team is actively sending these fixes to upstream KEDA, although some of them have been added to the V2 version.
​

Why does Alibaba Cloud standardize KEDA as an automatic scaler for its applications

When it comes to auto-scaling features, EDAS initially used the CPU and memory of the upstream Kubernetes HPA as two indicators. However, with the growth of the user base and the diversification of needs, the EDAS team quickly discovered the limitations of upstream HPA:
​

1. has limited support for custom indicators, especially for application-level fine-grained indicators . Upstream HPA mainly focuses on container-level indicators, such as CPU and memory, which are too crude for applications. Metrics that reflect application load (such as RT and QPS) are not readily supported. Yes, HPA can be expanded. However, when it comes to application-level metrics, this capability is limited. The EDAS team is often forced to fork the code when trying to introduce fine-grained application-level metrics.
2. does not support scaling to zero . When their microservices are not being used, many users have the need to scale to zero. This demand is not limited to FaaS/serverless workloads. It saves costs and resources for all users. Currently, upstream HPA does not support this feature.
3. does not support the scheduled expansion and contraction . Another strong demand of EDAS users is the predetermined scalability. Similarly, upstream HPA does not provide this feature, and the EDAS team needs to find alternatives that are not vendor-locked.

Based on these requirements, the EDAS team began to plan a new version of the EDAS auto-scaling feature. At the same time, EDAS introduced OAM in early 2020, completely reforming its underlying core components. OAM provides a standardized and pluggable application definition for EDAS to replace its internal Kubernetes application CRD. The scalability of this model allows EDAS to easily integrate with any new features in the Kubernetes community. In this case, the EDAS team tried to combine the requirements for the new automatic scaling feature of EDAS with the standard implementation of the OAM automatic scaling feature.
​
Based on the use cases, the EDAS team summarized three standards:

1. The auto-scaling feature should present itself as a simple atomic function without the need to attach any complicated solutions.
2. The indicator should be pluggable, so the EDAS team can customize it and build on it to support various needs.
3. It needs to support scaling to zero out of the box.

After a detailed evaluation, the EDAS team chose the KEDA project, which is open sourced by Microsoft and Red Hat and has been donated to CNCF. KEDA provides several useful Scalers by default, and supports scaling to zero out of the box. It provides fine-grained auto-scaling for applications. It has the concept of Scalar and Metric adapters, supports a powerful plug-in architecture, and provides a unified API layer. Most importantly, the design of KEDA only focuses on auto-scaling, so that it can be easily integrated as an OAM feature. In general, KEDA is very suitable for EDAS.
​

Look to the future

In the next step, Alibaba is actively promoting the KEDA feature driven by AIOps, with the goal of bringing intelligent decisions to its automatic scaling behavior. This will essentially realize automatic scaling decisions based on expert systems and historical data analysis, using the newly implemented application QoS triggers and database measurement triggers in Alibaba's KEDA component. Therefore, we expect that a more powerful, smarter, and more stable KEDA-based auto-scaling function will be released in KEDA soon.

Copyright statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.