Alibaba's cloud-native hybrid system Koordinator is officially open source

Born out of Alibaba, after years of polishing on Double 11, Koordinator, the co-location system that saves the company billions every year, announced today that it is officially open source. Through open source, we hope to open better co-location and scheduling capabilities to the entire industry to help enterprise customers improve the efficiency, stability, and computing costs of cloud-native workloads.

What is the mixed department?

Many Internet companies in the industry have more or less the technical direction of co-scheduling workloads of different characteristic types, making full use of the peak-shaving and valley-filling effect between loads, so that workloads can be more stable, efficient, and cost-effective. to use resources. Such a system or mechanism is the concept of "co-location" that is often mentioned in the industry.

Alibaba's mix:

Alibaba began to explore container technology in 2011, and started co-location technology research and development in 2016. After several rounds of technical architecture upgrades, it has finally evolved to today's cloud-native co-location system architecture, achieving a full-service scale of over 10 million cores. The cloud-native co-location of cloud-native co-location, the average CPU utilization per day of co-location is over 50%, which helps Alibaba save a lot of resource costs.
Co-working is a cost control core built heavily within Internet companies, and it has accumulated a lot of business abstraction and resource management thinking and optimization experience. Therefore, co-working usually requires several years of polishing practice to gradually stabilize and generate production value. Does every enterprise need a high threshold to use co-location, and need a lot of investment to generate value? Then let our Koordinator try to answer.

Koordinator is based on the practical experience of internal super-large-scale co-location production, aiming to create solutions for users with the lowest access cost and the best co-location efficiency in cloud-native scenarios, and help user enterprises achieve continuous dividend release after cloud-native.

What is a Koordinator?

Koordinator: Taken from coordinator, K for Kubernetes, pronounced the same. The semantics fit the problem to be solved by the project, which is to coordinate and orchestrate different types of workloads in the Kubernetes cluster, so that they run on a cluster and a node with the optimal layout and posture.

Google has a scheduling system called Borg, which is the first system to do container co-location. It has been a very mysterious existence in the industry before its paper was published. The cloud-native container scheduling and orchestration system Kubernetes is inspired by Borg's design ideas and redesigned by the designers of the Borg system based on the needs of application orchestration in the cloud era.

The good scalability of Kubernetes makes it adaptable to various workloads, helping users to solve the daily operation and maintenance efficiency of workloads.

Koordinator is fully extended based on the standard capabilities of Kubernetes, and is dedicated to solving the scheduling, runtime performance and stability challenges of mixed workloads in a cluster and node scenario. The project includes a complete set of solutions for mixed workload orchestration, including three major blocks: refined resource scheduling, task scheduling, and differentiated SLO. Through such a set of solutions to achieve:

1. Help enterprise users access more workloads to Kubernetes, especially workloads related to big data and task processing, and improve their operational efficiency and stability
2. Through open source technical standards, help enterprise users to achieve a consistent technical architecture on and off the cloud, and improve operation and maintenance efficiency
3. Help enterprise users make rational use of cloud resources and achieve sustainable development on the cloud

What are the advantages of Koordinator?

Co-location requires a complete and self-closed scheduling loop, but in the process of enterprise application co-location, the two major challenges that will be faced are:

1. How to access the application to the co-location platform
2. How the application can run stably and efficiently on the platform

Koordinator has learned from Alibaba's internal production practice experience and lessons for many years, and designed solutions for these two major challenges, aiming to help enterprises to use the mixed department and Kubernetes in a real sense, rather than showing off their technical muscles.

The overall architecture of Koordinator 1.0 is shown in the figure below. It provides users with a complete solution for co-located workload orchestration, co-located resource scheduling, co-located resource isolation and performance tuning, helping users to improve the performance of latency-sensitive services and exploit idle time Node resources are allocated to computing tasks that are really needed, thereby improving the overall resource utilization efficiency.

Practical experience in ultra-large-scale production

After the 2021 Double 11, Alibaba announced "the first time! The unified dispatching system has been implemented on a large scale to fully support Alibaba's double 11 business":

As Alibaba's core project, Alibaba Cloud (container team and big data team), together with Alibaba's resource efficiency team and Ant's container orchestration team, has spent more than a year of R&D and technical breakthroughs, and has realized the transition from "mixed technology" to today's "unified" Scheduling technology" comprehensive upgrade.

Today, unified scheduling has achieved the overall unification of Alibaba e-commerce, search promotion, and MaxCompute big data scheduling, the unification of pod scheduling and task high-performance scheduling, the complete resource view unification and scheduling coordination, and the realization of a variety of complex The co-location of business forms and the improvement of utilization rate fully support the large-scale resource scheduling of dozens of data centers, millions of containers, and tens of millions of cores around the world.

As a practitioner of cloud-native co-location, Alibaba is a real tool to promote the concept of co-location technology in the production environment, and completed the co-location scale of more than 10 million cores on Double 11 last year, helping Alibaba through co-location technology. Baba Double 11 saves more than 50% of the cost of resources for the promotion, and increases the speed by 100% on the fast-up and fast-down links of the big promotion, helping the promotion to achieve a smooth user experience.

Looking back, Alibaba's firm promotion of co-location technology is mainly due to the following problems:

• Unbalanced utilization rate : In the non-co-located era, the resource utilization rate among several major resource pools is unbalanced, the utilization rate of the big data resource pool is extremely high, and the computing power is lacking for a long time, while the daily utilization rate of the e-commerce resource pool is relatively low and idle. A large amount of computing resources, but due to the disaster recovery design, the machine cannot be directly dropped to improve the online density. The original intention of co-working is to make global resource scheduling more reasonable. In the daily state, big data tasks are dispatched to the e-commerce resource pool through co-working, so as to make full use of this part of idle resources.

• Low efficiency in preparation for the big promotion: In order to reduce the procurement of resources during the big promotion, it is hoped that the big data resource pool can be borrowed during the big promotion, and the e-commerce task can be deployed to support the traffic peak at the same time. In the non-coordinated era, such elastic resource borrowing can only be promoted by moving machines, and the efficiency of large-scale promotion support is low and difficult to implement on a large scale.

Driven by peak scenarios such as Double 11, Alibaba's co-location scheduling technology has continued to evolve, accumulating a lot of practical production experience, and today it is the third-generation cloud-native full-service co-location system. Such a set of co-location technology solutions based on the cloud native concept was born out of Alibaba, and hopes to radiate to the entire industry through the open source community, helping enterprises to speed up the development of cloud native container scheduling.

Focus on co-location technology to support rich scenarios

Hybrid is a set of resource scheduling solutions for the refined orchestration of delay-sensitive services + mixed deployment of big data computing workloads. The core technologies are:

1. Refined resource scheduling to meet performance and long-tail latency requirements, the key points are refined resource scheduling and scheduling strategies and QoS awareness strategies
2. Intelligent resource oversell, meet the computing resource requirements of computing tasks at a lower cost, and ensure computing efficiency without affecting the response time of delay-sensitive services

The above picture is the oversold model of Koordinator co-working resources, which is also the most critical and core part of co-working. The basic idea of oversold is to use those allocated but unused resources to run low-priority tasks. The four lines shown in the figure are:

1. limit: gray, the amount of resources requested by high-priority Pods, corresponding to the Pod request of Kubernetes
2. usage: red, the amount of resources actually used by the Pod, the horizontal axis is the time line, and the red line is the fluctuation curve of the Pod load over time
3. short-term reservation: dark blue, it is based on the resource usage of usage in the past (shorter) period, and its estimated resource usage in the future period of time, between reservation and limit, that is, allocated unused (estimated will not be used for some time in the future), which can be used to run short-lived batch tasks
4. long-term reservation: light blue, similar to short-term reservation but the estimated historical period of use is longer. The resources from reservation to limit can be used for tasks with a longer life cycle, and the available resources are compared to short-term reservations. Fewer but more stable

This set of resource models supports the co-location of Alibaba's internal businesses. It is sufficiently refined and flexible. The entire co-located resource scheduling building of Koordinator is built on the basis of such a resource model, and combined with priority preemption, load sensing, interference identification and QoS guarantee technologies, the underlying core system for co-located resource scheduling is constructed. The Koordinator community will invest in construction around this idea, continue to develop the scheduling capabilities of co-location scenarios, and export Alibaba's rich experience in scenarios to the community to solve real business scenarios faced by enterprises.

Double zero intrusion, ultra-low access cost

The biggest challenge for enterprises to access the co-location is how to make the application run on the co-location platform. The threshold of this first step is often the biggest obstacle. In response to this problem, Koordinator designed a "double zero intrusion" co-location scheduling system based on internal production experience.

The first zero intrusion refers to zero intrusion on the Kubernetes platform. Most people in the industry know that when Kubernetes is applied to complex scenarios within an enterprise, it is always necessary to make a certain amount of modification to Kubernetes for one reason or another, especially the node management (Kubelet) part. This part of the modification itself It has a large technical threshold, and it also brings huge challenges to subsequent Kubernetes version upgrades. In order to solve this problem, enterprises often need a dedicated team to maintain these customized modifications, and there is a huge silent cost. When there is a problem online or a new version needs to be upgraded, students who are familiar with this modification may not know it. where to go. This brings great technical risks to enterprises, and often hinders the promotion of co-located technologies. The Koordinator co-location system is designed to ensure that there is no need to make any modifications to the community's native Kubernetes. It only needs to install the Koordinator component into the cluster with one click without any configuration, which can bring the co-location capability to the Kubernetes cluster. . At the same time, when the user does not enable the co-location capability, the original Kubernetes cluster will not be disturbed in any way.

The second zero intrusion refers to zero intrusion into the workload orchestration system. Imagine that after providing the co-location capability on the Kubernetes cluster within the enterprise, the problem that will be faced is how to access the workload of the enterprise and run it in a co-located manner. In general, two situations will be faced:

1. Workloads have the characteristics of enterprise private operation and maintenance. The platform or the system of the operation and maintenance team manages the daily upgrade, release, expansion and shrinkage of these workloads. However, between the container or SRE team of the enterprise promoting the mixed department and the platform operation and maintenance team, there is a Given the organizational gap (big or small), how to push the platform team to transform the workload management mechanism and connect the co-location protocol is also a big challenge.
2. Workloads are managed in a native Deployment/StatefulSet/Job manner, and the cost of implementing or transforming their Kubernetes internal design exceeds the team's expectations, which will also become a challenge to implement co-location.

Koordinator designed a separate workload access layer (Colocation Profile) for the transformation cost of the application access layer to help users solve the problem of workload access to co-location. Users only need to manage the configuration of co-location (YAML) to be flexible It is very simple and flexible to schedule which tasks to run in the cluster in a mixed manner. Currently, Koordinator provides users with examples of running Spark tasks together. In the future, the community will continue to enrich the features of the workload access layer to support zero-intrusion access in more scenarios.

Consistent user experience on and off the cloud

The Koordinator open source project is the key battle of Alibaba Cloud Native 2.0. In addition to experiencing the technical dividends brought by the Koordinator hybrid in their own environment, users can also deploy it to any cloud vendor to maintain a hybrid cloud and multi-cloud environment. The architecture is consistent. Of course, it is also possible to obtain a consistent user experience in a variety of cloud products provided by Alibaba, and one design can be connected to multiple places to play value.

It can be seen that in addition to supporting internal super-large-scale business co-location, Koordinator is also a solution for the integration of Alibaba Cloud container services. The community will continue to maintain vitality and strive to turn co-location into a civilian, generalized, and standardized technical capability. .

Why open source?

The earliest container co-department was Borg, which has been running within Google for more than 15 years. The latest public information is Borg: the next Generation[1]. Domestic Internet companies have been promoting co-location for nearly 10 years. Among them, Alibaba's co-location technology has also undergone three generations of technical architecture upgrades and changes, and finally reached the ultimate form of global co-location. The co-location has helped Alibaba's e-commerce, search, and big data businesses to greatly improve the efficiency of preparations for the big promotion, and it has also saved a lot of computing resources for the Double 11 promotion over the years.

We firmly believe that cloud-native co-location is the inevitable direction for the development of enterprise container scheduling technology. Only through the hybrid orchestration of workloads can we achieve better resource utilization efficiency under the business multi-availability zone disaster recovery architecture and give full play to different types of workloads. Therefore, the potential of computing resources can be fully exerted and the value of cloud computing can be maximized.

The open source of Koordinator hopes to allow more enterprises to see and use the capabilities of cloud-native co-location and help enterprises accelerate the process of cloud-native. Technically, Koordinator can help enterprises to realize that more loads can be connected to the Kubernetes platform, enrich the types of workloads for container scheduling, and then play the characteristics of workload staggered and time-sharing, so as to achieve efficiency and cost benefits, maintain A healthy form of long-term sustainable development.

Currently, Koordinator has already supported the co-location of Spark task scenarios, and also provides a low-cost solution to access co-location. I look forward to seeing your co-location application cases and hearing your feedback! In the future, the Koordinator community will continue to enrich the co-location scenarios and business forms, support Flink, Hadoop, AI Jobs, audio and video tasks, etc., look forward to it.

Join the Koordinator community

Are you planning or implementing a project to improve the resource utilization of your Kubernetes cluster?

Are you having trouble with the stability and performance tuning of the runtime resources of the Pod on the Kubernetes cluster?

Are you experiencing multiple complexity caused by inconsistent scheduling systems on and off the cloud?

You are very welcome to join us through Github/Dingding/WeChat to participate in the Koordinator open source community. We look forward to hearing from you:

Github address:
https://github.com/koordinator-sh/koordinator

Join the community Slack channel (English):
https://koordinatorgroup.slack.com/archives/C0392BCPFNK

Welcome to scan the QR code below or search the group number 33383887 (Chinese) to join the Koordinator community DingTalk group!

Related Links

[1]Borg: the next Generation
https://research.google/pubs/pub49065/