Clusternet-a new generation of open source multi-cluster management and application governance project

author

Xu Di, Tencent Cloud Container Technology Expert.

Ru Yingzhe, Senior Product Manager of Tencent Cloud.

Summary

In the past few years, the field of cloud computing has undergone many huge changes. At present, more and more organizations deploy applications on multiple infrastructure platforms on local and cloud . These platforms may be two public Cloud service providers, or two private clouds, or multi-region edge clouds.

The new form leads to a change in the way management and application management infrastructure, the traditional technical architecture and management methods increases the complexity and risk, it is difficult to meet the challenges of deployment and management of application services across multiple platforms, on behalf of the industry latest ideas The Clusternet project came into being.

Open source Clusternet project

Clusternet (Cluster Internet) is Tencent Cloud's open source cloud native project that combines multi-cluster management and cross-cluster application orchestration, making managing multiple clusters as simple as surfing the Internet. . Whether your Kubernetes cluster is running on a public cloud, private cloud, hybrid cloud or edge cloud, you have a consistent management/access experience, using K8s API to centrally deploy and coordinate multi-cluster applications and services.

Clusternet uses the Addon plug-in method to facilitate user one-click installation, operation and maintenance and integration, and easily manage millions of Kubernetes clusters, making cloud computing , free and convenient .

Based on the Clusternet project, a multi-cloud multi-cluster management platform is implemented to provide users with cross-cloud, cross-cluster, and cross-region/zone distributed container services, which will better meet the needs of multiple scenarios.

● Multi-tenancy

● High availability and disaster tolerance

● Multi-cloud multi-center service

● Edge Computing

Architecture concept

Clusternet is designed for the future cloud-native multi-cloud multi-cluster. The leading architecture supports users to uniformly manage each cluster and application from a global perspective, and easily publish user services to the world. everywhere in one release.

Cluster management cluster

Clusternet follows the cloud native concept , and all management components are flexibly deployed in the Kubernetes cluster through Addon to realize the K8s-on-K8s cluster management cluster.

Compared with other multi-cluster management solutions in the community, there is no need to learn and maintain complex platform software and systems, nor to maintain additional control plane components and etcd. Clusternet cluster management completely reuses the existing Kubernetes clusters and ports, and works through AA ( Aggregated APIServer ), which is convenient for flexible expansion. greatly reduces the complexity of operation and maintenance and resource consumption .

The Clusternet architecture guarantees light weight and simplification, and the components include:

1、 clusternet-hub
The clusternet-hub component is deployed and run in the parent cluster, and works by means of AA (Aggregated APIServer).

mainly responsible for:

● Approve the registration request of each sub-cluster and create dedicated resources for it, such as namespace, ServiceAccount, and RBAC rules.

● Maintain a long link between the parent cluster and each sub-cluster

● Provide Kubernetes-style REST API to access each sub-cluster, especially for the access to the edge sub-cluster, and also support the exchange of services of the sub-cluster

● Support multi-cluster application distribution and governance

2、clusternet-agent
The clusternet-agent component is deployed in each sub-cluster.

mainly responsible for:

● Automatically register the current cluster to the parent cluster as a child cluster.

● Establish a TCP full-duplex websocket secure tunnel with the parent cluster. It supports selecting whether to establish a secure tunnel through FeatureGate "SocketConnection". If this feature is turned off, it means that the parent cluster can access the child cluster through a direct connection.

● Report cluster heartbeat information, including Kubernetes version, platform information, healthz/readyz/livez health status, cluster capacity, node status, etc.

Based on the lightweight and flexible architecture of Clusternet, it supports self-registration of the parent cluster. Clusternet-hub can publish applications to the cluster where it is located. This method maximizes the use of parent cluster resources, and can quickly expand the user’s existing cluster by Have the ability to manage massive public cloud, private cloud, and edge cloud clusters.

Multi-cluster distribution of any resource type

Clusternet support distributed to different clusters and manage application resources , including native Kubernetes various resources (Deployment / StatefulSet / ConfigMap / Secret, etc.), all kinds of CRD resources, as well as HelmChart applications and so on.

The following figure is the Clusternet's multi-cluster application distribution model . The green module needs to be created by the user, and the purple module is the resource object for circulation within the Clusternet. Clusternet provides the kubectl plug-in, which can be used to create resources through the "kubectl clusternet apply" command.

The Clusternet resource distribution model adopts a loosely coupled design. Users do not need to change or rewrite existing resource objects. They only need define additional distribution strategies (Subscription) and differentiated configuration (Localization/Globalization) to achieve multi-cluster Application distribution.

● **Compatible with any resource type
It is fully compatible with K8s standard resources**, such as Deployment, StatefulSet, DaemonSet, and various custom CRDs, and there is no need to learn the CRD definition of complex multi-cluster resources.

● **Subscription
Define the resources you want to install in the cluster**. For each matching cluster, a corresponding Base object will be created in its dedicated namespace

● Localization & Globalization
When distributing multi-cluster applications, can also use Localization and Globalization differentiation strategies for differentiated configurations between different clusters . Among them, Localization describes the differentiated configuration strategy of namespace-scoped (namespace scope), and Globalization describes the differentiated configuration strategy of cluster-scoped (cluster scope). Both support management and configuration according to Priority (priority). This feature is very flexible and practical for scenarios such as blue-green releases, canary releases, and version upgrades for multiple clusters.

● Base & Description
Clusternet will Base and Description objects to observe and track the distribution of various application resources. Among them, Description is the object obtained after the Base object is rendered through the differential configuration of Localization and Globalization, that is, it describes the object definition to be finally deployed to the target sub-cluster.

Cloud native ecology is fully compatible

At the beginning of the project design, fully compatible with the cloud native ecosystem .

Clusternet provides multi-platform version support, including common Linux platforms, such as x86, arm64, armv6, ppc64le, s390x.

Clusternet also provides the kubectl plug-in ( kubectl-clusternet ), which can be operated directly through the kubectl command line, without introducing additional CLI tools, and without switching the context of kubeconfig, you can create, update, edit, and delete , Expansion and contraction of and other operations are convenient to be compatible with the user's existing CICD publishing system.

In addition, Clusternet also provides support for client-go, which is fully compatible with all versions of client-go. Clusternet by wrapperFunc to non-invasive manner provides Clusternet multi-cluster docking capability. Developers can refer to the document guide , and pass the three lines of code as possible to achieve multi-cluster and application management capabilities in your system.

Clusternet supports cloud-native ecological software and projects. Since the v0.2.0 version, it has supported the multi-cluster distribution of Helm Charts. Based on Clusternet's native support for any K8s resource, all K8s ecological projects and software can achieve multi-cluster capability through the Clusternet project.

Note: kubectl-clusternet about to add become Kubectl official plugin , detailed information refer PR , the future can be by "kubectl krew install clusternet" one-click install the plugin , the first stage may be by way of direct download binary file use.

Future outlook

Clusternet is Tencent’s latest open source multi-cluster management and application governance project. The new architecture design follows the concept of cloud native, is fully compatible with native K8s resources and expands to multiple clusters, and supports users to quickly and low-cost docking with the K8s cloud native ecosystem to help your Business transformation to the future distributed cloud.

Currently Clusternet is still in rapid development and iteration. release , bringing more excellent features, welcome everyone to continue to pay attention. Clusternet is willing to work with colleagues in all fields to be realistic and innovative, and forge ahead, and jointly create a new era of intelligent connection of all things, symbiosis of virtual and reality, and true interconnection.

join us

When the Clusternet project is open sourced, please pay attention to https://github.com/clusternet/clusternet support.

We are convening the first batch of open source technology fans of Clusternet in a limited amount. As long as you are interested in Clusternet and related technologies, you are welcome to join. Participation method: Follow-Public-Public-Number: [Tencent Cloud Native], add a small assistant to the group.

Our open source technology fans will be able to enjoy the following special benefits~

1. Exclusive open source technology fan exchange group

2. Direct communication with project authors, industry experts, etc.

3. More community activities and benefits are waiting for you to participate

[Tencent Cloud Native] Yunshuo new products, Yunyan new technology, Yunyou Xinhuo, Yunxiang information, scan the QR code to follow the public account of the same name, and get more dry goods in time! !