KubeVela 1.3 is released: an out-of-the-box visual application delivery platform, introducing new enterprise-level features such as plug-in ecology, permission authentication, and versioning

Author: KubeVela Community

Thanks to the participation of hundreds of developers in the KubeVela community and more than 500 code submissions from more than 30 core contributors, KubeVela version 1.3 is officially released. Compared with the v1.2 version [1] released three months ago, the new version provides a large number of new features in the three aspects of the OAM core engine (Vela Core), the visual application delivery platform (VelaUX) and the community plug-in ecosystem. The birth of these features is derived from a large number of in-depth practices of community users such as Alibaba, LINE, China Merchants Bank, and iQiyi, and finally contributed to the KubeVela project to form functions that everyone can use out of the box.

Pain points and challenges of modern application delivery

So, what pain points and challenges have we encountered in modern cloud-native application delivery and management?

1. Hybrid clouds and multi-clusters have become the norm in business. The composition of applications includes not only containers, but also cloud resources and various self-built services.

On the one hand, with the continuous development of domestic and foreign cloud vendors, the way most enterprises build infrastructure has become a model based on cloud services, supplemented by self-construction. More traditional enterprises can directly enjoy the business convenience brought by the development of cloud technology, use the elasticity of the cloud, and reduce the cost of self-built infrastructure. Enterprises need a standardized application delivery layer, which can include containers, cloud services, and various self-built services in a unified manner, so that cloud-on-cloud interoperability can be easily achieved, reducing the risks brought by tedious application migration, and worry-free cloud migration.

On the other hand, due to security risk control factors such as infrastructure stability and multi-environment isolation, it is also limited by the size of the Kubernetes cluster [2] . More and more enterprises are beginning to adopt multiple Kubernetes clusters to manage container workloads. How to manage and orchestrate container applications at the multi-cluster level, solve problems such as scheduling, dependencies, versions, and grayscales, and at the same time provide business developers with a low-threshold user experience is a big challenge.

It can be seen that the hybrid cloud and multi-cluster involved in modern application delivery are not only multiple Kubernetes clusters, but also diverse workloads and operation and maintenance capabilities including cloud services, SaaS, and self-built services.

2. How to choose more than 1000+ cloud native ecological technologies and products on demand.

Let's take the open source projects that have joined the CNCF ecosystem as an example, the number of which has exceeded 1,000. For teams of different scales, different industries, and different technical backgrounds, it seems that the R&D team is doing similar business application delivery and management, but with changes in requirements and usage scenarios, huge differences in technology stacks will be derived. This involves a very large learning cost and threshold for integration and migration. And CNCF's thousands of ecological projects are always tempting us to integrate new projects, add new features, and better accomplish business goals. The era of static technology stacks is long gone.

Figure 1 CNCF Landscape

Next-generation application delivery and management requires flexible assembly capabilities. According to the needs of the team, on the basis of the minimum capability set, new functions can be expanded at a small cost. At the same time, various technologies can be effectively and intelligently collaborated. but not significantly improved. The traditional PaaS solution based only on a set of experience-cured packaging has been proven to be difficult to meet the changing scenario needs of a team during product evolution.

3. A new generation of DevOps technology for complex and diverse infrastructure delivery and management applications.

For more than a decade, DevOps technology has been evolving with developers with the goal of increasing productivity. Nowadays, the production process of business applications has also undergone great changes. From the traditional coding, testing, packaging, deployment, and operation and maintenance observation, to the continuous enhancement of cloud infrastructure, various SaaS services directly use API. The form becomes an integral part of the application. From the diversification of development languages, to the diversification of deployment environments, and then to the diversification of components, the traditional DevOps tool chain is gradually unable to cope, and it is the increasing complexity that is mapped to the user layer.

With the same DevOps concept, we need different solutions. For modern application delivery and management, we still have the same pursuit of reducing human input as much as possible and becoming more intelligent. The new generation of DevOps technology needs to have easier-to-use integration capabilities, service governance capabilities, and management capabilities that integrate observation and operation and maintenance. At the same time, the tools need to be simple and easy to use, and the complexity stays within the platform. When choosing, enterprises can combine their own business needs, cooperate with the new architecture and legacy systems, and assemble a platform solution suitable for their own team, so as to avoid the new platform becoming a burden for business developers or enterprises.

The solution and path of KubeVela

To build the next generation application delivery platform, we do:

Figure 2 Layered OAM/KubeVela Ecosystem

1. OAM open application model: standard first, continuous accumulation of methodology through practice

Based on the internal practical experience of Alibaba and Microsoft, we launched OAM, a brand-new application model and concept in 2019. Its core lies in the separation of concerns. Through the unified abstraction of components and operation and maintenance features, it standardizes business research and development in the cloud-native era. Collaborate with operation and maintenance personnel to improve delivery and operation and maintenance efficiency, and at the same time expect to avoid the complexity caused by differences in different infrastructures through a standardized application layer. We then released KubeVela as a standardized implementation of the OAM model to help companies quickly implement OAM while ensuring that OAM-compliant applications can run anywhere. In short, OAM describes the complete components of a modern application in a declarative way, while KubeVela runs according to the final state declared by OAM. Through the control loop oriented to the final state, the two jointly ensure the consistency and consistency of application delivery. correctness.

Recently, we have seen a paper published by Google announcing the results of its internal infrastructure construction "Prodspec and Annealing" [3] . Its design concept and practice are strikingly similar to "OAM and KubeVela". The practice in the field of native application delivery has the same goal, which also confirms the correctness of the standardized model and KubeVela practice. In the future, we will continue to promote the development of the OAM model based on the community's practice and evolution of KubeVela, and continue to deposit best practices into methodology.

2. Create a hybrid environment and multi-cluster delivery control plane to fully meet the needs of users of different scales and scenarios to build their own platforms

The kernel of KubeVela exists in the form of CRD Controller [4] ​, which can be easily integrated by the Kubernetes ecosystem, and the OAM model is also compatible with the Kubernetes API. In addition to the abstraction and orchestration capabilities of the OAM model, KubeVela's microkernel is also an application delivery control plane that is naturally designed for multi-cluster and hybrid cloud environments. This also means that KubeVela can seamlessly connect diverse workloads such as cloud resources and containers, and orchestrate and deliver them in different clouds and clusters.

In addition to the basic orchestration capabilities, one of the features of KubeVela is that it allows users to customize the delivery workflow. The steps of the workflow can use ready-made functions such as deploying components to the cluster, waiting for manual approval, sending notifications, etc., or through the CUE-based configuration language Integrate any IaC-based process, such as K8s CRD, SaaS API, Terraform module, mirror script, etc. When the workflow execution process enters a stable state (such as waiting for manual approval), KubeVela will also automatically maintain the state. KubeVela's IaC scalability enables it to integrate Kubernetes' ecological technology at a very low cost. It is very suitable for platform builders to integrate into their own PaaS or delivery systems, and can integrate existing capabilities of enterprises through KubeVela's scalability. , and presented to users as a standardized capability along with other ecological capabilities.

3. An out-of-the-box visual application delivery platform that directly meets the business needs of small and medium-sized teams

In addition to the advanced models and extended kernels, we have also encountered a large number of user voices in the community who want to get an out-of-the-box product for better and faster KubeVela adoption. Since version 1.2, the community has invested in the development of the Visual Application Delivery Platform (VelaUX) project. It is based on the core capabilities of KubeVela, runs through the concept of standardization and extensibility, and creates a platform for Visual application delivery platform for CI/CD vertical scenarios. We hope that enterprises can directly use VelaUX to meet business needs, and have strong customization capabilities to meet the needs of future business development.

Figure 3 KubeVela product architecture description

Around this path, in version 1.3, the community brought the following updates:

Core engine: Kubernetes multi-cluster control plane capability enhancement

The application does not need to be transformed, switch to multi-cluster

After the enterprise has completed the application cloud-native transformation, is it still necessary to perform configuration transformation when switching to multi-cluster deployment? the answer is negative.

KubeVela is naturally built on a multi-cluster basis. For the same application description, we only need to specify the cluster name to be delivered in the delivery policy, or filter specific clusters by tags. As shown in Figure 4, the application configuration represents a An application of the nginx component is published to all clusters with the label region=hangzhou.

Figure 4 OAM Application Description - Select Deployment Cluster

Of course, the application description shown in Figure 4 is completely based on the OAM recommended specification. If your current situation is that the application has been defined in the form of Kubernetes native resources, don't worry, we support the description of Kubernetes native resources inline or by reference. style, as shown in Figure 5 below, "Referencing Kubernetes resources for multi-cluster deployment", which describes a special application whose components rely on a Secret resource that exists in the control cluster and publish it to all clusters with the label region=hangzhou .

Figure 5 Quoting Kubernetes resources for multi-cluster deployment

In addition to the multi-cluster deployment of applications, the function of referencing Kubernetes objects can also be used in scenarios such as multi-cluster replication of existing resources and cluster data backup.

Handling multi-cluster differences

After the unified description of the application, there may be differences in the deployment of different clusters. For example, different regions use different environment variables and different mirror warehouse addresses; for example, different clusters deploy different components, or a component is deployed in multiple clusters. High availability, etc. For such requirements, we provide a deployment difference description strategy, as shown in Figure 6 below, which is the strategy part of the application configuration. The first and second topology types of strategies define two target strategies in two ways. The third difference strategy, which means to deploy only the specified components. The fourth difference policy represents the deployment of the specified two components and the difference mirror configuration of one of the components.

Figure 6 Multi-cluster differential configuration

KubeVela supports a flexible difference configuration strategy, which can be configured through component properties, traits, and other forms. As shown in the figure above, the third strategy expresses the component selection ability, and the fourth strategy expresses the difference of the image version. We can see that there is no target specified when describing the difference, that is, the difference description can be reused, and it can be flexibly combined with the target strategy in the workflow steps.

Configure a multi-cluster delivery process

The process of application delivery to different target clusters is controllable, and the deployment process is described by workflow. As shown in Figure 7, the steps of deploying to two clusters and the target strategy and differentiation strategy adopted respectively are expressed. Combining the above, it can be seen that policy deployment only needs to be defined atomically, and can be flexibly combined in the workflow part to meet the control requirements of different scenarios.

Figure 7 Custom multi-cluster delivery process

There are more usage scenarios for delivery workflow, including multi-cluster grayscale release, enterprise approval, precise release control, and more.

Version control, safe and traceable

The description of complex applications changes at any time with agile development. In order to ensure the security of application release, we need to have the ability to return our application to a previous correct state at the time of release or at any time after release. Therefore, in the current version, we have introduced a more accurate version recording mechanism.

Figure 8 Querying the historical version of the application

We can query the historical version status of the application, including its release time and success. We can compare the current changes based on the version, and we can also quickly roll back based on the configuration snapshot rendered by the previous successful version when we encounter a failure during the release. After the release of the new version, if you encounter failures and other requirements and need to re-release the historical version, you don't need to change the configuration source (the path may be long, and you may not remember what changes have been made), you can directly re-release based on the historical version .

Behind the version control mechanism is the centralized idea of ​​application configuration management. After the complete description of the application is rendered uniformly, it is checked, stored and distributed.

See more details on KubeVela core engine usage

• Multi-cluster application delivery:
  ​​​https://kubevela.net/en/docs/case-studies/multi-cluster
• Distributions refer to external Kubernetes objects:
  ​​​https://kubevela.net/en/docs/end-user/components/ref-objects
• Application version management:
  ​​​https://kubevela.net/en/docs/end-user/version-control

Platform: VelaUX introduces multi-tenancy isolation, user authentication and authentication

Multi-tenancy and isolation to meet the needs of multi-team enterprises

In VelaUX, we introduce the concept of "project" based on multi-tenancy of logic, including application delivery target, application and environment, members and permissions, etc. This capability becomes very important when there are multiple teams or multiple project groups in the enterprise that use the VelaUX platform to publish their business applications at the same time. Figure 9 shows the project list page. Project administrators can create different projects on this page according to the needs of the team, so as to allocate corresponding resources.

Figure 9 Project management page

Open Authentication & Authentication

As a basic platform, user authentication and authentication capabilities are one of the basic capabilities that must be possessed. Since version 1.3, we have supported user authentication and RBAC authentication.

For user authentication, we believe that most enterprises have built a unified authentication platform (Oauth or LDAP), so VelaUX integrates Dex to get through the single sign-on capability first, and supports LDAP, OIDC, Gitlab/Github and other user authentication methods, taking VelaUX as your One of the subsystems in the developer portal of . Of course, if your team does not need unified authentication, we also provide basic local user authentication capabilities.

Figure 10 Local user management page

For authentication, we use the RBAC mode, but we also see that the basic RBAC mode cannot handle precise permission control scenarios, such as authorizing the operation rights of an application to a user, which technically involves data authorization. We inherit the design concept of IAM and expand the permissions to the policy composition of resource + action + condition + behavior. The authentication system (front-end UI authentication/back-end API authentication) has implemented policy-oriented fine-grained authentication. However, in terms of authorization, the current version only has some built-in common permission policies, and subsequent versions provide the ability to create custom permissions.

At the same time, we have also seen that some large enterprises have built independent IAM platforms. VelaUX's RBAC data model is basically the same as that of common IAM platforms in the market. Therefore, users who wish to connect VelaUX to self-built IAM can extend support.

Centralized management of operation and maintenance configuration to ensure the security of multi-cluster application distribution

The application delivery process will inevitably face configuration management of some operation and maintenance requirements, especially on the basis of multi-cluster, configuration management requirements are particularly prominent, such as the authentication configuration of a private image repository, or the authentication configuration of the Helm product library, or the SSL certificate Wait. We need to uniformly manage the validity of these configurations and securely synchronize them where they are needed, preferably without business developer awareness.

In version 1.3, we introduced a module for integrated configuration management in VelaUX. Its bottom layer also uses component templates and application resource distribution links to manage and distribute configurations. Currently, Secret is used for configuration storage and distribution. The configuration lifecycle is independent of business applications, and we maintain the configuration distribution process independently in each project. For administrator users, you only need to fill in the configuration information according to the configuration template.

Figure 11 Integration configuration management page

Different configuration types are provided by different Addons, and users can define more configuration types according to their needs and manage them uniformly. For business-level configuration management, the community is also planning.

See more details on VelaUX usage

• Project management:
  ​​​https://kubevela.net/en/docs/how-to/dashboard/user/project
• User Management:
  ​​​https://kubevela.net/en/docs/how-to/dashboard/user/user
• RBAC authorization:
  ​​​https://kubevela.net/en/docs/how-to/dashboard/user/rbac
• Integrate and use single sign-on:
  ​​​https://kubevela.net/en/docs/tutorials/sso
• Configuration management:
  ​​​https://kubevela.net/en/docs/how-to/dashboard/config/dex-connectors

Ecology: Addon introduces version control

Addon version management, more secure upgrade

The Addon function was introduced in version 1.2, providing a management capability for the specification, installation, and operation and maintenance of extended plug-ins. The community can expand the ecological capabilities of KubeVela by making different Addons. When our plugins and frameworks are constantly iterating, the problem of version compatibility gradually emerges, and we urgently need a version management mechanism.

• Synergy with Vela versions: Support defines the supported Vela versions in the Addon metadata, and refuses to install when the installation environment version does not match.
• Addon versioned distribution: We develop and manage the community's official Addon in Github. In addition to the integrated third-party product version, each Addon also includes various configurations such as Definition, so after each Addon is released, we define it according to its definition. The version number is packaged and the history is preserved. At the same time, we reused Helm Chart's product distribution API specification to distribute Addon.

Multi-cluster Addon controllable installation

There is a type of Addon that needs to be installed in the subcluster during installation, such as the FluxCD plugin shown in Figure 12, which provides Helm Chart rendering and deployment capabilities. We need to deploy it to sub-clusters, and in the past this process was distributed to all sub-clusters. However, according to community feedback, different plug-ins do not necessarily need to be installed in all clusters. We need a differential processing mechanism to install extensions to specified clusters on demand.

Figure 12 Addon configuration management page

When enabling Addon, the user can specify the cluster to be deployed, and the system will deploy the addon according to the user's configuration.

Addon ecology adds new members

While iteratively expanding the capabilities of the framework, the existing Addons in the community are also continuously being added and upgraded. At the cloud service support level, the number of supported vendors has increased to seven. In terms of ecological technology, AI training and service plug-ins, Kruise Rollout plug-ins, Dex plug-ins, etc. have been added. At the same time, the Helm Chart plugin and the OCM cluster management plugin have also been updated for user experience.

See more details on Addon usage

• Addon use: ​​https://kubevela.net/en/docs/how-to/cli/addon/addon

Near-term planning (Roadmap)

With the gradual stabilization of the KubeVela kernel and the gradual release of the dividends of the scalable kernel, the community has gradually accelerated the evolution of the 1.2/1.3 version. In the future, we will gradually iterate new versions in a two-month cycle. In the next 1.4 release, we will add the following features:

• Observability : Provides a complete observability solution around logs, metrics, and traces, provides out-of-the-box observability of the KubeVela system, allows custom observability configurations, and integrates existing observability components or cloud resources.
• Offline installation : Provide relatively complete offline installation tools and solutions to facilitate more users to use KubeVela in an offline environment.
• Multi-cluster permission management: Provides in-depth permission management capabilities for Kubernetes multi-clusters.
• More out-of-the-box plug-in ecological capabilities.

The KubeVela community is looking forward to your joining to build an easy-to-use and standardized next-generation cloud-native application delivery and management platform!

Related Links

[1] v1.2 released three months ago

​​https://kubevela.io/en/blog/2022/01/27/blog-1.2

[2] Limitation of the size of the cluster itself

​​https://kubernetes.io/docs/setup/best-practices/cluster-large/

[3] "Prodspec and Annealing"

​​https://www.usenix.org/publications/loginonline/prodspec-and-annealing-intent-based-actuation-google-production

[4] CRD Controller

​​https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

You can learn more details about KubeVela and the OAM project through the following materials:

• Project code base: github.com/oam-dev/kubevela Welcome to Star/Watch/Fork!
• The official homepage and documentation of the project: kubevela.io, since version 1.1, Chinese and English documents have been provided, and developers are welcome to translate more language documents.
• Project DingTalk Group: 23310022; Slack: CNCF #kubevela Channel
• Join WeChat group: Please add the following maintainer WeChat account to indicate that you have entered the KubeVela user group:

Click ​​here ​​to view the official website of the KubeVela project.