Cloud native empowers the construction of the basic framework for intelligent networked vehicle message processing

In recent years, the automotive industry has rapidly evolved towards "electrification, intelligence, networking, and sharing", and the "software-defined vehicle" model and SOA concept have gradually deepened in the field of automotive R&D and design. Whether it is the TSP platform as the cloud base of the intelligent networked vehicle, the autonomous driving system based on the intelligent ADAS of the bicycle, or the SOA framework for realizing the software-defined vehicle, more flexible software development, iteration, reuse and operation architecture guarantee are required.

The rapid development and implementation of cloud-native technologies have greatly changed the traditional development and operation methods of IoV applications. With its characteristics of flexibility, elasticity, agility, automation, high availability, and easy expansion, it provides platform-level assistance for the development and operation of software related to intelligent networking and autonomous driving in the automotive industry, and solves the problem of the Internet of Vehicles platform under the new trend. the above challenges.

Cloud-native: As defined by the CNCF (Cloud Native Computing Foundation), cloud-native technology enables organizations to build and run elastically scalable applications in new and dynamic environments such as public, private, and hybrid clouds. Representative technologies of cloud native include containers, service meshes, microservices, immutable infrastructure, and declarative APIs.

This article aims to deeply analyze how cloud-native technology affects the construction of Internet of Vehicles IoT infrastructure. Based on the most critical areas of vehicle-side message collection, movement, processing and analysis in the system, combined with EMQ-related data infrastructure software, cloud-native vehicle Networking infrastructure architecture.

Challenges of building traditional car networking platforms

The message processing framework of the traditional Internet of Vehicles often uses local data center virtual machines/physical machines or cloud service provider virtual machines for deployment when building the underlying resources and the overall framework of the operating platform. This mode usually faces the following pain points and challenges when the number of connected vehicles increases rapidly and the data uploaded by the vehicle becomes more complex:

1. If major OEMs want to change the mode of providing Internet of Vehicles services in private data centers to providing Internet of Vehicles services through public clouds, the traditional architecture with virtual machines as application carrying methods will be too heavy and cannot smoothly realize the hybrid cloud migration of Internet of Vehicles. Requirements and deployment models of hybrid cloud;
2. With the rapid development of intelligence and networking, the IoV system has gradually increased the software iteration capability requirements of the message processing platform. The traditional software iteration mode cannot meet the intelligent network connection's agile, flexible and fast capability requirements for the IoV system. Unable to respond to complex message processing requirements and software iterations;
3. As the most important real-time communication system between OEMs and end customers, the IoV system needs to have extremely high reliability, availability and supportability. As a core application component, the message processing platform should have flexible resource acquisition capabilities and operational support capabilities such as automatic scaling and operation and maintenance. The traditional monolithic application architecture and virtual machine deployment mode cannot meet the requirements of the elasticity and self-reset capability of the message processing platform.

Cloud-native technology enables next-generation IoV message processing

Among the projects under the CNCF (Cloud Native Computing Foundation), the container orchestration system Kubernetes is the core and foundation. Kubernetes combines application containers into logical units for ease of management and service discovery. It is an open source system that can be freely deployed within an enterprise, private cloud, hybrid cloud or public cloud, allowing users to make free choices.

In the production and delivery scenarios of business platforms, more and more OEMs are using cloud-native technologies to create the following capabilities to help the evolution and development of ICV applications.

• Unified deployment : Provides the ability to shield the underlying resource-based infrastructure differences, build once, and use in multiple places
• Easy to implement : Provide CaaI (Config as an Infrastructure), that is, the ability to configure as a facility, and to achieve the ability to configure as a runtime effect
• Elastic capacity expansion : Quickly and elastically scale resource-based resources according to business usage, providing runtime scaling without the ability to perceive business applications
• Monitoring and alarming : Provide a complete monitoring and alarming system to meet the controllability of post-maintenance of the production environment
• Version iteration controllable : Provide risk-controlled version change means, including version traceability and rollback capabilities

Operator-based EMQX cloud native framework

The early cloud-native deployment of EMQ products adopted the Helm deployment method. The emergence of the Operator mode provides a standard solution for implementing custom resources, and solves the pain point that the general Kubernetes basic model elements cannot support complex automation scenarios in different business fields. Simpler and more efficient deployment of EMQX provides a whole new way.

Simply put, the Operator pattern is a collection of custom controllers and a series of custom resources managed by these controllers. We will no longer focus on basic model elements such as Pods (containers) and ConfigMaps, but will Aggregated into an application or service. The Operator makes the custom application reach the desired state through the coordination loop of the controller. We only need to pay attention to the desired state of the application, and achieve 7*24 hours of uninterrupted application or service declaration through the custom controller coordination loop logic Cycle management. The Operator-based EMQX cloud-native framework enables users to easily deploy and operate EMQX clusters based on the Kubernetes model.

Operator VS Helm

Operator management is not limited to Pods, but can also be multiple resources (such as SVC domain names, etc.). From this point of view, Operator, like Helm, also has the ability to orchestrate. From an orchestration point of view, Helm and Operator have so much in common that it is difficult to distinguish the roles of the two. Helm can also complete the deployment of distributed systems.

So what is the difference between Operator and Helm?

• Helm focuses on multiple resource management, and the management of life cycle mainly includes creation, update and deletion. Helm drives the entire life cycle through commands.
• Operator's management of resources is not only about creation and delivery. Since it can obtain the change events of related resources by means of watch, it can realize operation and maintenance operations such as high availability, scalability, and fault recovery. Therefore, the operator's management of the life cycle includes creation, failure recovery, high availability, upgrade, expansion and contraction, exception handling, and final cleanup, etc.
• If Helm is like RPM, then Operator is systemd. RPM is responsible for application installation and removal, while systemd is responsible for application startup, restart, and so on.

How Operators work

Operators use custom resources (CRs) to manage custom Kubernetes controllers for applications and their components, custom resource API extensions in Kubernetes, custom resource configuration CRDs will specify CRs and list all configurations available to Operator users, and Operators monitor CR types And take application-specific actions to ensure that the current state matches the desired state of the resource.

There are mainly the following objects in Operator:

• CRD: Definition of custom resources, the Kubernetes API server will generate a RESTful resource path for each CRD version you specify. A CRD is actually a place to define its own application business model. It can fully customize the resource objects it needs according to the needs of the business, such as EMQX Broker, EMQX Enterprise, etc. These are all custom resource objects that can be directly operated and managed by Kubernetes. .
• CR: A custom resource, that is, a specific instance of CRD, is a specific manageable Kubernetes resource object, which can be managed in normal life cycle, such as creation, deletion, modification, query, etc. At the same time, CR objects generally also contain The runtime state, such as what is the real state of the current CR, is used to observe and judge the real state of the CR object.
• Controller: In fact, it is the real use of the controller. It will process the actions in the work queue in a loop, and coordinate the current state of the application to the desired state according to the logic. For example, after observing that a CR object is created, the CR will be processed according to the implemented logic, so that the state of the CR object and the business logic responsible for the CR object will gradually approach the desired state, and finally achieve the desired effect. For example: if an EMQX Broker operator is defined, then when the EMQX Broker is created, it will always coordinate and observe whether the real EMQX Broker cluster has been created, and whether the status and availability of each node are healthy. Continue to coordinate once a state that does not meet expectations is found, always keeping an event-based mechanism constantly checking and reconciling to ensure the desired state.

Cloud Native Practice of EMQX in the Internet of Vehicles Scenario

Based on the Operator mode, we provide EMQX Operator to help customers quickly build and manage EMQX clusters in the Kubernetes environment.

EMQX Operator is a tool to help users quickly create and manage EMQX clusters in the Kubernetes environment. It can greatly simplify the process of deploying and managing EMQX clusters, turning it into a low-cost, standardized, and repeatable capability.

As the core underlying support component of the Internet of Vehicles, EMQX can be deployed, managed, and operated through Kubernetes Operator. Through the cloud-native message processing platform, it brings many benefits to customer development and operation and maintenance deployment in the Internet of Vehicles scenario:

• Insensitive and rolling update: The IoV message processing framework built with cloud native technology can easily realize the grayscale release of IoV applications, so that the IoV system does not need to be interrupted during the iterative process of upgrading the IoV system, so that users of IoV applications are completely free. Perception, realize non-sensing iteration and rolling update, and improve user experience;
• Unified monitoring: Based on cloud-native technology, operators of IoV systems can easily monitor and manage application clusters and nodes. Through the integration with monitoring tools such as Prometheus, the operation information of the most important message processing platform in the Internet of Vehicles can be easily obtained, so as to understand the business operation more intuitively and clearly, and carry out corresponding business iterations;
• Rapid deployment: Cloud-native technology allows IoV developers and operation and maintenance personnel to quickly deploy and adjust applications. Whether it is based on public cloud or private cloud environment, the corresponding EMQX cluster can be easily deployed to support business;
• Standardized image: EMQX provides a standardized official container image based on the cloud native environment. When the user system deploys and builds EMQX through the image, the corresponding work can be performed based on the standardized image. Construction and other requirements provide good support;
• Elastic scaling: With the deepening of Internet of Vehicles applications, the applications that need to be connected to the entire message processing framework gradually expand and the scale of the Internet of Vehicles increases, and the message processing platform has higher and higher requirements for resource elasticity. The resource support mode can acquire, increase and release resources according to the usage of the application, thereby saving resources and reducing operating costs;
• Rapid iteration: IoV applications built on cloud-native frameworks can utilize continuous integration and continuous delivery pipelines to achieve instant update and release of applications, supporting business needs for rapid iteration of IoV;

Epilogue

With the deepening of cloud-native concepts in various industries, we believe that cloud-native will also inject new impetus into the platform construction and application development model in the field of Internet of Vehicles. In the future, EMQ will focus on supporting the features of the new version of EMQX, continuously improve and iterate the EMQX Operator, and strive to provide richer and more reliable data infrastructure capabilities in the cloud-native mode to serve the Internet of Vehicles industry.

Other articles in this series

• The construction of the Internet of Vehicles platform from entry to mastery 01 | MQTT protocol in the Internet of Vehicles scene
• The construction of the Internet of Vehicles platform from entry to proficient 02 | Architecture design of the MQTT messaging platform for the Internet of Vehicles
• The construction of the Internet of Vehicles platform from entry to mastery 03 | MQTT theme design in the TSP platform scene of the Internet of Vehicles
• The construction of the Internet of Vehicles platform from entry to proficient 04 | MQTT QoS design: quality assurance of message transmission on the Internet of Vehicles platform
• IoV platform construction from entry to proficient 05 | IoV platform million-level message throughput architecture design
• The construction of the Internet of Vehicles platform from entry to mastery 06 | SSL/TLS protocol for Internet of Vehicles communication security
• The construction of the Internet of Vehicles platform from entry to proficient 07 | The application of State Secret in the security certification scene

Copyright statement: This article is original by EMQ, please indicate the source when reprinting.

Original link: https://www.emqx.com/zh/blog/cloud-native-smart-connected-car-messaging