MQTT theme design in the Internet of Vehicles TSP platform scenario

The author of this article: Tian Zhen, the former platform architect of SAIC Volkswagen, is now the head of the cloud technology of Zhongke Thunderbolt

foreword

In the Internet of Vehicles ecosystem, the TSP (Telematics Service Provider) platform occupies a core position in the industry chain. It is connected to automobile, in-vehicle equipment manufacturers and network operators, and content providers. It is the core data connection between OEM vehicles and services. platform. With the development of smart cars and the continuous improvement of the demand for application scenarios by car owners and users, the demand for equipment and application carrying capacity of the TSP platform by OEMs will continue to increase.

In the previous article " MQTT Protocol in the Internet of Vehicles Scenario", we mentioned that MQTT is lightweight, easy to expand, and a variety of message quality assurance (QoS) in the selection of data interaction protocol between in-vehicle equipment and TSP platform. , and the advantages of realizing data generation and data consumption system decoupling through the publish-subscribe model have become the preferred protocol for the new generation of TSP platforms of major OEMs.

In this article, we will introduce how to design the MQTT message topic during the construction of the Internet of Vehicles TSP platform.

The need for message channels in the Internet of Vehicles TSP scenario

In the TSP scenario of the Internet of Vehicles, the MQTT protocol is used as a business message channel between the "vehicle-platform-application", which not only ensures that the messages between the vehicle and the application can be interconnected in both directions, but also needs to identify and distribute different types of messages through certain rules. . The topic in the MQTT protocol is the label of these messages, which can also be regarded as a business channel.

In the Internet of Vehicles scenario, messages can be divided into the data upstream channel from the vehicle-platform-application and the data downstream channel from the application-platform-vehicle; for the Internet of Vehicles TSP platform, different data directions mean different business types, which need to be passed through. MQTT topics are clearly distinguished and isolated.

• From the perspective of the car:

  In the TSP platform, vehicle data reporting is the main business type of uplink data. With the continuous enrichment of Internet of Vehicles services, the computing and communication capabilities of in-vehicle systems such as T-box are continuously enhanced, and the business scenarios, data volume and frequency of vehicle data reporting are also increasing. Based on the requirements of business isolation, real-time performance and security, it has gradually developed from one vehicle, one theme in the early days of the Internet of Vehicles to one vehicle with multiple message channels.

• From the application side:

  The platform application serves as the vehicle data receiver and consumer, and also serves as the message sender for data delivery and instruction delivery. According to different business requirements, message sending types can also be divided into:

  1. One-to-one message: For some critical services such as vehicle control and high security requirements, it is necessary to provide a one-to-one message channel for each vehicle.
  2. One-to-many message: For a certain type of business or a certain vehicle model, instructions and data can be issued to the vehicle and equipment through the same topic channel.
  3. Message broadcast: For large-scale message notification, configure update scenarios, and send large-scale message broadcasts to devices connected to the platform.

What is the subject of the MQTT protocol

basic concepts

There are three roles in the MQTT protocol communication mechanism: message publisher (publisher), broker server (broker) and message subscriber (subscriber). The message is sent from the publisher to the proxy server, and then received by the subscriber, and the topic is the message channel agreed between the publisher and the subscriber.

The topic specified by the publisher sends messages, the subscriber subscribes to receive messages from the specified topic, and the broker acts as a broker that accepts and distributes messages according to the topic. In the Internet of Vehicles TSP platform scenario, in-vehicle devices, mobile terminals and business applications can all be regarded as MQTT clients. According to different services and different data directions, the roles of in-vehicle devices, mobile terminals and service applications will also switch between publishers and subscribers.

Definitions and Specifications of Subjects

The MQTT protocol stipulates that the topic is a UTF-8 encoded string, and the topic needs to meet the following rules:

• All topic names and topic filters must contain at least one character.
• Topic names and topic filters are case-sensitive. For example: ACCOUNTS and Accounts are different subject names.
• Topic names and topic filters can contain space characters. For example: Accounts payable is a valid subject name
• Topic names or topic filters are distinguished by a leading or trailing slash /. Such as: /finance and finance are different.
• Topic names or topic filters that contain only slashes / are legal.
• Topic names and topic filters cannot contain null characters (Unicode U+0000).
• Topic names and topic filters are UTF-8 encoded strings, and there is no limit to the number of levels for topic names or topic filters, except that they cannot exceed the length limit of UTF-8 encoded strings.

Topic hierarchy

MQTT protocol topics can be divided into multiple levels by slashes ("/" U+002F); as a message channel, clients can subdivide message types by defining topic levels;

For example: an OEM has multiple models, and each model has multiple IoV services. When we define the car-machine to send a message to the business system of a certain model, we can send a message to <model A>/ <vehicle unique identifier>/< business X> subject message;

Of course, in the MQTT world, topics can have many layers (there is no limit to the number of layers in the MQTT protocol), such as: <model A>/<vehicle unique identifier (chassis number)>/<business X>/<sub-business 1>

In this way, we can design according to the theme level when defining the hierarchical business channel of the Internet of Vehicles.

wildcard

Topic filters for subscribers' subscriptions in the MQTT protocol can contain special wildcards that allow clients to subscribe to multiple topics at once.

• Multilevel wildcards

  The \# character symbol ("#" U+0023) is a wildcard character used to match any level in the subject. A multi-level wildcard represents its parent and any number of child levels. For example, subscribers can receive by subscribing to <model A>/#:

  <Model A>

  <Model A>/<Frame No. 1>

  <Model A>/<Frame No.1>/<Business X>

  news on these topics.

• single-level wildcard

  The plus sign ("+" U+002B) is a wildcard for single topic level matching. For example: subscribers can receive by subscribing to <model A>/+

  <Model A>/<Frame No. 1>

  <Model A>/<Frame No. 2>

  Different from multi-level wildcards, when single-level wildcards are used, sub-level topics cannot be matched. For example, the topic messages of <model A>/<frame number 1>/<business X> cannot be received.

Best Practices for Theme Design Principles of IoV TSP Platform

In the previous article, we mentioned that in the Internet of Vehicles scenario, the MQTT topic defines the channel of business and data. The core of the topic definition is to distinguish business scenarios. How to reasonably define a theme needs to be designed according to certain principles. We can design and define themes from the following dimensions:

Distinguish according to business data direction

First, the different upstream and downstream directions of data determine who generates and consumes data. In the Internet of Vehicles scenario, the data uplink channel from the in-vehicle device to the platform and the downlink data from the platform application to the car need to be separated by topic. By distinguishing the design of upstream and downstream topics, it can help design, operation and maintenance and business personnel to quickly locate scenarios, problems and related stakeholders.

Some businesses may use both uplink and downlink themes, such as the data issued by the platform after the vehicle application data is issued, and the data reported by the vehicle after the platform requests the vehicle to go to work. In this case, due to the asynchronous communication mechanism of the MQTT protocol, the upstream and downstream topics of an overall business also need to be defined separately.

Differentiate by model

In the Internet of Vehicles scenario, different models mean that the data generated by the vehicles are not the same, the capabilities of the vehicles are not the same, and the business applications connected are not the same. We can thematically differentiate differentiated vehicle data and businesses based on model models. Of course, different models under the same OEM will also have the same business and data, and these businesses can be defined by cross-model themes.

Differentiated by vehicle

In the Internet of Vehicles scenarios, business scenarios with higher security levels such as vehicle control often require one-to-one topics as data communication channels. On the one hand, the business information between vehicles is isolated through topics, and on the other hand, the data can be interacted point-to-point. In topic design, it is sometimes necessary to implement a one-to-one message channel with the vehicle's unique identifier as part of the topic. A common scenario is to use the vehicle VIN number as part of the theme.

Differentiate by user

In actual usage scenarios, there are also one-to-one message channels that need to be implemented based on users (rather than vehicles) to the cloud. Such requirements often occur in scenarios such as user promotion, operations, and ToB services. When designing a theme, there are two common solutions. One is to use the user ID as part of the theme; the other is to convert it into a vehicle-level theme through the human-vehicle relationship. Under the scheme, both the production end and the consumer end need to add additional design and processing, which is relatively complicated.

According to the research and development environment

From the perspective of project engineering implementation, environment variables are generally added during theme design, and resource reuse and correctness checking in different R&D environments can be achieved through configuration.

Distinguish based on data throughput

Due to different services, whether it is uplink data or downlink data, the data sending frequency and packet size are not the same. Different data throughput will affect the processing and architecture design of the consumer. For example, we often need to consider the consumption capacity of the application layer when dealing with high-frequency vehicle data reporting services. At this time, we may use high-throughput message queues such as Kafka. To buffer data, prevent data backlog and data loss caused by untimely application consumption. Therefore, in the definition of MQTT topics, we often need to distinguish services with different data throughputs.

Application of MQTT protocol theme design in car networking scenarios

Active reporting of vehicle data

On-board equipment (T-box, car machine, etc.), as a collector of vehicle operation data, package and send data of various controllers and sensors in the car to the platform based on a fixed frequency. This type of data can generally be designed at multiple levels, such as the model, frame number, and business data type of the reported data.

For example, under the premise of the user's consent, the vehicle will report the location, speed, power and other information to the cloud platform at a fixed frequency during the driving process. Based on these data, the cloud application provides location search, speeding reminders, power reminders, and geo-fencing services to the terminal. user use.

Vehicle data report after platform request is issued

When the cloud platform needs to obtain the latest status and information of the vehicle, it can actively issue commands to require the vehicle to report data. Such scenarios can generally be designed thematically according to the frame number, business type and other levels.

For example, in a diagnosis scenario, the platform sends a diagnosis command to the vehicle through MQTT. After each device in the vehicle completes the diagnosis operation, the diagnosis data will be packaged and reported to the cloud platform. The vehicle diagnosis engineer will carry out the diagnosis based on the collected diagnosis data. Overall analysis and problem location.

Platform Instruction Issuance

Vehicle remote control is the most common and typical scenario in the Internet of Vehicles business. All OEMs provide various remote control functions in mobile apps, such as remote start, remote door opening, remote flashing lights and horns, and so on. In such scenarios, the mobile app sends control commands to the cloud platform. After the platform application has undergone a series of operations such as permission check and security check, the command is sent to the vehicle for execution through MQTT. After the vehicle is successfully executed, the platform is asynchronously notified of the execution result.

Such scenarios can generally be designed according to multiple levels such as uplink and downlink, frame number, business type, and operation type.

The platform data is issued after the vehicle client requests

In the context of SDV (software-defined vehicle), many configurations in the car can be dynamically changed, such as data collection rules and security access rules. Therefore, after the vehicle is ignited, it will actively request the latest relevant configuration of the platform. If the configurations on both sides are inconsistent, the platform side will send the latest configuration information to the vehicle, and the vehicle side will take effect in real time.

Such scenarios can generally be designed according to multiple levels such as uplink and downlink, frame number, and business type.

Using EMQX for the theme design of the Internet of Vehicles TSP platform

EMQX the world's leading MQTT IoT messaging middleware. Based on outstanding features such as distributed clusters, large-scale concurrent connections, and fast and low-latency message routing, EMQX can effectively handle high-time-sensitive business needs in the Internet of Vehicles scenarios and greatly reduce End-to-end latency provides standard MQTT services for rapid deployment of large-scale IoV platforms.

Advantages of EMQX in the Internet of Vehicles Scenario

Massive theme support

With the continuous increase of business in the Internet of Vehicles scenario, the number of topics carrying business channels is also increasing, especially the demand for one vehicle, one topic and one vehicle multi-topic required in vehicle control scenarios is increasing. In this context, the number of topics carrying capacity of the MQTT server has become an important evaluation indicator of the TSP platform.

EMQX has planned the ability to support massive device connections and massive topics in the initial underlying design. A common 16-core 32G memory 3-node EMQX cluster can support millions of topics running at the same time, providing a flexible design space for the theme design of the TSP platform.

Powerful rule engine

EMQX provides a built-in rule engine, which can provide search, filtering, data splitting, and message rerouting for different topic data based on the rule engine. Using the rule engine, we can reprocess the data in the existing theme by creating new routing rules and data preprocessing rules in the scenario where the existing in-vehicle devices and application themes have been established. After the vehicle is on the market, support for new business applications is realized by defining new rules on the platform side.

In EMQX Enterprise Edition , the rule engine provides data persistence docking capability, and data in different topics can be directly docked with different persistence schemes through the configuration in the rule engine. For example, data with high data throughput can be connected to high-throughput message queues such as Kafka and Apache Pulsar through the rule engine for data buffering; and small-throughput and low-latency topic data such as vehicle alarms can be directly connected to applications to achieve fast routing and consumption of data.

Proxy subscription function

EMQX provides a proxy subscription function. When the client establishes a connection, it can automatically establish a user-preset subscription relationship without sending an additional SUBSCRIBE message. This allows the platform side to directly manage the topic subscription relationship of in-vehicle devices, which facilitates unified management on the platform side.

Rich topic monitoring and slow subscription statistics

EMQX Enterprise Edition provides running data monitoring with the topic as the monitoring dimension. You can clearly see the total number and current rate of message inflow, outflow, and discard under the topic in the EMQX Visual Dashboard.

Since version 4.4, EMQX provides statistics on slow subscriptions. This function will track the time consumption of the entire message transmission process after the QoS1 and QoS2 messages arrive at EMQX, and then use the exponential moving average algorithm to calculate the average message transmission delay of the subscriber, and then rank the subscribers according to the delay. .

By continuously monitoring the data reception and consumption of various topics during the operation of the TSP platform, the platform operator can continuously adjust the platform business design and application design according to business changes, and realize the continuous optimization and expansion of the platform.

Matters needing attention

When we use EMQX as the Internet of Vehicles TSP platform MQTT Broker, we need to pay attention to the following issues in the process of designing the theme:

• Wildcard usage and topic number hierarchy

  Because EMQX uses the data structure of topic tree to filter and match topics. In the scenario where wildcards are used to match multiple topics, if there are too many topic levels, EMQX will consume a lot of resources. Therefore, when designing the theme, it is not recommended to have too many layers, generally not more than 5 layers.

• Themes and memory consumption

  Since the number of topics and the length of topics in EMQX are mainly related to memory, we should also focus on monitoring the memory usage of the EMQX cluster while hosting a large number of topics.

Summarize

With the widespread popularity of the MQTT protocol in the Internet of Vehicles business, the theme design of the MQTT message on the Internet of Vehicles TSP platform will be an issue that all OEMs and TSP platform solution providers must face. This article is a multi-dimensional summary of the MQTT theme design ideas for the Internet of Vehicles business based on our years of TSP platform construction experience, hoping to give industry colleagues some help and inspiration in the early stage of platform design and business expansion.

Copyright statement: This article is original by EMQ, please indicate the source when reprinting.

Original link: https://www.emqx.com/zh/blog/mqtt-topic-design-for-internet-of-vehicles