NanoMQ Newsletter 2022-07｜v0.10: Multi-channel bridging, NanoSDK supports MQTT 5.0

NanoMQ continued its steady update in July. v0.10.0 was officially released in early August ( https://github.com/emqx/nanomq/releases/tag/0.10.1 ). This release mainly enhances the bridging function and adds an HTTP API for publishing messages. Also added MQTT 5.0 support to NanoSDK. Various performance optimizations and bug fixes are also ongoing.

Bridge feature update

The bridging function has been significantly updated in version 0.10, adding functions such as multi-channel bridging and AWS IoT Core built-in bridging. These updates also affect the configuration file format. v0.10 is no longer compatible with the bridge configuration file (nanomq_bridge.conf) before v0.9.0. Users need to pay attention to updating the configuration file format when upgrading.

Multi-way MQTT bridge configuration

In the field of IoT, heterogeneous computing and computing offloading are common scenarios. Data from various wireless sensor networks on the terminal side often needs to be synchronized with multiple clouds or repeatedly consumed by multiple applications. MQTT broker is a shortcut to solve the reusable requirement of data stream, and NanoMQ, as a message service mainly for edge computing scenarios, multi-cloud bridging is also a function commonly needed by users. Before v0.9.0, NanoMQ's MQTT bridge only supported a single target. The multi-channel MQTT bridge is officially launched in version 0.10 after testing and verification. It can support users to make NanoMQ synchronize real-time data with multiple remote MQTT services at the same time through custom bridge naming.

For example, we want to let NanoMQ synchronize the messages of a specific topic (msg1/#, msg2/#) to the cloud, and at the same time receive messages from the topic (cmd/topic1) in the cloud and forward them to the local device. A simple configuration example is as follows:

 ## Take EMQX & EMQX cloud as example ##
## 以同时桥接到 EMQX 公共服务器和 EMQX Cloud 为例，配置桥接URL ##
bridge.mqtt.emqx.address=mqtt-tcp://broker.emqx.io:1883
bridge.mqtt.cloud.address=mqtt-tcp://emqx.cloud:1883

## 设置桥接的MQTT协议版本，可选MQTT V4和MQTT V5 ##
bridge.mqtt.emqx.proto_ver=4
bridge.mqtt.cloud.proto_ver=4

## 开启桥接状态 ##
bridge.mqtt.emqx.bridge_mode=true
bridge.mqtt.cloud.bridge_mode=true

## 设置桥接客户端的Client ID ##
bridge.mqtt.emqx.clientid=bridge_client
bridge.mqtt.cloud.clientid=bridge_client

## 设置桥接客户端的心跳间隔 ##
bridge.mqtt.emqx.keepalive=60
bridge.mqtt.cloud.keepalive=60

## 设置需要上行同步数据的主题 ##
bridge.mqtt.emqx.forwards=msg1/#,msg2/#
bridge.mqtt.cloud.forwards=msg1/#,msg2/#

## 设置需要下行同步数据的主题 ##
bridge.mqtt.emqx.subscription.1.topic=cmd/topic1
bridge.mqtt.cloud.subscription.1.topic=cmd/topic1

After starting NanoMQ, it will automatically start the bridge channel according to the URL of the configured remote MQTT service, and manage and monitor the connection health status and synchronization data.

Special reminder: The version after adding the multi-channel bridging function uses a bridge configuration file in a different format, so v0.10 is no longer compatible with nanomq_bridge.conf before v0.9.0.

AWS Bridge

AWS IoT Core is one of the public cloud IoT services widely used in Europe and the United States. However, because it is different from the standard MQTT connection and does not support QoS 2 communication quality, it is difficult for many client devices using the standard MQTT SDK to connect and enjoy overseas services.

NanoMQ version 0.10 adds built-in AWS bridge functionality to address this issue. NanoMQ is responsible for forwarding the received data of the specified topic to the remote AWS IoT MQTT Broker , and subscribing to the specified topic from the AWS IoT MQTT Broker .

Using this built-in bridging feature requires the AWS IoT SDK to be installed in the default environment:

Install the AWS SDK

 ## 下载源码
wget https://github.com/aws/aws-iot-device-sdk-embedded-C/releases/download/202108.00/aws-iot-device-sdk-embedded-C-202108.00.zip

## 解压
unzip aws-iot-device-sdk-embedded-C-202108.00.zip

## 编译
cd aws-iot-device-sdk-embedded-C
mkdir build
cmake -G Ninja -DBUILD_DEMOS=OFF -DCMAKE_C_STANDARD=99 -DINSTALL_TO_SYSTEM=ON ..
ninja

## 安装到系统
sudo ninja install 
sudo cp ../demos/logging-stack/logging_*.h  /usr/local/include/aws/
sudo ldconfig

Then configure the configuration file /etc/nanomq_aws_bridge.conf in the default path of NanoMQ, or specify the configuration file nanomq start --bridge <nanomq_aws_bridge.conf> when starting NanoMQ through the command line

Configuration parameters

• Bridge mode switch (default false off)

   bridge.mqtt.aws.bridge_mode=true

• AWS IoT Core Endpoint address

   bridge.mqtt.aws.host=a2zegtl0x5owup-ats.iot.us-west-2.amazonaws.com
  bridge.mqtt.aws.port=8883

• MQTT protocol version

   bridge.mqtt.aws.proto_ver=4

• Forward Topic (multiple topics are separated by commas)

   bridge.mqtt.aws.forwards=topic_1,topic_2

• Subscribe Topic

   bridge.mqtt.aws.subscription.1.topic=cmd/topic1
  bridge.mqtt.aws.subscription.1.qos=1
  ...
  bridge.mqtt.aws.subscription.{n}.topic=cmd/topicn
  bridge.mqtt.aws.subscription.{n}.qos=1

NanoMQ can build a bridge with AWS IoT Core for data synchronization.

Since AWS's C embedded SDK does not support cross-platform, this feature is currently not packaged into binary installation packages. Users who need to use AWS bridging need to compile and install the AWS IoT Core bridging function of NanoMQ by themselves through the following commands.

 cmake -G Ninja .. -DENABLE_AWS_BRIDGE=ON

sudo ninja install

Added HTTP API

MQTT is a message protocol designed based on asynchronous mode, but in many scenarios, it still interacts with many third-party applications through HTTP REST API, which requires MQTT service to be compatible and convert asynchronous and synchronous modes at the same time. So NanoMQ in version 0.10 also added support for publishing messages through HTTP API in response to the user's voice.

A simple example of usage is as follows:

HTTP publish message

/api/v4/mqtt/publish

 $ curl -i --basic -u admin:public -X POST "http://localhost:8081/api/v4/mqtt/publish" -d \
'{"topic":"a/b/c", "payload":"Hello World", "qos":1, "retain":false, "clientid":"example"}}'

## NanoMQ回复：
{"code":0}

HTTP Bulk Post Messages

/api/v4/mqtt/publish_batch

 $ curl -i --basic -u admin:public -X POST "http://localhost:8081/api/v4/mqtt/publish_batch" -d '[{"topic":"a/b/c","payload":"Hello World","qos":1,"retain":false,"clientid":"example"},{"topic":"a/b/c","payload":"Hello World Again","qos":0,"retain":false,"clientid":"example"}]'

## NanoMQ回复：
{"data":[{"topic":"a/b/c","code":0},{"topic":"a/b/c","code":0}],"code":0}

NanoSDK supports MQTT 5.0

NanoSDK has previously supported QUIC as the MQTT transport layer to solve problems such as weak network environment and network switching.

In mid-July, this project received a major update: the newly released NanoSDK 0.7.0 version supports MQTT 5.0 and adds more highly packaged APIs for MQTT users. At present, NanoSDK has been able to support most of the features of the MQTT 5.0 protocol.

NanoMQ command line tool separation

There is another change in NanoMQ 0.10 that users need to pay attention to. This version starts to separate the original command line toolbox from the MQTT Broker service and start it from a different command entry.

The original NanoMQ startup command:

 ## NanoMQ Broker：
nanomq broker start
## NanoMQ 客户端：
nanomq pub --url "mqtt-tcp://broker.emqx.io:1883" -t topic -m hello

Startup commands after version 0.10:

 ## NanoMQ Broker：
nanomq start
## NanoMQ 客户端：
nanomq_cli pub --url "mqtt-tcp://broker.emqx.io:1883" -t topic -m hello

Coming soon: MQTT 5.0 + MQTT over QUIC bridge

Thanks to NanoSDK's support for MQTT 5.0 and QUIC transport layer, NanoMQ's bridging function is about to get a major upgrade in version 0.11. Users will be able to choose the MQTT 5.0 version for bridging, or use QUIC as the transport layer for the bridge connection, thus providing transport layer conversion for local end-side devices that cannot integrate QUIC SDK or do not have a suitable MQTT over QUIC SDK selection, so as to play QUIC with EMQX 5.0 Advantages in weak network environment.

For a detailed explanation of QUIC, please refer to: MQTT over QUIC: There are more possibilities for IoT message transmission

Copyright statement: This article is original by EMQ, please indicate the source when reprinting.

Original link: https://www.emqx.com/zh/blog/nanomq-newsletter-202207