From a small glimpse to the big, from a street lamp to see the road to the Internet of billions of things

Abstract: IoT, Internet of Things, the Internet of Things, as the name implies, is the connection between things. To achieve "things connected", there are a total of three phased tasks, and these three phased tasks are also accompanied by huge challenges.

This article is shared from the HUAWEI cloud community " [Cloud-based co-creation] from a small glimpse of the big, from a street lamp to see the road to the Internet of billions of things ", author: Qiming.

The inevitable trend and huge challenge of the Internet of Things

Before introducing Huawei Cloud IoT device access management service business, we need to understand what "IoT" is.

IoT, Internet of Things, as the name suggests, is the connection of things. There are three phased tasks in order to achieve "things connected", and these three phased tasks are also accompanied by huge challenges:

Stage 1: Let the "things" speak.

Just as babies need to be able to speak in order to integrate into the world, our "things" also need to be able to communicate with the outside world. Perceive the physical world and turn it into a digital signal: Let all things perceive temperature, space, and five senses, and digitize what they perceive. In the process of digitizing, we found the following challenges:

• A large number of existing equipment has not been digitized;
• Diversified equipment, various industry protocols, and various communication networks;
• The equipment requires low power consumption and low cost;
• Outstanding equipment safety issues

Phase 2: Let "things" speak a word

Babies can't speak alone. To be able to communicate with the outside world, they also need to use a common language with the outside world. The same is true for "things". We need to turn numbers into online and provide standardized data so that the outside world can understand what the "things" say. This requires wide connectivity and multi-protocol access, data standardization, and a unified object model. However, the status quo is:

• Millions of data elements, lack of standards and specifications;
• Diversified data format
• Various vendors have developed in a chimney style, with numerous isolated data islands, and difficulties in data interoperability

Stage Three: When making "things" valuable

On the basis of being able to communicate in the same language, we also need to allow babies to speak valuable words in order to achieve efficient communication. The same is true for "things". Especially in the "things" world, we need to deal with massive data processing and analysis, so the requirements for computing power are extremely high, and real-time is required. The final result is industry digitization, digital industry. However, as mentioned earlier:

• Massive data brings computing power and storage problems;
• Real-time streaming, timing, offline, multi-dimensional analysis is complex;
• The complexity of industry application scenarios;
• Business closed loop, value transformation and distribution issues

A small glimpse of the big picture, from a street lamp, to see the world of interconnected everything

First of all, let us paint a street lamp in our minds. Don’t you think it’s simple?

Let's review the history of the development of street lamps:

From the time controller in the 1950s and 1960s + local loop control, timing switch, the simplest logic processing relationship; to the 1970s, with the addition of the photoelectric controller, it is a little "smarter" and can sense the intensity of the surrounding light, thus Adjusting the switch, but at this time it was only controlled in the local loop; in the 1990s, it was possible to achieve remote loop control based on the three-remote monitoring system of the wireless private network; finally, to the present, the intelligent system based on the Internet of Things technology has realized Information fusion, smart scenes and application expansion, so as to realize smart lighting scenes (as shown above).

It can be seen that the smart street lamp in the smart scene is equipped with many smart devices, such as monitoring components, LED screens, centralized controllers, electric vehicle charging piles, etc., so that a street lamp can realize the basic functions that a road can achieve.

The above is the wisdom of a lamp. In this process, the Internet of Things technology has played a great role. A single-time control street lamp has multi-scene and multi-application functions step by step, and finally evolved into a smart street lamp. .

The Huawei Cloud IoT device access management service is based on this technology.

Introduction to Huawei Cloud IoT Device Access Management Service

First of all, we must have a correct understanding of the Internet of Things: the Internet of Things is composed of multiple services, not a single service. The figure below is an overall IoT solution.

On the left side of the picture is the device, which reflects what we said earlier: let the device speak, let the device speak the same time, and let the device speak valuable words. Then, we must first make these devices "perceived". In this process, we enable them through the Internet of Things operating system (LiteOS) or SDK to make them intelligent; then the sensed things pass through the gateway in the form of data , Edge gateway or direct upload. Where should I upload it?

Considering the massive problems of equipment, we need to have an IoT platform. The data is uploaded to the device access service in the IoT platform through the aforementioned method. In the device access service, multiple functions such as multi-network access, multi-protocol access, and device shadowing can be realized; and on the overall Internet of Things platform, it also includes services such as global SIM connection, device issuance, data analysis, and so on.

On the right side of the figure, it can be more connected to other HUAWEI CLOUD products, such as DIS/DCS/OBS, etc., to help customers quickly build IoT applications that meet customer requirements, such as smart cities, smart parks, and Internet of Vehicles.

Back to the technology itself. As we talked about earlier, different devices can be accessed through HUAWEI CLOUD device access services, but in fact, their access methods and access protocols are different, so what about HUAWEI CLOUD device access services? What about shielding the difference at the bottom to achieve device access?

Let's first look at the following picture:

communication model: defines the "communication model" in Huawei Cloud Device Access Service. Also take the baby talk we said before as an example. Although there are many languages for communication between people, the way of expression is still mainly voice (speaking) or visual (sign language). And between things, similarly, the main content is mainly in several categories: command issuance, command response, attribute equipment, and so on.

device shadow: and on this basis, we also have the concept of "device shadow". The IoT platform supports the creation of "shadows" of devices. The device shadow is a JSON file used to store the online status of the device, the last device attribute value reported by the device, and the configuration expected by the application server. Each device has one and only one device shadow. The device can obtain and set the device shadow to synchronize the device attribute value. This synchronization can be shadow synchronization to the device or device synchronization to the shadow.

General Protocol Access Layer: After has a "communication model", we need to associate the device data with the communication model, and the association is the "General Protocol Access Layer". Through the communication protocol access layer, we can convert different data on different protocol types (such as MQTT/CoAP/TCP, etc.) to the communication model.

encoding and decoding processing: the device of a product of 1616e3bc1eb6ad reports data, if the "data format" is "binary stream", the product needs to be codec plug-in development; if the "data format" is "JSON", the product No codec plug-in development is required.

Taking the NB-IoT scenario as an example, the CoAP protocol is used for communication between the NB-IoT device and the IoT platform. The payload of the CoAP message is application layer data, and the format of the application layer data is defined by the device itself. Since NB-IoT devices generally have high requirements for power saving, application layer data generally does not use the popular JSON format, but a binary format. However, the IoT platform and the application side use the JSON format for communication. Therefore, coding plug-ins need to be developed for the IoT platform to call to complete the conversion between binary format and JSON format.

HUAWEI CLOUD IoT device access management service cloud architecture interpretation

IoT device access management service architecture

Before explaining the entire framework, we must first make it clear that the inside of Huawei Cloud IoT device access service is actually an architecture based on Cloud Native (cloud native) microservices. All business-related services in it are stateless. A stateless service means that it can be created at any time, or it can be killed at any time.

In other words, even if there is a problem with any service instance, the service can be created to ensure that the business is not damaged. At the same time, in the case of massive concurrency, through flexible shrinking and expansion, the service can meet the needs of the corresponding business volume.

In the framework based on stateless services, in addition to the load balancing layer (the purpose is to allow a large number of devices to access the service), the above can be divided into two layers:

The first layer is the CIG cloud communication gateway layer. This layer actually converts the different protocols and different data formats of various devices into a unified model;

The other layer is to connect & manage business services: data and protocols are converted into a unified model, and after being transferred to this layer, business services will do the processing, such as message flow push, device shadow, rule engine service, and task management service and many more.

The upper layer is the public cloud support management service: through public cloud support services, such as CBC/IAM/SMN, etc., the overall construction of device access management services.

The above is a general framework of equipment technology management services. Through this framework, we can understand the internal business processing logic and help everyone understand our subsequent sandbox experiment experience.

Product model (Profile)-unified data & communication model

The product model, also known as the Profile, is simply the mapping of the communication model.

The figure above is an example of how to map different types of data to the communication model.

On the left side of the figure, there are 3 different meters. Although they are all water meters, they are produced by different manufacturers, so the data format and protocol are also different, so how do we collect their data to form a unified format?

answer to 1616e3bc1eb7cd is the product model. can convert these different data into data in the same format through different plug-ins in the product model. For equipment manufacturers, there is no need to modify equipment codes, and plug-ins can be used to adapt to the demands of different companies, reducing the maintenance complexity of equipment manufacturers. For upper-layer applications, the complexity of device management by upper-layer applications is reduced. By organizing data from different manufacturers on the platform, a unified profile is formed, and upper-layer applications do not need to perceive the differences between different devices.

Next look at the rules engine. rule engine is a unique capability of HUAWEI CLOUD IoT device access services. Through the rule engine, data flow and device linkage can be realized. Seamlessly connect with other public cloud services to realize the on-demand transfer and processing of device data;

The device linkage is also realized based on the rule engine, which is triggered by conditions and based on preset rules to trigger coordinated responses of multiple devices, such as reminders, alarms, status reports, and so on.

HUAWEI CLOUD IoT Smart Street Light Sandbox Experiment Demonstration

After introducing the HUAWEI CLOUD IoT device access management service, I will then take you to learn about related sandbox experiments.

Introduction to Sandbox Experiments

The network structure of the smart street light sandbox experiment is as above. In order to make it easier for everyone to complete the road test online, we need to have a virtual device, here we use a device simulator. The device accesses the platform, reports the attributes, and then receives the commands issued by the platform. In the upper layer of the platform, there is an implementation of the application layer based on Huawei Cloud SoftKing. The general steps are as follows:

1. Build a street lamp simulation application through the HUAWEI CLOUD software development platform (the real application can refer to its construction method);

2. Apply for ECS cloud service, deploy the built application on ECS, connect to IoT device access service through EIP, and subscribe to notifications;

3. Run the MQTT simulator on the experimental desktop to simulate the behavior of street lighting equipment

Device simulator

Before officially starting, let's take a look at the device simulator (as shown below).

connection parameter configuration area:

After the device simulator is opened, open the "MQTT Device Simulator" tab. In the "connection parameter configuration area", you need to specify the address information to enter the platform, get the "registered device successfully" or the device ID and device key, and establish a connection with the platform;

message reporting area:

After establishing a connection with the platform, the device reports data to the platform. There is a "message report area" below. After the specified device reports the attribute topic, paste the jason format data below and report it;

Log Print Area:

On the right is a log print area similar to message tracking, where you can see the message records reported by the message, including the commands issued by the platform, and a print tracking will also be done here.

Topic subscription area:

The Topic subscription area is another tab. When the device needs to subscribe to the topic related to the command issued by the platform, it can specify the topic here to subscribe.

Application Implementation-API Introduction

In the experimental desktop, after creating the code repository, you can view the detailed source files. The more typical implementations of several source files are the implementation of application-side APIs, such as:

Create device API: src/main/java/com/huawei/service/RegisterDirectConnectedDevice.java

Create subscription API: src/main/java/com/huawei/service/SubscribeDataChg.java

API for issuing device commands: src/main/java/com/huawei/service/CreateDeviceCommand.java

Based on the sample code of these source files, developers can do customized development based on their own ideas.

The key file of the front-end web page layout is an index html source file, plus a js script:

Page layout: src/main/resources/static/index.html

Script file: src/main/resources/static/js/common.js

Practical steps of smart street light sandbox experiment

The smart street light sandbox experiment is divided into 7 steps:

1. Compile and build a smart street light application DevCloud

2. Create a topic and subscribe to SMN

3. Create an elastic cloud server ECS

4. Deploy DevCloud, a smart street light application

5. Import product model IoTDA

6. Configure the smart street light application Web

7. Result verification

After the developer logs in to the HUAWEI CLOUD account and enters the sandbox experiment, there will be the above page:

The experimental account is displayed in the upper left corner. The developer uses this experimental account to log in to HUAWEI CLOUD through the firefox browser in the experimental desktop, and access various services through the console.

There is a step by step experiment step guide under the account, just follow the steps.

There is a completion progress display in the upper right corner, refresh in real time, you can get progress display and experiment countdown.

In the left part, there are some related commands to be executed, which can be convenient for everyone to quickly copy when they are executed.

Due to time constraints, we omit the simple steps 1 to 4, and start from step 5 and then create the product. Normally, we will follow the formal experiment process.

PS: The first thing you need to pay attention to in the first 1~4 steps is that when you create a topic and subscribe, the platform will send a confirmation SMS to the user's mobile phone. The user needs to view the SMS, click on the link inside, and confirm the actual. subscription.

Next, we formally enter the experimental operation:

Import product model:

Create a product

In the HUAWEI CLOUD console, expand the left navigation and select "Service List" -> "Internet of Things" -> "Device Access IoTDA", as shown in the following figure:

Select "Product" in the left column, click "Create Product", the parameters are as follows:

①Owned resource space: Default_APP_Sandbox_user_iot

②Product name: custom, such as: StreetLight

③Protocol type: MQTT

④Data format: JSON

⑤Vendor name: custom, such as: Huawei_IoT

⑥Equipment type: custom, such as: StreetLight

After completing the filling, click "Create Now", the product is created successfully, please record the "Product ID" information, and check the "Details".

Upload model file

Click the created product to enter the details, then click "Upload Model File", select the SmartStreetLight.zip file, and click "Confirm" to complete the upload.

Configure smart street light application (subscription push plan)

Access the Smart Street Light web interface

Create a new browser tab, enter http://EIP:8080/index.html (replace "EIP" in the address with the elastic public network IP of the elastic cloud server in step 3.2) to access the smart street light web page.

parameter settings

In the pop-up "Parameter Setting" window, fill in the parameters as follows:

①Data source: select "Subscription Push" in the drop-down box on the parameter setting page.

②Local IP: "Control Panel" -> "Service List" -> "Compute" -> "Elastic Cloud Server ECS" enter the server list, check and copy the elastic public network IP.

③Subject URN: "Control Panel" -> "Service List" -> "Management and Supervision" -> "Message Notification Service SMN" Enter the subject list, view and copy the subject URN.

④APPID: "Control Panel" -> "Service List" -> "Internet of Things" -> "Device Access IoTDA", select "Resource Space" in the left navigation bar, and copy the "APPID" required to obtain the product.

⑤Account name: The experimental account name of HUAWEI CLOUD assigned by the system.

⑥ IAM user name: The experimental user name of HUAWEI CLOUD assigned by the system.

⑦IAM user password: the Huawei Cloud experiment password assigned to you by the system.

⑧Project ID: In the user name drop-down box in the upper right corner of the console, select the "My Credentials -> API Credentials" page to copy and obtain the project ID of the "North China-Beijing 4" area.

⑨Product ID: "Control Panel" -> "Service List" -> "Internet of Things" -> "Device Access IoTDA", select "Product" in the left navigation bar, and copy the "Product ID" required to obtain the product.

After filling in, click "OK", the pop-up window prompts as follows, indicating that the parameter setting is successful.

Register the device

Enter the device ID (customized, such as 123456789f), and click "Register Device".

The pop-up window prompts as follows, indicating that the device is successfully registered. Please record the "Device Id" and "Key" information for subsequent verification.

Note: Please record the device id and key for later use!

After the device is successfully registered, it is in the "inactive" state, please select "Service List" -> "Internet of Things" -> "Device Access Service", select "Device> All Devices" in the left column, and click the device identification code to enter, View the device ID, or modify the key.

Result verification (device attribute report/subscription/device command issuance)

The simulator establishes a connection with the platform

Open the MQTT device simulator

Double-click "Xfce Terminal" on the experimental desktop to open Terminal, execute the following command to open the device access simulator (HubSimulator.jar), and then switch to the MQTT device simulator tab.

establish connection

On the MQTT device simulator tab, fill in the parameters as follows:

①Service IP: Go to "Service List" -> "Internet of Things" -> "Device Access IoTDA", select "Overview" in the left column, and click "Platform Access Address".

Copy the domain name using MQTT/MQTTS access information.

② Device ID: Configure the smart street light application, the device ID returned after the device is successfully registered

③Device key: configure the smart street light application, the key returned after successful registration of the device

After configuring the above parameters, click the "Establish Connection" button.

After the connection is successfully established, the log on the right shows that the MQTT service is successfully connected, as shown in the following figure:

If the connection cannot be established, open the "Command Prompt" window and enter "ping service IP" to check whether your host can communicate with the HUAWEI CLOUD IoT platform normally. If it cannot communicate normally, please change the host's network environment.

View device status

Back to the list of all devices in the Internet of Things ("Service List" -> "Internet of Things" -> "Devices" → "All Devices"), the device status changes to "Online", as shown in the figure below:

Simulator device attribute reporting

The simulator sends a message to report the device properties.

①Enter the following code in the Topic column of the simulator window: $oc/devices/{device_id}/sys/properties/report, where {device_id} needs to be replaced with the actual device ID.

$oc/devices/{device_id}/sys/properties/report

②Message content:

Among them, the light intensity can be customized, and the light status is recommended to be OFF.

Enter the following code in the message edit bar in the simulator window:

{"services":[{"service_id":"Sensor","properties":{"luminance":"200","light_state":"OFF"}}]}

After the message is sent successfully, the log on the right shows that the information has been pushed successfully (the picture is only for illustration, please refer to the actual reported data).

View application interface changes

Return to the smart street light web interface, and at this time, the "Environment Brightness" is displayed as 200LUX, and the "Light Status" is off.

Note: There will be a short delay between the simulator and the web page. You don't need to refresh the page, you can find that the page will change after a few seconds!

Simulator subscription Topic

Switch to the "Subscribe Topic" tab and enter the following code in the Topic column:

$oc/devices/{device_id}/sys/commands/#, where {device_id} needs to be replaced with the actual device ID.

After entering, click the "Subscribe" button to complete the subscription.

$oc/devices/{device_id}/sys/commands/#

After the subscription is successful, the log on the right shows that the topic is subscribed successfully.

The platform issues device commands/emulator responses

Terminal joint control mode configuration

Return to the Smart Street Light Web interface, select "Terminal Joint Control Mode" for "Street Light Control", select "On" for the switch, click "Settings", and then click "Confirm".

①If the current light status is "off" and the switch is selected as "on", the service background will issue a light on command in real time, and the pop-up prompt will have a delay of about 5-10 seconds, please wait patiently.

②If the current light status is "off" and the switch is selected as "off", the command will not be triggered, and a pop-up window will prompt you that the setting is successful.

That is, when the "light status" is consistent with the "switch selection", the command is not triggered; when the "light status" is inconsistent with the "switch selection", the real-time command is triggered.

Command received

Because the current "light status" is off and the "switch selection" is on, check the simulator log area and you will receive a light on command. (The value of the payload is encrypted by base64, and the command content can be obtained after decryption).

The simulator sends a message to report the changed device properties.

①Topic: $oc/devices/{device_id}/sys/properties/report, where {device_id} needs to be replaced with the actual device ID.

②Message content:

Among them, the light intensity can be customized, and the light status is recommended to be ON.

{"services":[{"service_id":"Sensor","properties":{"luminance":"110","light_state":"ON"}}]}

View application interface changes

As shown in the figure below, it should display "Environment Brightness" as 110LUX and "Lamp Status" as ON.

SMS/email notification

After clicking "disconnect" on the simulator, repeat 7.4.1, select "terminal joint control mode" for "street light control", select "off" for the switch, and click "settings".

Because the light status is "on" and the switch is selected as "off", the service background will issue a light-off command in real time; and the device is offline, causing the command to fail.

At this time, the system will send a SMS/email notification to notify the subscriber that the command is failed; the content of the notification is the error message that the command failed.

Notification example: [Huawei Cloud] Error message: {"error_code":"IOTDA.014016","error_msg":"Operation not allowed. The device is not online."}

To change the device to the "offline" status, please log in to the HUAWEI CLOUD console, select "Internet of Things> Device Access Service", and in the left navigation bar, select "Device> All Devices" to view the device status based on the device identification code.

Simulator receives response

Because the current "Environment Brightness" is 110LUX and the "Light Status" is on, check the simulator log area, you should receive a light off command.

①If the current "ambient brightness" is greater than the "brightness threshold" and the "light status" is "on", the service background will issue a light off command in real time, and the pop-up prompt will have a delay of about 5-10 seconds, please wait patiently.

②If the current "ambient brightness" is less than the "brightness threshold" and the "light status" is "on", the command will not be triggered, and a pop-up window will prompt you that the setting is successful.

That is, when the "ambient brightness" is less than the "brightness threshold", and the "light status" is "on" or the "ambient brightness" is greater than the "brightness threshold", and the "light status" is "off", the command is not triggered; vice versa , It will trigger the issuance of real-time commands.

The simulator sends a message to report the changed device properties.

①Topic: $oc/devices/{device_id}/sys/properties/report, where {device_id} needs to be replaced with the actual device ID.

②Message content:

Among them, the light intensity is less than 100, and the light status is OFF.

{"services":[{"service_id":"Sensor","properties":{"luminance":"80","light_state":"OFF"}}]}

View the web interface

Return to the smart street light web interface, click "Settings", view the simulator log area, and you will receive a light-on command.

Delete resource

Delete ECS

In the HUAWEI CLOUD console, select "Service List" -> "Compute" -> "Elastic Cloud Server ECS" to enter, check the ECS server to be deleted, such as ecs-iot, and select "More> Delete", in the pop-up window Please tick to delete the flexible public network IP and data disk, and then click "Yes" to complete the deletion.

Delete device access service IoTDA

In the HUAWEI CLOUD console, select "Service List" -> "Internet of Things" -> "Devices" → "All Devices" to enter the list of all devices, and delete devices based on the device identification code, such as 123456789f. Then select "Product" in the left column to delete the product, such as StreetLight.

Delete the resource pack downloaded from the desktop

Double-click "Xfce Terminal" on the desktop to open Terminal, and enter the following command to download and delete the "HubSimulator.jar" file.

cd Desktop/

rm HubSimulator.jar

Next delete the "SmartStreetLight.zip" compressed package. The command is as follows:

rm SmartStreetLight.zip

So far the experimental experience is over.

Summarize

HUAWEI Cloud Device Access Service (IoT Device Access) is the IoT platform of HUAWEI CLOUD. It provides a large number of devices connected to the cloud, two-way message communication between devices and the cloud, batch device management, remote control and monitoring, OTA upgrades, device linkage rules, etc. , And can flexibly transfer device data to other Huawei Cloud services, helping IoT industry users to quickly complete device networking and industry application integration. The newly upgraded device access service IoTDA integrates the functions of the original device access service and device management service.

If you are interested, come and experience it together~ https://www.huaweicloud.com/product/iothub.html

Click to follow, and learn about Huawei Cloud's fresh technology for the first time~