For a copy of mock data, the road to salvation for Protobuf is opened

Recently, I am making a requirement that needs to interact with the back-end. In order to develop in parallel, the following dialogue is generated with the back-end:

Front end: Old iron, can you give a copy of mock data?

Backend: Mock data is too much trouble, do it yourself! ! !

Front end: How do I know what the data looks like and how to mock it! (pitful)

Backend: Just follow the agreed interface mock, and throw me a proto file directly

Front end: At this time, he is already confused. What is proto? How to mock a piece of data based on proto? Why use proto in the backend? Is JSON not fragrant? In order to make up for the missing link, Protobuf opened the road to redemption.

2. What is Protobuf?

Protobuf, as a cross-platform, language-independent, and extensible method of serializing structured data, has been widely used in network data exchange and storage. It currently supports a variety of development languages (C++, Java, Python, Objective-C, C#, JavaNano, JavaScript, Ruby, Go, PHP). For details, please refer to ( https://github.com/52im/protobuf ). It has the following advantages and disadvantages:

1. advantage

(1) Small size after serialization, suitable for network transmission

(2) Support cross-platform and multi-language

(3) Good upgrade and compatibility (with backward compatibility, after updating the data structure, the old version can still be compatible)

(4) The speed of serialization and deserialization is faster

2. shortcoming

Protobuf is a binary protocol, and the encoded data is poorly readable

Third, the structure of Protobuf

There are many uses of Protobuf. This time I will use an example to see its basic use. For specific use, you can search for relevant documents on the Internet for learning.

syntax = "proto2"
package transferData;

message transferMessage {
    required string name = 1;
    required int32 age = 2;
    enum SexEnum {
        Boy = 0;
        Girl = 1;
    }
    optional SexEnum SexEnum = 3;
}

1. syntax = "proto2";

This line is used to specify the syntax version. There are currently two versions proto2 and proto3. The two versions are incompatible. If not specified, the default syntax is proto2.

2. package transferData;

The package name used to define the package;

3. message

Message is the most basic data unit in Protobuf, in which you can nest message or other basic data type members;

4. Attributes

Each line in the message is an attribute, for example required string name = 1 , its composition is as follows:

Label	type	attribute name	attribute sequence number	[options]
required	string	name	= 1	Some options

(1) There are three types of labels:

required: required attributes;

optional: optional attribute;

repeated: Repeated fields, similar to dynamic arrays;

(2) There are many types, each language is different, for example: int32, int64, int, float, double, string, etc.;

(3) Attribute name: the name used to characterize the attribute;

(4) Attribute sequence number: In order to improve the function definition of data compression and selectability, protobuf needs to be defined in order, and no repetition is allowed;

(5) [options]: protobuf provides some built-in options to choose from, which can greatly improve the scalability of protobuf.

5. enum

When defining the message type, you may need a certain field value to be one of some preset values, and then the enumeration type will be able to play a role.

Note: There are many uses of protobuf, only a brief introduction is given here, and students who like it can learn more by themselves.

Four, actual combat

After talking so much, let’s enter the actual combat link. The actual combat will be in the node operating environment, build a TCP connection, and then the client will send the content serialized by Protobuf to the server, and then the server will parse the information after receiving the information. The proto The serialization and deserialization of files will use the protobuf.js package, which is a pure JavaScript implementation and supports node.js and browsers. It is easy to use, extremely fast, and can use files out of the box! ( https://www.npmjs.com/package/protobufjs)

4.1 Basic use

The protobuf.js package is used to parse the .proto file this time. The commonly used methods are mainly as follows:

1. load()

Use this function to load the corresponding .proto file, after the loading is completed, the message inside can be used and subsequent operations can be performed;

2. lookupType()

After loading the .proto, the message used needs to be initialized, that is, the process of instantiating the message is completed;

3. verify()

This function is used to verify that the ordinary object is a certain message structure that satisfies the corresponding;

4. encode()

Encode a message instance or a common js object that can be used;

5. decode()

Decode the buffer to a message instance, if the decoding fails, the error will be eliminated;

6. create()

Creating a new message instance from a series of attributes is better than creating a message from fromObject because it does not produce redundant conversions;

7. fromObject()

Convert any invalid ordinary js objects into message instances;

8. toObject()

Convert a message instance to an arbitrary ordinary js object.

There are some other ways to use the library, you can learn by looking at its corresponding documents. The above conversion relationship is shown in the following figure (from official document ):

4.2 Server

It is the server side. After receiving the message sent by the client, it uses the decode function in the protobufjs library to parse and obtain the parsed result.

const net = require('net');
const protobuf = require('protobufjs');

const decodeData = data => {
    protobuf.load('./transfer.proto')
    .then(root => {
        const transferMessage = root.lookupType('transferData.transferMessage');

        const result = transferMessage.decode(data);
        console.log(result); // transferMessage { name: '狍狍', age: 1, sexEnum: 1 }
    })
    .catch(console.log);
}
const server = net.createServer(socket => {
    socket.on('data', data =>{
        decodeData(data);
    });

    socket.on('close', () => {
        console.log('client disconnected!!!');
    });
});

server.on('error', err => {
    throw new Error(err);
});

server.listen(8081, () => {
    console.log('server port is 8081');
});

4.3 Client

It is the code corresponding to the client, which uses the protobufjs library to perform corresponding operations and sends the serialized content to the server.

const net = require('net');
const protobuf = require('protobufjs');

const data = {
    name: '狍狍',
    age: 1,
    sexEnum: 1
};

let client = new net.Socket();
client.connect({
    port: 8081
});

client.on('connect', () => {
    setMessage(data);
});

client.on('data', data => {
    console.log(data);
    client.end();
});

function setMessage(data) {
    protobuf.load('./transfer.proto')
    .then(root =>{
        // 根据proto文件中的内容对message进行实例化
        const transferMessage = root.lookupType('transferData.transferMessage');

        // 验证
        const errMsg = transferMessage.verify(data);
        console.log('errMsg', errMsg);
        if (errMsg) {
            throw new Error(errMsg);
        }

        // 转换为message实例
        const messageFromObj = transferMessage.fromObject(data);
        console.log('messageFromObj', messageFromObj);

        // 编码
        const buffer = transferMessage.encode(messageFromObj).finish();
        console.log(buffer);

        // 发送
        client.write(buffer);
    })
    .catch(console.log);
}

1. If you think this article is good, share it, like it, and let more people see it

2. Welcome to pay attention to the front-end point, line and surface , and learn the "front-end 100 questions" together to open the road to front-end salvation.