Aurora notes|Integrated SMS test pile based on CMPP protocol whole process practice sharing

Foreword:
Aurora SMS is a part of Aurora Push’s business. The biggest advantage of Aurora SMS is to re-send SMS for users who have not delivered the push message, which not only ensures the reach of the message but also saves costs, and requires the SMS function for authentication SMS or other business scenarios. .
The bottom layer of the Aurora SMS service is the real docking operator, which uses the cmpp protocol to send SMS. In the test environment, it is impossible to verify the SMS function in the real environment. The reasons are:
1. The test needs to send a lot of text messages, and the cost will be high
2. It is necessary to simulate various abnormal messages, and the real operator has many rules to trigger the abnormality
3. It is necessary to quickly return various errors to the operator, the real operator may be delayed, which is not conducive to testing
4. Various scenarios such as speed limit settings for large amounts of data are also required
Therefore, in view of the above 4 points, we finally chose to "create" an operator by ourselves to process the data sent by the data client to meet the daily SMS service test
Before the simulation, we need to confirm the underlying protocols, interaction methods, and data processing methods used by operators in order to better establish our own "operator."

Term explanation

1. Know the cmpp protocol

1.1 Network structure

As shown in Figure 1, the Internet SMS Gateway (ISMG) is an intermediary entity between the external information resource station entity (SP) and the SMS center in the mobile network. SMS center. On the other hand, the information of the mobile user's on-demand SP service will be sent to the SP by the short message center through the Internet short message gateway.

1.2 Overview of CMPP function

The following content comes from the introduction of the cmpp2.0 document
Short Message Mobile Terminated (SM MT) typical business operation example is shown in Figure 3:

Figure 3 Schematic diagram of MT that needs to be forwarded

1. The SP sends a data request (may be a short message notification or mobile phone ringtone, etc.), which is received by the source ISMG;
2. The source ISMG returns a response to the received information;
3. The source ISMG cannot find the gateway code corresponding to the mobile phone number segment in the local database, and sends routing request information to GNS (tandem gateway);
4. The tandem gateway returns the routing information;
5. The source ISMG forwards the request to the destination ISMG according to the routing information;
6. The destination ISMG returns a response to the received information;
7. The purpose ISMG sends the requested information to the SMC;
8. SMC returns a response to the target ISMG;

2. Protocol stack

The CMPP protocol uses TCP/IP as the underlying communication bearer, and the specific structure is shown in Figure 2:

Figure 2.1 CMPP protocol stack
ICP and ISMG establish a TCP connection in Clientmdash; Server mode for mutual submission of information between the two parties. After the TCP/IP connection is established, the Client initiates the establishment of an application layer connection. At this time, if the ICP or ISMG thinks that a TLS connection is needed, the TLS field is set in the transmitted data packet to establish a TLS connection between the two parties.
In the data transmission process after the application layer connection is established, if the ICP or ISMG needs to send encrypted information to the opposite end, a TLS connection can also be established. At this time, you only need to set the Tls_available (TLS layer) attribute field in the corresponding message body. Other attributes in the message body of this message will not be sent.
Messages are sent concurrently, with sliding window flow control, that is, if the receiver receives more messages than the size of the window before replying, it will be rejected. The flowchart is shown in Figure 3:

2.1. Message definition

The message definition is the "communication method" between the client and the server. The following is the definition of the cmpp protocol content (one of the authentication protocols)
Basic data type


Message structure

Message header format (Message Header)

CMPP_CONNECT message definition (SPàISMG)

CMPP_CONNECT message definition


CMPP_CONNECT_RESP message definition (ISMG à SP)

3. Build the server

3.1 Analysis of SMS test pile architecture

Business processing flow. As shown in Figure 3.2

Figure 3.2
Processing flow description
1. The server receives a request data with a length of 39 bytes. Since the header is fixed at 12 bytes, the remaining 27 bytes
2. According to the 27-byte length of the above protocol, it is necessary to establish a connection
3. Get the user name and password of the client's connection, and parse the AuthenticatorISMG field to verify whether the authentication is successful
4. Successfully proceed to the next step (requires a response to the successful authentication), after the connection is established, the acquired data is processed, and the content is returned
We extract its main code with the authentication part: Take out the header data packet

Obtain the main content, authentication content

Log display

3.2 Effect display

The SMS test pile has been running in the test environment for more than 2 years, helping the company save a lot of test costs. The following is the content of its display

Displayed on iportal, sending success status.

Displayed on iportal, sending failed status.

Show the sent status on iportal.

Show on Iportal, the state of insufficient balance

3.3 Problems encountered by SMS simulation pile
1. The amount of requested data sent has exceeded the window size
When the number sent by the client exceeds the processing size, the server will limit the flow, queue the task and return the response code to reduce the pressure on the server.
2. The code establishes invalid links and wastes resources

A connection is established and a connection is stored. When the connection is invalid, the connection needs to be deleted. How to determine that the connection is invalid? The heartbeat connection is not established or there is no message sent within 20 minutes after the connection is established (test environment). deal with
3. Language options, and plan design
You can choose Java and Python languages. Considering the high degree of use of Python in the company, choose Python
The Python class has the struct module, the user parses byte data, uses gevent to solve the coroutine, or the multi-process provided by Python
4. Message sticky package
When reading data, read the data in the ioArgs buffer in sequence according to the length information of the header. If the length requirement is not met, continue to read the next ioArgs to solve the sticky packet and half packet problem.
We gradually improve the test stub. It not only has the function of analyzing data, sending reports, keeping heartbeat, and processing different error codes. We also increased the processing speed. It used to process 10 in 1 minute, and now it can process 300 in 1 minute, which satisfies us. An increase in the speed of large-scale data has perfected our testing in various scenarios.