Streaming Media: Relying on Soundnet's Lianmai Solution

1. Background

In recent years, live broadcast Lianmai has been burning for 6 years in the field of streaming media. From the simple exploration at the beginning to the current mature full-link solution, it has to be said that the increasingly fierce competition has turned the original blue ocean field into a deep-sea mutual battle. In such a big environment, does it mean that small factories will no longer have the opportunity to chase the air of the streaming media industry and fight for the big with small ones? The answer is of course no. Thanks to the collision of technological ideas brought about by the market drive over the years, a group of technology suppliers specializing in communications have been born. It enables any organization to achieve streaming media empowerment through a reasonable design of the opposite end plan. As such a partner, Shengwang is one of the best.

This is what this article will talk about: the sound network-based peer-to-peer connection solution.

Second, the macro process

Before we officially start, the first thing we need to do is to divide the stage roles according to the participants of Lianmai and the stage of Lianmai. From the perspective of the overall process, for any two ends of the microphone, oneself is the anchor end, and the other party is the microphone end. Therefore, the only difference is not in the identity, but in who initiated the Lianmai request first. This will determine the complete link flow of the entire Lianmai from initiation to connection.

Therefore, we have the basic framework requirements:

1. Starting from being connected to the microphone, build a complete link to establish the room, start connecting the microphone, connect the microphone, and end the connection

2. The business layer processing flow, eliminating the rapid integration scheme realized by the underlying complex technology

3. Robustness with high call-through rate, ensuring timely and accurate requests from an implementation perspective

According to these three core demands, we provide the following Lianmai solutions.

Figure 2-1 Macro process description

Next, let's take a look at it separately.

Three, stage dismantling

Anchor start stage

For the anchor start-up stage, it is mainly based on the feature data of the current anchor service side to determine whether the anchor can open the mic-linked live broadcast room. Why is there a distinction between whether the live broadcast room is a Lianmai live broadcast room? It mainly involves more requirements for each link in the entire link of the Lianmai live broadcast room. On the one hand, it is not expensive for the host to make a distinction. On the other hand, it can also reduce the server's recording of such live broadcast rooms and help maintain a more appropriate pre-preparation strategy resource pool size.

When the host requests to start the broadcast, the broadcast address request sent to the server will be screened by the server according to the broadcast policy to open the live room type. According to the judgment result, the server will choose whether to open the live broadcast room with microphone extension capability and return its streaming address to the host. The host can start streaming with this address.

Figure 3-1 Schematic diagram of key links in the start-up stage

Lianmai launch stage

After the host opens the live broadcast room, all peers that meet the requirements for mic connection can initiate a mic connection request to the host. There may be competition between the party requested by Lianmai. Therefore, for the party who initiated the request first, before the host or the host did not explicitly deny the event for objective reasons. We should bind the anchor and the microphone to avoid bypass interference.

Of course, before establishing the Lianmai relationship, we need to verify the qualifications of the Lianmai party. The purpose is to ensure the actual effectiveness of the wheat-linking activities at that time. Otherwise, if we assign a mic-link relationship to those who do not meet the qualifications, it will be equivalent to a disguised preemption of the mic-link authority that is scarce for a single live broadcast room. This will bring a lot of security and management risks. At the same time, in order to avoid launching Lianmai Duan, I regretted the operation of Lianmai Duan within a relatively short period of time after the initiation. We need to set aside some time to provide a "return window period". The specific threshold of this part of the time strategy is defined, and the product needs to be determined by data analysis based on user characteristics.

Once the connection between wheat and wheat is determined, it is necessary for the wheat to enter the "waiting for wheat and wheat" state. At this time, the host will receive the request to connect to the microphone. According to the host's decision to agree/reject, decide whether to establish a Lianmai live room.

If the host chooses to refuse, the current mic-linking relationship is released, and the party that initiates the mic-link will receive the single push and get the reason for the failure, and other users can also re-compete for the mic-link.

If the host agrees, the resources pre-processed in the pre-preparation strategy will be directly allocated to the currently established mic-link activity. In this regard, the original live broadcast room was upgraded to a Lianmai live broadcast room. The server will start the dynamic switching strategy of the push stream address, and return the associated information to the push stream double end to complete the sound network link initialization.

Figure 3-2 Schematic diagram of key links in the initiation stage

The entire Lianmai launching stage is basically seamless for users. This is mainly achieved through mixed push streaming. Before the host connects to the microphone, the host is equivalent to directly pushing the local stream to the push server, and the push server performs subsequent operations including transcoding and distribution. When Lianmai was established, the two-end push stream was first mixed through the sound network server. After that, we forward the mixed stream to our own streaming server through the host push address that we passed. This is equivalent to replacing the direct push stream of our original anchor with the stream processed by the sound network, and the server only needs to complete part of the time stamp verification synchronization service, which reduces the impact of lag caused by stream switching.

Malting process stage

When the host and the mic-link party have established a complete connection, they enter the stage of the mic-link process. At this stage, in terms of heartbeat maintenance and push-pull streaming, in order to ensure a stable link and a certain degree of interaction scalability, a set of independent heartbeat requests should be maintained in addition to the judgment of the IM message of the sound network itself. Through the heartbeat request, the double-ended timestamp is sent to the server to synchronize the actual duration of the connection with the microphone. At the same time, it is also helpful for the server to collaborate with both ends on some key data according to the specific situation. On the other hand, if the strong binding mechanism is not used, that is, when the relevant stream push part such as the push stream CDN is a third-party provider that is not a sound network, we also need to obtain and judge the current network status through a heartbeat request. Of course, in addition to the heartbeat request method, there are other methods such as control flow message customization that can achieve the same effect. This way of keeping alive is relatively quick, convenient, and less expensive, and does not need to maintain long links, but requires careful security handling of requests.

Figure 3-3 Schematic diagram of key links in the wheat-linking process

In Lianmai, there are often some interactive activities on both ends. For operations similar to rewarding, it is recommended to implement it through the message pipeline of the live broadcast room. In this way, there is no need to independently maintain a set of self-built IM for two-way two-terminal message transmission, but can directly use the existing message pipeline in the live broadcast room to realize two-terminal communication. In this way, no matter how the live broadcast room is switched, the message pipeline will continue to exist and does not need to be restarted, ensuring the experience of the host and the audience. In fact, if there are no special customized operations, even the message processing of the microphone is the same as the rest of the audience, that is, the message logic is consistent.

Mic disconnection stage

So when will Lianmai end? There are actually many situations for this. The most common is the active disconnection of the anchor or the microphone, in addition to the disconnection of network fluctuations and the disconnection of server policy. Generally speaking, it can be divided into two categories, namely active disconnection and passive disconnection.

Active disconnection refers to a short-term operation unilaterally or agreed upon by one or more of the two ends participating in the connection.

Passive disconnection refers to the termination of the connection due to external interference or the influence of server strategy.

For passive disconnection (such as disconnection, insufficient cost, continuous disconnection, etc.), it is best for us to have sufficient strategies for fault-tolerant processing before the problem occurs. The purpose is to ensure that the live broadcast room can be restored to normal after disconnection, without causing robustness problems.

Figure 3-4 Schematic diagram of key links in the disconnection of microphones

In Lianmai, we use Soundnet as the real push-streaming terminal. The Soundnet mixed-stream server pushes the mixed data to our self-built/third-party push-stream server and distributes it to the CDN. When one end exits, the exit message will be pushed to the opposite end along with the video stream (Soundnet encapsulates the corresponding status message on the key field in the data stream to ensure the consistency of the linking party’s picture and the end of the receiving operation) . After receiving the message, the opposite end also needs to send a room exit message to the currently occupied Shengwang live broadcast room, so that the Lianmai room can truly become a double-end unrelated room (sometimes there are audit requirements, and there may be other than Lianmai in the Lianmai room. Multiple users outside the two parties, these users belong to the server-side review robot or manual account), and then the server-side releases the room resources and returns them to the resource pool, or redistributes them to a new scene.

When the microphone connection is over, the host chooses whether to request the server's live room address update (personal suggestion to implement, let the server choose whether to update), and switch the current push stream address to the acquired address after obtaining it. The Lianmai party closes the streaming module, restarts the streaming module, and re-enters the host’s live broadcast room of the Lianmai.

At this point, the Lianmai process is complete.

Four, summary

Compared with the large amount of work to independently develop the mic-link function, adopting the hybrid implementation scheme we introduced can not only ensure the low bundling and high compatibility of the overall framework, but also reduce the risk and cost with the help of the sound network's mature communication processing system. At the same time, the loose coupling with all third parties also means that we can replace in terms of access at any time.

However, I believe that Soundnet will continue to use stable and efficient network and professional sinking technology to make you sigh: It is more efficient and convenient to use Soundnet directly. At that time, the self-developed hybrid method of connecting the microphone with the sound net and connecting the microphone may not be an elegant choice.