"Easy+" Open Source | Mobile End of NetEase Conference Open Source

Introduction:

The NetEase conference component is now open source, and the source code has been uploaded to Github. This project is self-developed by the NetEase Yunxin team, and a conference system is built by combining the related communication functions of the NetEase Yunxin system, real-time audio and video, instant messaging, whiteboard, live broadcast and other functions. Practice landing.

Text｜Zhao Chong NetEase Yunxin Senior Mobile Development Engineer

1. Background

In recent years, online conference systems have become more and more popular, and major enterprises have gradually begun to use related conference systems. As a communication PaaS service provider, NetEase Yunxin actually has a set of conference system related functions (real-time audio and video, whiteboard, chat, live broadcast, etc.), but we do not have a relatively complete conference system, so we hope to be able to create a network based on NetEase. At the same time, we also hope that through open source, other companies in the industry can easily have the ability of efficient and stable conference system, and can develop a ZOOM of their own within a week.

2. Thinking and Challenges

(1) Language diversity

The mainstream native languages ​​of mobile terminals are not the same. At present, Android is mainly JAVA and Kotlin. iOS requires support for the Object-C and Swift languages. In addition to the official native, many large factories and small and medium-sized Internet companies also reserve the current popular cross-platform development, and there is also a great demand for cross-platform efficient development. In order to meet business needs and pain points, NetEase Conference uses Google's official Flutter cross-platform technology for development.

​

The main purpose of choosing Flutter is to empower business and improve efficiency, reuse a set of code at multiple ends, and improve developer integration efficiency. Secondly, it supports the scenarios of multi-language interfaces on the mobile terminal. Currently, NetEase Meeting and NetEase Meeting Components (hereinafter referred to as meeting components) support Flutter and Native diversified interfaces. The core control layer logic can support a set of code multiplexing capabilities.

(2) Cross-platform componentization of

The functions of the conference component are very rich and complex. NetEase conference is a specific scenario solution collaboration solution, and the conference component needs to be componentized to support functions, so that the project can develop a pan-application scenario based on the simple combination of functions of the room. The account system, conference reservation system, conference member management and other functions are divided into components, and multi-terminal reuse is achieved in language.

(3) audio and video external rendering

In cross-platform development on the mobile side, Flutter provides the ability to link native controls. For example, PlatformView, which displays custom Views on Android and iOS. You can also use ExternalTexture (external texture), developers can externally render their own graphics data. Because audio and video also need to support beauty, the performance canvas is relatively high, so when designing the video rendering component, we developed an external rendering video plug-in.

3. Ideas and plans

The system design of NetEase Conference utilizes the original PaaS capabilities of NetEase Yunxin. We use IM Server for conference control notification and multi-party chat, and through Media Server to realize media data forwarding. On this basis, we focused on the design of conference-related features, including multi-tenant account system, secure conference membership, conference reservation, conference control, view layout, role control and other functions.

(1) Building a conference system based on NetEase Yunxin PaaS service

(2) Component design and integration

The whole set of conference system client supports Android, iOS, Web, Electron, Windows, MacOS and other platforms. How to ensure fast user access and the consistency of each end, we have carried out a series of split designs for the conference client.

Base: This layer mainly includes the basic SDKs of NetEase Yunxin, and also includes some third-party SDKs.

Room Kit: Room Kit is a room service component without UI. This layer carries the server communication logic and handles some conference control logic. It will also evolve to pan-conference room service in the future. If users don't want to reuse our default UI layout, they can implement their own meeting UI based on Room Kit.

The mobile terminal is implemented based on Flutter. In order to facilitate the access of native projects, we also provide a hybrid development mode to allow access to native languages ​​such as Java and OC.

Meeting SDK: Meeting SDK is a meeting service component with UI. Compared with RoomKit, we provide a complete set of standard meeting UI implementations, and we also provide limited UI customization functions.

The mobile terminal is implemented based on Flutter. In order to facilitate the access of native projects, we also provide a hybrid development mode that allows access through native languages ​​such as Java and OC.

App: NetEase conference application of NetEase Yunxin Standard Edition, which can be directly compiled into an independent application.

(3) Cross-platform componentization

The conference component adopts the design method of high cohesion and low coupling, and each module is divided and conquered. Each module can also be secondary developed through the native interface, and the UI can also be pluggable and integrated, which is convenient for developers to do secondary high-efficiency development.

(3) open source version management

The Android version of the conference component uses Gradle for package management, and the native version management for iOS uses the Pod library. Flutter is published to our official NetEase Yunxin address https://pub.dev/publishers/yunxin.163.com/packages

(4) audio and video beauty external texture

NetEase Conference uses the ExternalTexture solution, which automatically generates the MethodChannel channel through Pigeon, calls the createVideoRenderer of the Native layer, and creates the Render. The Native saves the current TextureId through the FlutterTextureRegistry, and the Flutter layer binds the textureId and Dart layer passed back from the native layer to the Texture Widget. The Native audio and video SDK will call back the data format to Flutter, and obtain image data for rendering according to the shared texture method.

 

4. NetEase Conference Open Source Exchange Group

This article mainly introduces the implementation of NetEase conference components on the mobile terminal of NetEase conference, hoping to help customers and third-party developers to quickly build their own conference APP applications, and also hope that the sharing of this article can inspire other teams' research and development. The following is the open source exchange group of the NetEase conference. For more open source related information, you can follow this official account. We will continue to explore in the next time.

NetEase Conference Open Source Exchange Group

 

NetEase Conference GitHub address

https://github.com/GrowthEase/NetEase_Meeting
 

Author

Zhao Chong, senior mobile terminal development engineer of NetEase Yunxin, is keen on researching cross-platform development technology and engineering efficiency improvement. Currently, he is mainly responsible for the related research and development of video conferencing component SDK.