Zhao Haipeng: How to design and develop OpenHarmony audio feature architecture

Editor's note: During the ecological development of OpenHarmony, a large number of outstanding code contributors have emerged. This topic aims to recognize contributions and share experience. The content in this article is from guest interviews and does not represent the views of the OpenHarmony Working Committee.

In this issue of OpenAtom OpenHarmony (hereinafter referred to as "OpenHarmony") developer story, we specially interviewed the best contributor to the code in February, a developer who has been in contact with OpenHarmony for about a year and completed a difficult development project in early 2022 - Runhe Software senior software development engineer Zhao Haipeng.
Zhao Haipeng is the head of the media field of Runhe OpenHarmony's southbound business, mainly responsible for Audio development. In the process of adapting the Audio Driver Model of the RK3568 platform, in the case of the sudden epidemic in Xi'an, both hardware and communication problems were faced with huge challenges. Facing the urgent project needs, Zhao Haipeng and his partners rose to the challenge and passed the Coordinate equipment through various channels, send the finished firmware, coordinate software partners to do remote testing, including welding, etc., work and communicate online for more than 12 hours almost every day, and finally overcome difficulties and successfully complete the task.
We chatted with Zhao Haipeng about his original intention of joining the OpenHarmony ecosystem, his understanding of the adaptation of the OpenHarmony architecture, the difficulties encountered in his work and the process of overcoming them, as well as his experience and lessons in the open source process. The content of the interview is now organized as follows, hoping to inspire you.

Q1 Please briefly introduce yourself and your development team

Hello everyone, I am Zhao Haipeng, a senior software development engineer at Runhe Software. I have been in contact with the OpenHarmony open source project since October 2020 and began to understand the framework and structure. Currently in Runhe Software, he is mainly responsible for the media field of OpenHarmony's southbound business.

Q2 As a well-known technology expert in the development field, why did you initially choose to join the OpenHarmony ecosystem and participate in open source co-construction? What do you think is the most attractive point of the OpenHarmony project?

The first level, from the big environment, OpenHarmony is an innovative operating system, which is the primary factor that attracts me.

The second level, in terms of personal growth, I hope to join OpenHarmony in the early stage of development, which will make me more clear about the evolution of the entire system framework, and there are relatively more opportunities for personal growth.

Q3 Is it convenient for you to tell us about this product, or this experience? You have achieved such good results in such a short period of time. What are your "secrets"?

There is no "secret", mainly in the process of study and work, ask yourself more questions, and study and research with problems; at the same time, continuously summarize and accumulate problems encountered in the process to form a knowledge base.

I'll go on to talk about the features of the main contribution. Our goal is to adapt the Audio driver of the non-HiSilicon third-party platform RK3568 in the community. Because the Openharmony Audio driver framework is ADM and the native driver is ALSA, the difference is relatively large. In order to speed up the progress, a partner of the Coordination Software Institute and I jointly developed, just in time for the epidemic in Xi'an, I have been focusing on research and development at home, and communicated online if needed. There will be many difficulties in the process. Debugging the Audio driver requires the support of some hardware equipment (oscilloscope, logic analyzer, etc.). In the epidemic environment, some equipment is lacking, and express delivery from Xi'an is also difficult to come in. We Coordinate the equipment through various channels, and then send out the finished firmware, so that the partners of the Institute of Software of the Chinese Academy of Sciences can do remote testing, including welding and so on.

In addition, the time node of our task is relatively tight, only less than a month or so, and there are still 30,000 lines of Audio driver code after cutting, that is, we have to read the 30,000 code and adapt it to OpenHarmony. The work also increased the difficulty, but we all overcame one by one, persevered, and finally completed the task.

You and your team must have paid a lot for Q4 to develop such an excellent product and integrate the core code into the trunk. Could you please share with us the entire process of developing this product, including the early, middle, and late stages, what work have you done, and how much manpower and resources have you invested?

In the early stage, there are 10w+ codes related to Audio in the kernel code, which need to be cut into the smallest set. In addition, the code framework of ADM on the main line needs to be sorted out, refer to: https://gitee.com/openharmony/docs/blob/master/ en-us/device-dev/driver/driver-peripherals-audio-des.md .

In the middle stage, when entering the real development process, I will do a good job of the framework first, and then cooperate with the development according to the division of labor. At that time, because it was an online office, the working hours were more than 12 hours a day. The two parties communicated through online meetings, and problems were communicated and resolved in a timely manner.

The later stage is mainly the debugging stage. At that time, there were some problems with the signal. The hardware engineer of the Institute of Software of the Chinese Academy of Sciences helped us to weld, then sampled and sent the signal image back to us for analysis, and then adjusted the next plan. Some difficult problems will also turn to the person in charge of the ADM framework. In order to ensure higher work efficiency, these are all communicated in online meetings.

In addition, during the debugging process, it was found that there are some unfriendly and imperfect parts of the framework. During the adaptation process, it was continuously improved, forming a relatively simple adaptation scheme for Linux and forming a document, which was released on the community. The problem with this solution is that it is not compatible with LiteOS and does not fully realize the optimization capability of ADM.

Q5 During the entire development process, what technical or other difficulties did you and your team encounter? How are these problems solved one by one? In the process of solving these problems, what valuable experiences or lessons have you drawn?

Technical problem: RK809 is used in the codec component of the RK3568 platform. This chip is not a single codec function but also includes a power management module. It uses the same I2C control channel, which is difficult to split. It may also be necessary to design a power management module.

Solution: With the help of Linux native driver, ADM's driver interface initialization node calls the corresponding probe function. According to this idea, the other modules also follow the same operation, reducing the dependence of driver code development on registers and improving development efficiency. The specific solution is described in the RK3568 driver adaptation document, please pay attention.

Q6 What is your biggest surprise since joining the OpenHarmony ecosystem? Or what specific gains?

The first level of gain is that my previous work experience was relatively single module or single feature, and now I have the opportunity to face the whole system. At the same time, OpenHarmony is going through the process of going from 0 to 1. In the process of our work, we can deeply understand the whole system, obtain a more comprehensive understanding, and have a relatively large room for improvement of capabilities.

The second level is for the design of the system. In the past, I only needed to consider the internal implementation logic, process, and interface of the requirements. When designing requirements now, first consider external dependencies, define interfaces, and then design the framework of specific requirements, software layering, and so on.

Q7 OpenHarmony is still in the development and exploration stage. Many co-construction units and ecological partners still do not know how to play open source projects, or do not know how to start development. Could you please share with us a piece of experience that you think is the most important or worth sharing?

I think the most important thing is to combine your past work background or environment. If you don't have much experience, you can start with the mini system. If you have some Android or Linux experience, you can start with the standard system. In short, you must start with the modules you are familiar with, so that you can learn by analogy. By learning and dismantling, the familiarity will become higher and higher.

After getting started, you need to focus on a single point for in-depth research. After you have a deep understanding of one point, you will learn other points faster. At the same time, we must also look at the overall architecture. If we do not understand the architecture, it is not enough to support subsequent development and project work. At least a conceptual understanding is required.

Q8 Open questions, you can speak freely. Do you have anything else to tell everyone?

In terms of driver system, the current OpenHarmony driver is developed based on HDF, which can run on Linux or LiteOS, which is convenient for porting. However, the current maturity is not enough, and the adaptation difficulty is relatively high. It is not very friendly to developers. I hope that all co-construction units and open source developers can improve it together to make platform-driven adaptation easier.