Huang Ji - How to adapt to OpenHarmony's own audio framework ADM?

Editor's note: During the ecological development of OpenHarmony, a large number of outstanding code contributors have emerged. This topic aims to recognize contributions and share experience. The content in this article is from guest interviews and does not represent the views of the OpenHarmony Working Committee.

OpenAtom OpenHarmony (hereinafter referred to as "OpenHarmony") is booming, but the open source community is still a young and new thing in China. How to participate in the community open source contribution has become a topic of increasing concern for developers. Mr. Huang Ji from the Institute of Software, Chinese Academy of Sciences will explain some experiences and experiences of deeply participating in the OpenHarmony community from the perspective of a developer.

Q1 Please briefly introduce yourself and your development team

Hello everyone, my name is Huang Ji. I am currently working in the Institute of Software, Chinese Academy of Sciences (hereinafter referred to as "Institute of Software, Chinese Academy of Sciences"), where I am responsible for the development of OpenHarmony transplantation. The Chinese Academy of Sciences Institute of Software team is mainly responsible for OpenHarmony's technical research and development, technical support, community support, SIG warehouse CI access control support and maintenance, event marketing support, and other community governance related work in the OpenHarmony project team.

Q2 As a well-known technology expert in the development field, why did you initially choose to join the OpenHarmony ecosystem and participate in open source co-construction? What do you think is the most attractive point of the OpenHarmony project?

I can't talk about Daniel, there is still a lot to learn in my technical ability, professional knowledge and other aspects. I think this choice is a two-way street. On the one hand, I am deeply attracted by the positive open source spirit of the OpenHarmony project. I feel that it is really cool to contribute code to the open source community, and I have the subjective motivation to participate in the OpenHarmony ecosystem. It is the OpenHarmony community that adheres to the principle of openness and inclusiveness, and accepts many ordinary developers, so I have the opportunity to learn about the OpenHarmony project and try to contribute code to it. Open source co-construction is also the most attractive and important feature of the OpenHarmony project. Now the OpenHarmony project has taken a solid step towards the road of open source in China. This road may not go so fast, but it has indeed stepped out bravely, which is enough.

Q3 When did you join the OpenHarmony open source project team? It took a long time to develop the ADM driver of RK3568, and it was integrated into the backbone of thousands of lines of code. Now it is rated as the monthly code contribution star, which is really amazing! Is it convenient for you to tell us about this product, or this experience? You have achieved such good results in such a short period of time. What are your "secrets"?

I joined the OpenHarmony open source project team in February 2021. At that time, OpenHarmony only had a lightweight and small system. Now the capabilities of the standard system have become more complete, and it has its own hap application development tool. It can be seen that OpenHarmony's capabilities are constantly iterating, evolving and improving.

Looking back on the ADM (Audio Device Model, audio device model) development process of the Dayu200 development board, it is full of joy and harvest. ADM is an audio sub-module under HDF (OpenHarmony Driver Foundation). ADM already supports development boards such as Hi3516DV300, and what we have done is the adaptation of Dayu200 development board. Before we did the adaptation, the audio driver completely relied on the TinyALSA library. Now it is fully HDFized, which not only solves the dependency problem, but also has milestone significance and provides a reference for the porting of other third-party development boards. During the adaptation process, we found that it is unrealistic to implement the interface from scratch. Building a wheel not only needs to consider many aspects such as stability, audio codec, format matching, DMA transmission, etc., but also the workload is huge, which is not conducive to the follow-up. maintenance, so we took a different approach. The Linux native function is used for adaptation. The Codec layer obtains the object that operates the I2C bus by registering the RK809 native object, and then passes in the corresponding regmap function to perform register read and write operations; the DAI layer obtains and operates the I2S by registering the RK3568 native object. The object of the bus, and then pass in the corresponding regmap function to read and write the I2S register; and the DMA part uses the native dma_engine related functions of Linux to complete the request DMA channel according to the standard process, configure the DMA channel, and preprocess the DMA channel. , DMA data submission, DMA data processing and a series of operations. Because manufacturers generally maintain the kernel part, and their hardware peripherals are managed by the kernel driver, using the native Linux interface is equivalent to building a bridge to connect the upper-level framework with the kernel driver, which is easier to maintain. At the same time, this adaptation idea and method are original and very meaningful.

If there's any secret, it's unrelenting courage, indomitable perseverance, and a never-ending thirst for knowledge. It is very common to encounter problems. "There will be times when the wind and waves break, and the sails will be directly attached to the sea." Only by facing the difficulties can solve the problems. Sometimes it is very likely that many paths will not work, and sometimes there will be frustration, but perseverance will lead us to go out and slowly find a new direction. Of course, there are still many things that we haven’t been exposed to or don’t know very well, but the thirst for knowledge will push us forward, push us to take the initiative to learn, to understand the unknown, to seek advice from the people around us, and ultimately to help us grow.

You and your team must have paid a lot for Q4 to develop such an excellent product and integrate the core code into the trunk. Can you share with us the entire process of developing this product, including the early, middle, and late stages, what work did you do specifically, and how much manpower and resources did you invest?

The transplantation of Dayu200 development board based on OpenHarmony is led by Runhe Software, and colleagues from many member units of the community are deeply involved in it, including Huawei, Runhe Software, Shenzhen Kaihong, Institute of Software of Chinese Academy of Sciences, etc. The team of the Institute of Software, Chinese Academy of Sciences has undertaken the task of transplanting the ADM module. After taking over the task, we contacted the technical staff of Huawei and Runhe software, obtained the necessary information for research and development such as the chip and schematic diagram of the target development board, and then became familiar with the hardware parameter design, chip register configuration and other information, and gradually built the code. frame. In fact, it takes a lot of time and energy to get familiar with the process. For a completely unclear structure, you need to read the documentation bit by bit. If you encounter unclear details, you must contact the developer to align it bit by bit. This is a test of patience. the process of. The team of the Institute of Software of the Chinese Academy of Sciences has maintained close communication with colleagues from other units throughout the development process, and will organize meetings almost every day to discuss the research and development plan. In response to the problem that the read and write interface using HDI cannot act on the target chip, the Linux native is used. function to carry out the adaptation solution, which effectively improves the progress. After completing the preliminary function and verification of the code, we upload the locally adapted code and submit it to the OpenHarmony code warehouse. During this period, we also go through code compilation, code testing, and code specification review for CI access control, and the problematic parts will be modified. until approved. This process also takes into account the details. It is possible that the code specification review will report hundreds of specification issues or warnings at once, including space specification issues, line feed specification issues, comment specification issues, macro definition specification issues, etc., which need to be carefully checked and modified. . Finally, after continuous revision and verification of the code and sorting the code into a state that complies with community norms, these codes were successfully merged into the trunk. In general, this is the result of the concerted efforts of all parties, completing the closed loop of a series of issues from progress tracking to task assignment to tackling technical problems.

Q5 What technical or other challenges have you and your team encountered throughout the development process? How are these problems solved one by one? In the process of solving these problems, what valuable experiences or lessons have you drawn?

There are the following difficulties in the whole development process. First, the audio chip of the Dayu200 development board is quite special. It contains various functions such as Codec, RTC, and PMIC. It cannot be operated simply by using the ADM interface. In order to avoid affecting the normal operation of other functions, we use the native Linux device driver interface to operate I2C, I2S, DMA communication and data transfer, avoiding the risk of abnormal operation. Second, after playing for a period of time, stop playing, and there will be a sharp and small sound. In response to this problem, we have registered the Trigger function, and according to the received status information, if it is in the stopped state, the codec related devices will be powered off. Third, the volume adjustment function of RK must write the same value to the left and right channel registers to take effect, but the framework did not support assignment to the left and right registers at that time. Later, we negotiated with the framework developers, so that the framework added this function, and finally adapted the volume adjustment ability.

In terms of experience, it is necessary to communicate with different technical developers more often. Sometimes you may fall into a fixed mindset and often fail to find problems in your own code, while others can see the key at a glance and point it out directly, then the problem It was solved.

Q6 What is your biggest surprise since joining the OpenHarmony ecosystem? Or what specific gains?

Since joining the OpenHarmony ecosystem, my biggest surprise is to understand how the open source community works. In the OpenHarmony community, I can learn more from other people's code, and my own code can be seen by more people. Good places are not good. Everywhere, someone will bring it up, and in this kind of rapid feedback, you can better understand your own shortcomings.

I think this period has been full of harvest. First of all, I have met many like-minded partners. Everyone actively participates in open source projects and contributes their own code. The road to open source is full of thorns, and there are so many partners in the open source community to move forward together. , the road has traveled a lot. The second is to gain tempering and growth. In front of excellent people, you can see the gap, so as to learn the advantages of others; in front of excellent code, you can see the gap in the framework concept, so as to learn other people's coding ideas , which is really beneficial.

Q7 What aspects of OpenHarmony are expected to be improved and provide more support in the future?

Objectively speaking, OpenHarmony still has some deficiencies, such as the lack of introductory materials reflected by the community, the lack of clear framework analysis materials, and the untimely response to issue requests. The main reasons are in the following aspects. First, in the early stage of OpenHarmony's development, it is the most cutting-edge large-scale open source system in China. Its development is a process of crossing the river by feeling the stones. The problem is quite normal; second, there are differences in the technical level and technical focus of developers. Some developers may need more detailed entry materials to enter the threshold, and some developers may pay attention to the compilation and generation method of the underlying driver lib library. , some developers may be concerned about how the OpenHarmony SDK generates hap applications, but do not know how to find the development materials they want; third, the management of each code warehouse in OpenHarmony is relatively independent, managed by its own committer, on the one hand, many Developers may not be able to find the correct code repository to file issues. On the other hand, some committers may not reply to issues in a timely manner due to busy transactions. I look forward to the future OpenHarmony can effectively guide developers, provide corresponding development materials and development resources according to their needs, and can further strengthen the relationship with developers, listen more to the voices of developers, and give benign and effective feedback.

Q8 OpenHarmony is still in the development and exploration stage. Many co-construction units and ecological partners still do not know how to play open source projects, or do not know how to start development. Could you please share with us a piece of experience that you think is the most important or worth sharing?

I think OpenHarmony is a very free and open project, and each co-construction unit or ecological partner can choose according to their own needs and ideas. If you want to develop application hap, you can refer to the JS development materials under the docs warehouse of OpenHarmony. If you want to port the development board, you can refer to the chip porting information under the docs warehouse of OpenHarmony. On this basis, if you want to contribute code, you can contact the relevant organizations of OpenHarmony SIG to go through the process of code uploading. As for development problems, it is recommended to ask questions on Zulip in the OpenHarmony community ( https://zulip.openharmony.cn , unregistered users need to register by email), or ask questions in the R&D discussion group, or raise issues in the relevant code warehouses, In short, the channels are very broad.

JS development materials: https://gitee.com/openharmony/docs/tree/master/zh-cn/application-dev/reference/apis

Chip porting information: https://gitee.com/openharmony/docs/tree/master/zh-cn/device-dev/porting

Q9 Open questions, you can speak freely, what else do you want to tell everyone?

The development of OpenHarmony is a long way to go. Now OpenHarmony is like a child who has just learned to walk, but it will take a long time to grow, and its growth needs someone to guide it. The development of the OpenHarmony open source community is inseparable from the support of developers, the support of the media, and the participation of major co-construction units and ecological partners at home and abroad. I very much hope that everyone can pay more attention to OpenHarmony and give more opinions to the community. Your voice is the driving force for the healthy development of OpenHarmony. At the same time, I hope that everyone can submit more code to OpenHarmony, even if they are all built up. It is a great force. I believe that as long as everyone's efforts are combined, OpenHarmony will be able to develop in a positive direction.