Wu Sheng: SkyWalking and the Apache Software Foundation | DEV. Together 2021 China Developer Ecological Summit

content source: On June 5, 2021, the 2021 China Developer Ecological Summit hosted by SegmentFault came to a successful conclusion. At the meeting, Wu Sheng, director of the Apache Software Foundation, Tetrate.io founding engineer, and founder of Apache SkyWalking, delivered a speech on the topic "Those Things about SkyWalking and Apache Software Foundation".

Sharing guests: Wu Sheng, Director of Apache Software Foundation, Founding Engineer of Tetrate.io, Founder of Apache SkyWalking

shorthand compilation and release: SegmentFault Editorial Department

My content will be relatively lighter. Today we will talk about some of the things we have experienced in the entire SkyWalking project from 2015 to the present six or seven years, and the choices we made in the process. I come from Tetrate, a North American company, which is a service mesh company. I am a director of the Apache Foundation and the founder of Apache SkyWalking.

In fact, SkyWalking is now Apache's top-level project, and it should be the world's largest open source APM system. If you are familiar with the field of monitoring and application performance management, since 2000 or 1998, there has never been a software in this field that has been built entirely on the open source ecology or the ecology with the background of open source. All the world-renowned APM software They are all commercialized backgrounds. So SkyWalking was not initially as complicated as it is seen today. It was actually just to solve a very simple problem, and the open source and open source community pushed it to the position it is today. Today's SkyWalking is actually very, very complicated. It covers the logs, indicators, distributed tracking, event detection that everyone monitors and you see, as well as all kinds of common daily operating systems, containers, and some front-end Architecture. Some people are even expanding a series of technology stacks, including mobile phones, WeChat applets, etc., to explore the indicators, performance data, and front-to-back perspective performance of each platform, so it will be a very, very large state.

You will also find that there are a large number of technology stacks, including Prometheus, OpenTelemetry, Zipkin, Filebeat, Fluentd, and many other ecosystems of global manufacturers. We didn't do everything ourselves from the first day. We will cooperate with a lot of global manufacturers and foundation projects. This is why the project can go to this day. We have a lot of friends, friends from other manufacturers around the world, to give us relevant support, to make things more convenient for everyone to integrate, and to use it in line with everyone's habits. This is a picture I recently grabbed from the system. The project was open sourced at the end of 2015. It has a very long stable period. Not many people participated in the project. In 17 and 18 years, all of the project contributors It's very, very little. When we joined the Apache Foundation at the end of 2017, there were only more than 20 contributors to the project plus me. As of today, it is not too complete statistics, there are nearly 500 code contributors, and it does not include documentation or other publicity.

No matter which company SkyWalking is, there has never been a so-called community operation position. We rely on the developers themselves, or on the project, on our understanding of the open source ecology, to cooperate with everyone, and to go global Cooperate with other projects to complete the incubation and expansion of the project. This is the global visits we found on the official website of Apache on SkyWalking.

Not surprisingly, all Apache projects, including the Apache official website, China is definitely the dominant traffic. No matter which foundation you are in, China must be the largest user of open source projects, but at least we are not the largest contributor to these projects. For SkyWalking, for the entire Apache Foundation, for CNCF, Linux foundation, all the global foundations that everyone can think of, we are not the main source of contributors. But everyone will see that the projects incubated in China are those that used to write code comments in Chinese. If you are really willing to do globalization, and you are willing to do it in a reasonable way, then you can also harvest global users, so that you can see that there is almost no IT industry except for certain disconnected regions or Africa. Outside the region, there are very, very large users all over the world. Moreover, SkyWalking has different manufacturers in China, North America, India, East Asia, Eastern Europe and Western Europe. Based on SkyWalking, they are engaged in the construction of their commercial products. For example, there are British manufacturers, Polish manufacturers, and South America. Of developers, they are all doing their own commercialization, including my own company in North America. If you operate open source in a reasonable way, you should do some thinking that suits you. If your project is not that exclusive, you can actually have a very, very large imagination, just like what everyone sees today. We often talk about the Linux operating system. All of its economic benefits are not earned by the people of Linux. What it gets is more industry-level support, more companies, and more different directions, but I get a positive industry cycle from the matter, so I hope to share this with you. Everyone will see that 15,000 sessions visit our system every month. The real traffic is even more difficult to calculate because of all the indicators. They are all recorded based on the session level. Back in the first 15 years, we are actually talking about how open source projects are, which is the starting point for good open source projects. All open source projects, if it is just for open source, or if you do open source ideas, then things are often difficult to succeed. SkyWalking is the same. If it is just to solve the problem, what is its problem?

In the 10 to 15 year period, everyone should start with the earliest Dubbo ecosystem, from the beginning to the springcloud ecosystem, and from the ESB SOA of the year, all applications have changed from single applications to integrated applications. So what kind of problems did we encounter that year? We were still working on a project for China Unicom. We encountered a problem with eight manufacturers working on a project. You can imagine how disastrous this state is. Everyone is accusing other people’s system of instability and causing the entire distribution system to be unstable. It's the same as when everyone in the group accuses the other group of poorly written programs. But when you become eight manufacturers, communication will be even more difficult, because you still have a boss in the company to come up with the decision. Now you don’t even have the opportunity to decide. So when you were doing SkyWalking, the developers accepted it. A training, we let everyone know that when your distribution system has a problem, it may not be a problem with your code. Your upstream, downstream, network, and middle esb components have become any shortcomings.

So SkyWalking was originally just to solve such a simple problem. At that time, there were already two well-known open source projects in the field, Zipkin and Pinpoint, so why did we decide to start from scratch? Zipkin is still there. You can check it out. First of all, its function is very simple, and it is not ready to be a complicated system. Its design is prepared for simple functions, which means that it will really become what we are doing now. It is just a demo, not something that can be used in mass production. Pinpoint is a product made by NAVER, a South Korean search engine company, and it used big data that was very popular at the time. The reason why it is useless is actually very simple. Today, everyone has many reasons to benchmark Pinpoint and SkyWalking, but as a monitoring product, if you use big data, then you may be more difficult to stabilize than the target business system. Because Hbase is a highly distributed system, if you use a distributed system to monitor another distributed system, who crushes first? The monitoring system is the first to crush. We didn’t think about making a world-class apm, we thought that at least we could solve the current problem, so we wrote a very very simple version of SkyWalking, everyone can now go to the earliest 1.0 code on GitHub, very, very Simple, even at that time we tried to use Hbase. Let's see how it is done. How many problems will we have if we use their mode to do it? So, Skywalking has been a testing project for a long time. We are hanging on GitHub, there is no publicity activities, there is no external contact, just put the code on it.

Even at that time, the license was the same, and it should have been GPLv3 for a period of time. In terms of our consciousness, we are not prepared to play it as a community, we think more, anyway, I put a project, and it has a certain protective effect, put GPLv3, everyone found that from the earliest v1 to 3.2.6 It took two and a half years. SkyWalking is a testing project throughout the time. It does not have any kpi, it does not complete any specified functional areas, it is just that people participating in the project contribute code to solve daily problems, write what they can write, and complete what features can be completed, and more It is entertaining work. In other words, it is a test. It is not a test technology. It tests some ideas in everyone's mind to see if they can be implemented. Why 3.2.6 is 3.2.6, because it is the last version of 3.0. And 3.2.6 is the first time I stand on the stage. At the GOPS Global Operation and Maintenance Conference, I stand on the stage for the first time and talk about: With such a project, what kind of functions it has to complete, and what can it help you solve? For the same problem, it is an Apache 2.0 open source project. This is the first time that we regard it as a real open source project and let ourselves be a participant in the real open source project to see the project. After the open source is released, in fact, everyone will find that, for example, the initial applicable users have begun to improve rapidly, but everyone's feedback is more: Oh, you are not complete yet, you can't do this.

Because of the monitoring system, it is very, very difficult to go online. It is difficult for a user to put his own wealth and life on the unstable monitoring system. All his data comes from this monitoring system. Although you do not carry any commercial data, if you hack, it is equivalent to the user’s entire production system. It’s hacked. This is a very dangerous thing. SkyWalking has not been available in production for a very long period of time, including 3.2.6. In fact, it has not been running in a complete production environment. Run in a just-in-time production and test environment, or run in a compressed environment. Use it more as a tool than an apm platform. At the initial stage, we hope to expand the community. We have two choices, Apache or CNCF. I already went to Huawei at that time, so in theory, the CNCF road is open. If you are not familiar with the difference between Foundation, CNCF will be more like a manufacturer-driven, and the atmosphere of business will be stronger. He needs the intervention of the manufacturer, and he needs the manufacturer's investment or manpower to protect it. And projects will have more corporate logos, and SkyWalking was a personal project that year, and it was not born in Huawei, because we did it in our spare time. It did not assign IP to any company from beginning to end, and naturally there was no way to simply transfer it to the so-called Huawei, and let Huawei follow the CNCF path.

If you search for the proposal of a new CNCF project, you will actually find SkyWalking, because we have applied for it before, and CNCF’s coo Kris actually invited us to do presentations, but we didn’t choose to go through a very long process in the end, because it is It's too busy. And for the project, at the time we were more concerned about whether more people could participate in the project. As for whether there are so many people promoting the project, it is not that important to us, because from the beginning to the end, this is not a KPI project, because no one evaluates it. Even if we were at Huawei back then, it was not under an evaluation system and has no direct relationship with the company. It doesn't mean how many contributors you want, how many star companies do you want to evaluate you, and there will be bonuses today. This matter has never had much to do with SkyWalking, and we chose Apache. There is a very convenient reason because Apache needs mentor. As I will talk about later, I happen to have friends around me, one in Australia, one in Huawei, and one in Shanghai. We can quickly find three Apache members to become SkyWalking mentors and enter the Apache incubator. We made adjustments before entering the Apache incubator and changed from GPLv3 to Apache.

There are actually two reasons for the change. This reason is not triggered by Apache. The reason comes from Ali. Ali's team contacted us and said that they wanted to cooperate with us to build some functions.

In the very early days of that time, it was actually a rare thing for developers to make a big thing. But they said that they have concerns that your thing cannot be GPLv3, otherwise I will not be able to report it in the company. In fact, it was a very simple reason at the time. We thought that as long as more people participated, it didn’t matter whether it was GLPv3 or not. Who could use it? Commercialization is done best, who can use it to make money, we are doing this from a fair angle.

We quickly made a resolution in the community to change GPLv3 to Apache v2. Today, if we look at it from a more rigorous level, what v2 actually gives you is a better opportunity for commercial distribution. For example, more products include SkyWalking for secondary public release. You see a lot today. The private cloud platform and many products will natively include all or secondary distribution versions of SkyWalking. This is very common, and you may see part or all of it in a lot of open source projects. This also provides a good foundation for us to do cross-organizational collaboration in the future.

But when we really step into Foundation, we will find that there is a very large gap in the middle. For a long time, open source was entertainment for me. But when I brought the project into Apache, I became more like a professional open source developer. At the same time, you will find that you need to be in different roles. When I speak on behalf of Apache, when I say anything in Apache, it cannot be commercial. It is to pursue the quality and safety of the software project itself. And iterative future development, maybe there is a way to make money in my mind, but that cannot be placed on the project level. So you will later find that you will be subject to various constraints. This is also what Foundation brings to everyone. It gives you a very good name. You think you are very famous. You are a project of the Apache Foundation, but at the same time it will give you You are more constrained. You can't use the branding in the Apache Foundation without restrictions, and you can't use this well-known trademark right without restrictions and without editing, because the Apache Foundation doesn't want to deal with it.

The Apache Foundation has more than 350 click projects. In the world, basically all the front-end and back-end components of the software that everyone uses must come from the Apache Software Foundation. But at the same time, the annual operating cost of the Apache Software Foundation is only less than 1.6 million US dollars, and almost none of the money is sent to the members of the Apache Software Foundation in the name of salary. Everyone is using some logic. For work, you have done a certain project, perhaps because of hobbies, or writing books or doing training, you have your own logic, which makes your activities in the foundation beneficial.

Of course do you have hobbies? meeting.

However, the hobby participant must be relatively low-intensity. For example, it is difficult for him to spend more than five hours a week, because more than five hours is equivalent to wasting a day, and this is for most people. It's impossible, especially for many people who would have to work for six days and come back for another five hours. This is unlikely. And what do the vast majority of volunteer contributors in China look like? Although he gets off work late, he is interested in this matter, and he thinks he needs to contribute something, so in the emails submitted by SkyWalking, he often finds that someone mentions a pull request at two o'clock in the morning and three o'clock in the morning. This is called a hobby. If you are faced with a professional developer, he will definitely not be like this. SkyWalking has many professional developers. The professional developers we see at night must be from the United States, South America, Argentina, and Brazil, because their evenings are normal working hours. This is a professional developer. Faced with different people, know that their preferences are different. Everyone knows that the foundation has a history of 22 years. The earliest is 98 years and 99 years. The foundation is a relatively old-fashioned foundation. Discussions will be slow because it is all asynchronous. It takes 72 hours to wait, because it takes three days to make it visible to all people in the world and to bypass the weekend. Any resolution, any small matter, as long as it rises to the resolution level, is a long wait.

There are also many things in the foundation that people feel unaccustomed to. We have been talking about the Apache Foundation, so what is the Apache Foundation? This is actually Apache's official explanation. Everyone has their own explanation for these three sentences.

Some people explain business logic, some people explain it from the perspective of popularity and interest chasing, some are software development, and the love of coding, some people have the idea of helping others, and some people want to reduce others. The burdens are many, many, and everyone's ideas are different.

The foundation is diversified, and the starting point is different. Don't use your own starting point to randomly speculate why the other party wants to do this. His explanation is likely to be completely beyond your expectation. Because some people are really working for hobbies. For example, we have seen very young engineers in some parts of Eastern Europe, 16-year-old and 17-year-old engineers. He will participate in projects for his dream of starting a business after 20 years old. Years of development of a function that may not be used by two people for a very long time. There is a sub-project of SkyWalking developed by Eastern Europeans, which leads all your monitoring data back into the IDE. When you see the code, it can tell you what the system indicators of this line of code are in the production environment. Looks like, and this thing does not currently have any commercial scenarios, but he has been working on things for nearly two years, he is very young, he may be in the long-term, he thinks that this is good for him in the future whether it is to start a business or Using this energy to prepare yourself for a business is helpful. For a professional developer like me, I think it is very uneconomical. My time should not be used for long-term, high-complexity things that have no commercial benefits. The amount of code is very large.

This is a simple example. I want to tell everyone that when you participate, you have to maintain a relatively humble attitude and ask more questions. At the same time, everyone should know that when someone asks you, he is not necessarily opposed to you. He wants you to describe the situation more truthfully. We also saw several local projects donated to the Apache incubator this year, and we found many problems in the process. questions are often understood by the Chinese environment as questioning , because China’s habit is that all of our things are discussed first. When I post something public, it means that the resolution has been formed and no one will make it public. A word in English on occasion is argue, and the translation of argue into Chinese is basically the concept of quarrel. In the entire cultural environment, you need to be able to tolerate a certain degree of tolerance. How do you persevere when the person you are cooperating with is not in the same culture as you. So the diversified things just mentioned are also diversified. It must ensure that all manufacturers contribute. For example, your project cannot be controlled by a single manufacturer, especially the top-level projects. It is diverse, no matter from gender, region, skin color, age, all aspects must be diverse, its code must be trustworthy, and malicious programs must not be deliberately buried in it. For example, we can see in China that some projects are actually collecting user deployment information when users use open source versions. He can actually know how many points the project has deployed on the customer’s online open source users. If this is a by-product of a commercial contract, I don’t think there is a problem, but if you are also collecting information from open source users, it is actually a very dangerous move. This situation is even suspected of being illegal in some countries and regions, because you are stealing users' private data. security is more that the existing code itself can obtain the security , that is to say, once the code is delivered to you, I cannot withdraw it. In addition, there is also the level of security. Because it has opened up the source code and opened all the copyrights to you, you can fix security vulnerabilities faster, and even patch yourself. This is also a very common thing.

In fact, Junping also said that community over code in the morning. Everyone talks about it when they talk about Apache culture. In fact, everyone has a different understanding of this sentence, but in the end, everyone must agree: people are the most important.

So how do you make the project truly transparent, so that everyone is willing to take the time to participate in the project, and everyone believes in you or the project is good. A simple conclusion: between ASF and CNCF, if you want to be the main manufacturer, you need to quickly carry out commercial marketing, there is no emotional color here (because our company also does it in CNCF, for example, many people in our company are doing Envoy, Because doing Service Mesh, using Envoy, istio, or even an open source project that is not enough by a single vendor), it’s not that ASF is better than CNCF. At least from my personal point of view, I don’t think the two are comparable. This is a completely different operating model. . You need to make a normal choice, but you have to know the difference. ASF must not be profit first, it must be the developer first, and the developer first. How to unite is the most important thing, and CNCF is that the manufacturer’s interests are higher than the interests of the manufacturers. all.

I want to take the vendor as the background. The CNCF project requires at least three manufacturers to participate. It does not require you to be a developer from many places. It cares a lot. Which company are you, so when signing a CLA with CNCF, you have to say which company you are from, and the mailbox you signed is basically the company's mailbox, and the company must sign the CCLA together with the affiliated company.

After choosing the foundation, in fact, we will have a long-term process of operating the project. Of course, there are many, very many stories, very, very many pains or successful experiences, but we have compared a lot. Because I myself have participated in many open source projects in China, from Apache projects such as Echarts, ShardingSphere, APISIX, DolphinScheduler, or CNCF including OpenTracing and zipkin, I actually have them in these projects. What we can see, whether in China or abroad, as the leader of the project or a member of the core team of the project, you have to ensure that you can do these things. Although you may not be successful, you have not done it. It will do great harm to your project and your community. Openness is a simple word that can be said, but when your other core personnel are not from your company and do not share business interests with you, how can you trust their decisions? And how to choose those you can trust?

I believe it is two-way. The more you believe in him, and the iteration speed he brings to you, the stronger the characteristic abilities he brings to you, and he will gradually deepen it. And if you have everything in your hands, then I am afraid that your time investment is the limit of the project. Over 80% of SkyWalking's code shouldn't be written by me. I don't know the details. More often, I follow the documents when asked by others, because I don't know what they changed into the project. I know what it looks like in general, I know what data flow is like, but this is the limit, and this is in the case of me as a professional developer. If you are not 100% full-time, you probably understand even less.

The remote mode is actually the asynchronous mode in disguise. It is also an eternal pain in the hearts of many engineers in China. You ask a question and find that no one takes you. When others answer you, it has been several hours. Chinese engineers like it very much. When asking a sentence, ask half a sentence, is this okay, what is it, what is it, and what is it? Someone will reply, okay, that day is over. Isn't this nonsense, you have to master some of the most basic remote working skills. And when everyone works like me, because my own company is in the United States, our engineers range from the easternmost New Zealand city in Japan to the westernmost city in Messi, and there are more than five engineers in almost every city. All of our company's products are completed asynchronously, and we have almost no company-level synchronous meetings.

This is the situation we usually face. You would imagine going to Run Open Source Community in a company. You must know these things, otherwise the company can't do its work. Let’s do a Google docs review. Then someone left a Google docs for you. You agree, disagree, or want to improve, or you think the other party made a mistake from the beginning. What about you? Express your wishes as clearly as possible once in words, this is actually the meaning of the remote mode.

During the epidemic last year, we saw a lot of very funny things in China. Everyone is forced to work at home, constantly checking the progress, and constantly synchronizing the progress, the leader wants to know if you have gone out for a desertion. In a company like ours, our leader will send you an email request at ten o’clock in the morning and say, if I can hold a meeting for you in an hour, I can answer and say no, I’ll go and take the kids to play for a while. Either we will meet again in the evening, or you can send me an email and tell me in words, are you in a hurry? If you are in a hurry, wait for me for two hours, and we will talk again at one o'clock in the afternoon. The whole culture is from top to bottom. The engineers below me can also talk to me like this. He said that I am not free now, taking children or picking up my daughter-in-law. I often send WeChat to our Heilongjiang colleagues at 4 o’clock in the afternoon saying you are ? He replied that I was picking up my daughter-in-law, and I said I would talk about it then, and then I went to fill in an email to him.

If you want to make the open source in that state, you must be able to accept this working mode, and you must learn to let everyone run at a high speed. Because the biggest problem of China's WeChat culture, QQ culture, and real-time communication culture is that people who are contacted are frequently interrupted. Everyone feels that they are running in the most efficient state, but the conclusion is not efficient, because people are constantly being interrupted. You have to understand this model, you have to really accept it, and you have to be able to live in it, so that you can integrate with the global community. SkyWalking has a Node.js and Python probe. Both probes come from an engineer from an Eastern European company in South America. The time we can communicate with them is only one hour, from 9 pm to 10 pm, if This matter needs to be communicated in real time and must be completed within this hour, but it is very likely that he is not online, or I am not online, and everything is done online. I still don't know what that engineer looks like. I haven't done a video conference. Everything is text. When you need to write with others, you need to know what other people expect of you, because they don’t want to be dragged by you, because you are not led by them, and you don’t pay them. Why do you drag them every day? Contribute a line of code? Is it okay? This is the actual problem you want to solve when you are doing remote.

And what is more called below? we often call the bona fide dictatorship in the community, you try to understand everything, all your code must conform to your habits, you have to make your own decisions, or there is a small group in it to make decisions, or Any other similar things are very hurtful. After everyone has experienced it once, they will ignore you in future decisions, because you don’t take others with you when making decisions anyway. There is a difference in this matter. I held a public meeting, I held a public mailing list, but you have no feedback, so I will do this, and I will notify you after I made a resolution offline and give others the feelings. Is completely different. You can not participate, you can pass it by default, you can ignore it, you can challenge it later, you at least give people a chance to go back, and they know why you made this decision back then. Often what people oppose is not your decision itself, but opposes your decision list. Your business has not been opened up, and it did not say that I am going to do this today because of abcd and other reasons. This is what everyone should pay attention to.

The last one, the most difficult be remote for Chinese people is that others do not agree with you. This is the most difficult problem for Chinese engineers to solve. There is a saying in English culture called agree to disagree. You have to agree to some different opinions. Everyone can work according to his demands. If there are some directions, he can find the way first. It is not necessarily when making a decision. Everyone must pass a vote by show of hands. Absent voting is very common.

When doing this, it is very important whether you can do this, and whether you can tolerate it as a leader of the community. And whether your software technology can survive in this environment, for example, if a wave of groups has passed a relatively unsatisfactory plan, will it cause a devastating blow to your entire ecosystem? Maybe you will be the next one The version function is completely abolished. Then we can guarantee the quality of your software. Whether it is your automated testing, chaotic testing, stress testing, or fully automatic testing, can you guarantee that something will not happen? Can you pass theoretical proofs in the early stage? Data proofs and experimental tests can lead to a conclusion, saying that you are not good, and I object to you. This matter is based on data. Rather than saying that my experience is not good, your experience is not good, what Chinese people like to say is: Our company has such a large amount of traffic, you do not, so this thing is not good. In this way, when you are in open source, no one will talk to you about technical solutions, because you don't take other people's words seriously. Maybe it is true that the traffic is not good under the situation of 10 million, but it works well under the situation of 100,000. Many systems are like this. You have to understand how many people are facing 10 million traffic. It’s like we were talking about a lot of Double Eleven plans before, but how many companies need Double Eleven? There are very few companies like Ali, and your plan is very complicated, but is it useful? This is a very serious problem.

This is the incubation process. I will not go into details. If you are interested in Apache incubation, the core point is that you have to find three mentors. To put it bluntly, incubators and foundations are all social games. You can find people. , You can persuade them, use money to persuade or use favors to persuade, anyway you find a way, no matter what is social skills. If you can handle it, you can do it. Then in the Apache Foundation, you will have a lot of levels, but this is just the so-called level. Anyone at any level does not mean that you have more rights than others, but often more obligations. Go up, the more work you have to do. In fact, there is no power, the only difference is from contributor to communier, your change is that you can submit code directly to the library. However, many mainstream open source software, such as myself, cannot submit code directly to SkyWalking. If we didn't go through other pmc or review, my code won't get in either.

The following content is what we hope to let you know, in the environment of China's Open Source, there are several common problems that you have. This is the biggest pain I think after I have felt the open source community on both sides for so many years. Chinese developers have infinitely high requirements for the maintainer of open source projects, which are so high that they are not as high as the requirements raised by customers.

For example, Chinese developers say that the documentation is not well written, and the source code is given to you. Do you care if I write it? If you want to use it, you can use it, if you don't want to use it, then you can use it. If the project is very early and you want to find some seed developers, this is a bet between yourself and the other party, you are betting that the other party's project is easy to use, and the other party is betting that you will recognize him this project. But as the project matures, there will be no such process. If you look through all the top-level projects of Apache native to China, I can say responsibly that SkyWalking must be the community environment and the attitude of answering questions is the worst. Because the most common sentence is to go back to read the document, go back to read the code, this is the answer to most of the questions you ask. Why does this happen? Because this is not reciprocal, I have not provided you with commercial services, why should I provide reciprocal capabilities? In other words, SkyWalking focuses on developers who can do it by themselves. They are the value of the community. If you can contribute a line of code to the community, it is worthwhile for me to spend ten minutes talking with you. If you have a problem when deploying on site, then I don't want to care at all, because there is no problem when I deploy. Many people in the community said that this matter is very anxious, there is a problem in production, and what does it have to do with me? Why should I spend so much energy helping you to see the stack at nine o'clock in the evening, am I crazy? Is it good for me to watch TV?

So these are some unrealistic hopes that people often put forward to Chinese projects in logic. Chinese open source software authors complain every day. I am so tired because you are used to him. Basically, you want to be used to it. , A willingness to get used to.

Second point, time. so-called I don't have time, I was in 996, I was in 997 or more. But if you calm down and think about two things: the first thing, how many workers can really work in 996 and 997? Actually not. Most of the night everyone was eating, playing mobile games, and going to the gym downstairs. When the time came, they clocked out, or they were working at night and came back at 12 in the morning. In fact, they only worked eight hours a day. Because normally, why in western IT companies, we would say that the contract I signed with you is eight hours, and I don’t want to work today. Why? It engineer writes programs, he can't write it if you ask for overtime. He had a bad mind to write programs, and there were three programmers behind to help him fill in the blanks, all of which were bugs, so it's better not to write. This is especially true for infrastructure software. Your hidden bugs are devastating. After you write a three-minute program, the other party may not be able to call out the bug for three months. If he is such a person, you can think that he is not a practitioner of open source software. I am not saying that such a person does not exist. Open source culture is not about persuading everyone, not telling everyone that open source is hot now, so you have to go open source. Not everyone will pay for it.

Just like what Apache often says, Community Over Code is not suitable for all open source projects, and it’s not wrong to not adapt to it. Many things that don’t comply with the Apache Way have succeeded, and the Linux operating system is typical. Typical, so don’t treat it as an absolute standard.

Finally, when you are working on a project, you must also connect to all the resources you can use. engineers, ecosystems, players of other projects, and various Global vendors. You need from personal relationships, social skills to your cooperation projects, the same foundation, cross-foundation, other companies, or professional players, these professional players will be in your promotion, future promotion It plays a pivotal role, as well as the professional developers who are backed by you to do commercialization. Which one will be needed at different stages. They are not good or bad. Money is a good thing or not a good thing. It depends on how you use it and how you distribute the structure of the community. Every community and every routine is different. Has its own subtraction.

In fact, if we look back at the six years, we will say: patience. Many times you see SkyWalking today, I can only put the first three pages, I can say how powerful SkyWalking is. I can also do a 30-page ppt to tell how powerful the project is, how large the deployment, and how large the environment is. I can talk about it all afternoon, but what you need to know is that it actually didn’t have anything on the first day, and you’re backed by it. Is the largest developer in the world. More than 90% of SkyWalking's functions involve Chinese engineers, and his users are all over the world, and the codes written by a large number of Chinese engineers in open source are considered by Western engineers to be too complicated functions. This is what they think is asynchronous. Impossible under the environment. Many Chinese engineers rely on their own ability to complete their work, so I don't think that Chinese engineers lack the ability to open source, but what they lack is the correct method of collaboration. We are very anxious about a lot of things. For example, SkyWalking has a lot of functions, which were blown out two or three years ago. We can only get the result today. I don’t know if you have noticed it. I wrote it on my earliest ppt. In a word, SkyWalking's version 8.5 released today and last month has just completed what we boasted five years ago, and only then has those functions truly turned into a state where the code can be used, and this process took six In one year, with 500 engineers and countless tests, we don't know how many people around the world have tested this program. So be patient. Similarly, after completing what we did six years ago, I will find that what we thought about six years ago was the entire landscape. Among the landscapes we can see today, we only completed 50%. We may even need six years to finish what we think of today. This is open source. Our Landscape will get bigger and bigger, and things will get deeper and deeper. From a java probe, SkyWalking has achieved full language exploration. Needle, we have achieved the service mesh. Now we have to connect to the operating system, we have to build our own database, we have to connect to other collection software, we have to connect to other protocols, and we will find that things will increase.

Finally, we will return to ApacheCon next month and next month. Of course, because of the epidemic this year, we will still do it online. The last ApacheCon offline is in Las Vegas, the 21st one. At that time, the epidemic began. We brought ApacheCon Asia to China for the first time during the 22nd session. After four years of hard work from 17 to 21 years, China finally has enough Apache members, which can enable the Apache Foundation I believe we can do an ApacheCon that belongs to China and Asia. We also hope that next year, when the epidemic situation is better, we can let Apache pass the resolution of bringing ApacheCon Asia to a certain city in China for the first time to hold some offline events. Of course, this year is actually very good, because 70%-80% of our ApacheCon Asia sections are available in Chinese. This is a benefit that ApacheCon Asia brings to us, because China’s Apache member almost covers all track chairs in the entire conference. They have more say, and we can better pass the entire screening of ApacheCon. This is what we can bring to everyone. In the future, we hope that more engineers can go from ApacheCon Asia to the global ApacheCon.

This year’s ApacheCon’s global time zone is also in North America. Like monitoring topics, basically both Chinese and foreign engineers will use SkyWalking related topics. We also hope that more Chinese projects can go out and share more in English. Many topics. Because I have been on the Internet for a long time. This is my Twitter. I will spend more time on it than on the domestic platform. I hope you can talk about open source, SkyWalking, or some of the foundation’s culture, everyone. You can find me on Twitter. thank you all.