The strategic cooperation announced at the Open Source Gravity Summit of the 2021 Cloud Home Conference, what exactly is the Grafana service?

Introduction to Those who follow the Yunqi Conference these days will surely find that Alibaba partner and Alibaba Cloud senior researcher Jiang Jiangwei (Xiao Xie) specifically mentioned a service called Grafana in his speech at the Open Source Gravity Summit of the Yunqi Conference Products, and deliberately spent a page of PPT to introduce this cooperation. What kind of product is worth a grand introduction?

Author | Young Qi

Those who follow the Yunqi Conference these days will definitely find that Alibaba partner and Alibaba Cloud senior researcher Jiang Jiangwei (Xiao Xie) specifically mentioned a product called Grafana service in the speech at the Yunqi Conference Open Source Gravity Summit, and specially It took a page of PPT to introduce this cooperation. What kind of product is worth a grand introduction?

So, let's talk about the Grafana service together~ Before the official start, everyone can answer the following questions:

• Have you been immersed in the sea of technology of buying machines, configuring networks, building environments, installing and deploying, preparing domain names and IP addresses?
• Have you ever encountered a service that is unable to start and cannot be accessed? SLA is not guaranteed? Or after a period of time, you forgot the service IP address and password and can't find it?
• Do you want to integrate the data of various cloud products such as ARMS, SLS, cloud monitoring, RDS, Trace, etc. into a unified market, and you don't know where to start?
• Do you want Grafana to connect to the private network VPC to build their own data source business data, and provide email, SMS alerts and regular reports?
• Have you installed various plug-ins and adjusted configuration parameters on the command line?

What is Grafana?

In order to solve the above problems, Grafana came into being. The official of Grafana introduces itself like this: Grafana is an open source visualization and analysis software. It allows querying, visualization, reminding and exploring indicators, no matter where they are stored. Simply put, it provides tools for converting time series database (TSDB) data into beautiful graphs and visualizations. You still don't know what Grafana is? Then you must have seen the following similar visual boards, which are configured in Grafana.

Why do Grafana managed services?

As a unified display solution for cloud native observability, Grafana can cover various data sources and monitoring systems downwards. There are thousands of users on Alibaba Cloud who build Grafana by themselves to complete the visual display of data. When using Grafana, they also have to endure various aspects such as operation and maintenance deployment, personalized settings, account management, Alibaba Cloud data source docking, and alarms. inconvenient.

In order to solve the various pain points of users in using Grafana, our cloud native team and the Grafana official (Grafana Labs) conducted a year-long communication and negotiation and reached a cooperation. Currently, the Grafana hosting service is in public beta (see the application address at the end of the article) . This cooperation is also the first in-depth cooperation between Grafana and cloud vendors in the Asia-Pacific region. Both parties will provide the first Grafana managed service in China.

For details, please refer to: Grafana Labs, together with Alibaba Cloud, will provide the first domestic Grafana hosting service

What are we doing in this process?

Before determining the system architecture, we clarified that as a Grafana managed service product, there are many demands from users, including:

• The convenience of migrating self-built Grafana to hosted Grafana (migration tool/configuration import);
• Fully managed automatic flexibility (no server operation and maintenance, high availability);
• Support multi-party data sources (ARMS, Prometheus, SLS, user-built data sources), alarm function integration;
• Alibaba Cloud's official data source pre-built data market;
• Data access security (cloud account management/sub-account authorization/user group/user authorization);
• Provide a variety of version selection, plug-in version selection, enterprise-level plug-in support;
• Provide independent access to domain names/links.

Therefore, we must comprehensively consider the above factors when designing and selecting the system architecture to provide users with stable, easy-to-use, and powerful Grafana hosting services. like:

• For better operation and maintenance management and cost considerations, the managed service cluster adopts the ACK cluster;
• In order to ensure the security of user data isolation and the convenience of database upgrade, migration and maintenance, we adopt an independent database solution (currently NAS sqllite);
• In order to be able to connect the data in the customer's VPC and comprehensively consider the performance, cost and the scope of support for the VPC, we use dual network cards to open up + FC network open up;
• Others such as service state machine design, cloud service access, user authentication, authorization, independent domain name, etc. are all considered and weighed;

As shown in the figure above, the middle layer hosts the customer Grafana instance through the ACK cluster, and connects to the cloud service data source, the data source in the VPC (FC channel), and other cloud service data sources under it. The upper layer is divided into customers to provide Grafana services and consoles by connecting to various external services:

• Client use (service itself), Client -> Proxy -> Grafana
• Monitoring management (console), Client -> Console -> Grafana

The following figure shows the entire process of users opening and using Grafana:
From the perspective of the control end, the administrator user or the main account can open and purchase the Grafana workspace through the console, and complete the authorization of other users or sub-accounts. At this time, we will create an independent Grafana Pod instance for the user in the ACK cluster. A series of supporting resources (such as: NAS storage, configuration, self-monitoring, alarm integration, default plug-ins, cloud service integration, etc.); from the perspective of Grafana usage and management, the actual user or sub-account of Grafana can log in to the Grafana workspace and it will be normal Complete the configuration of the data source, large panel, etc.; if the administrator has configured the VPC connection capability, then the data source in the customer’s VPC can also be accessed through our data proxy.

What features does the Grafana service provide?

1. Function demonstration

After applying for the public beta, we will review and approve it regularly every week, and then you will be able to experience the Grafana hosting service.

A brief description of each module is as follows:

• Homepage: First, entering the system homepage will show Grafana's core advantages and functions, as well as access steps;
• • Currently under the ARMS domain name, a separate domain name and entry will be provided in the future;
• Create Workspace: After entering the welcome page, you can click the "Create Workspace" button to create a Grafana hosting service
• • There is no charge for creating a workspace during the public beta, you just need to simply fill out the form.
  • Version type: Only the standard version will be provided during the public beta, and the enterprise version will be provided for you to choose from in the future.
  • Version number: Only 7.5.x version will be provided during the public beta, and 8.1.x and other versions will be provided for you to choose from.
  • Password: Refers to the login password of the admin account. You can also log in directly with the Alibaba Cloud account without using this password.
• Workspace list: After creation, you can manage your Grafana workspace
• Work area details:
• • On the workspace details page, you can view various information in the workspace, and click the "Login" button to directly use your Grafana.
  • On the login page, you can choose to log in through the "Alibaba Cloud" login method, and you can start using the Grafana service.
• Performance monitoring:
• • On the performance monitoring page, you can view various data of your workspace (users, organization, market, resource consumption, request frequency and delay, etc.)
• parameter settings:
• • Through parameter settings, you can manage various personalized settings of Grafana in a white screen
  • Such as: Oauth, service port, domain authentication-free sharing market panel, etc.
• Account management:
• • On the account management page, you can reset the password of the administrator Admin, or authorize other accounts to access the Grafana instance.
  • The function of inviting users to join the Grafana workspace via SMTP will be added in the future;
• Cloud service management:
• • The cloud service management function currently supports the synchronization of ARMS Prometheus data sources and large disks to the Grafana workspace.
  • In addition, it also supports the data source configuration of SLS CMS (see the page help document for the configuration method);
  • In addition to supporting the above cloud service data sources, ARMS front-end monitoring and ARMS link monitoring data access are also planned;
• VPC data source channel management:
• • You can install the VPC data source channel by specifying the VPC, switch, and security group, so that you can access the data source in the VPC on Grafana.
  • Access the data source in the VPC, just like the current Grafana is deployed in the VPC (for details, please refer to the help document on the page)
• Alarm management:
• • The alarm management integrates the ARMS alarm channel. Users only need to simply configure the contact person to configure the Alerting rules in Grafana;
• Plug-in management:
• • Plug-in management function, providing convenient installation and uninstallation of plug-in functions, users do not need to care about tedious technical details such as low-level commands and restart configuration adjustments;

2. Core advantages

• Flexible and free of operation and maintenance: With Grafana managed services, you can use highly available services without managing the operation and maintenance server.
• Unified visualization: support Alibaba Cloud data sources (such as ARMS, Alibaba Cloud Prometheus monitoring, SLS, etc.), self-built data sources and third-party cloud vendor data sources (cloud monitoring, link monitoring, and front-end monitoring data will be supported in the future).
• Data security and authorization: Support Alibaba Cloud account SSO and self-built account system to realize the refined management of data sources and large disks, as well as data backup and recovery (under development).
• Data access across multiple VPCs: Supports the same Grafana workspace to connect to multiple private network data sources in VPCs through a private network.

3. Functional characteristics

• By default, various cloud services are integrated by default. Cloud services such as ARMS, Alibaba Cloud Prometheus Monitoring, Cloud Monitoring, SLS, Alibaba Cloud Elasticsearch are integrated by default, and data source configuration, preset large disks and one-click alarms of various cloud services are provided.
• A variety of plug-ins can be used to connect your tools and your team using Grafana plug-ins. The data source plug-in connects to existing data sources via API and presents the data in real time, without the need for you to manually obtain or migrate the data.
• The custom alarm system uses Grafana alarms. You can create, manage and mute all alarms in a simple UI, allowing you to easily integrate and centralize all alarms (part of the function is under development).
• Multi-dimensional data query supports cross-data source query, data source renaming, summary, combination, and calculation.
• Self-built data source supports opening up multiple VPCs in the same region or adding data sources in multiple VPCs to the same workspace, and supports unified query display and alarms.
• The panel editor easily configures, customizes, and browses all panels through a consistent UI to set data options across all visualization panels.

I hope that through the Grafana hosting service, developers can solve various pain points encountered when using Grafana and get a more convenient experience. I also hope that all students can put forward valuable suggestions in order to make Grafana hosting services go further. Welcome to join the Grafana community, scan the code or search for "34785590" to join the group!

Click on the link below 👇 to apply for service now!

https://survey.aliyun.com/apps/zhiliao/vJytm8\_sj?spm=5176.24855332.J\_9932635580.2.4a633d1bQzGm1l

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.