How to integrate ckman in the product

Preface

ckman is a visual open source tool developed by the database team, used to manage and monitor ClickHouse It simplifies and visualizes the configuration and monitoring of the ClickHouse github address is ckman , interested students can download the source code from here.

Although ckman provides the web interface, but for some companies, it may API , but more expect to integrate it into their own products. For example, through your own product page, you can directly call ckman of API , and use ckman as a server end, so that it is convenient for cluster management, database table management and data monitoring.

For security reasons, ckman added support https token for authentication. token greatly improves the security of the data link, but it provides a certain degree of difficulty for development (because we can get what token is through the login interface, but once ckman restarts or token expires, Will cause token fail).

It is precisely because of this situation that ckman provides another access plan in addition to token As long as the request header carry in userToken fields, and meet ckman default format specified, you can skip token authentication mechanism, so as to ensure that there will be no token invalid or expired automatically jump to the login page of the issue.

So userToken ensure the link security?

Here, ckman uses the encryption technology RSA Users can provide their own public and private keys, userToken the content of 06100268855908 with the private key, and put it in the requested header When the request is received by ckman userToken header , the corresponding field will be used. The public key is used to decrypt, thereby establishing a valid connection.

What we only need to do is to configure the public key to the configuration file of ckman

server:
  id: 1
  port: 8808
  https: false
  pprof: true
  session_timeout: 3600
  public_key:  #该字段配置公钥

So, userToken the default format of 06100268855964?

ckman provided by userToken is a class (structure) that contains fields such as timestamp and timeout. Its structure is as follows:

type UserTokenModel struct {
    Duration           int64
    RandomPaddingValue string
    UserId             int64
    Timestamp          int64
}

Among them, we only need to care about the Duration and Timestamp fields. Duration refers to a timeout, ckman will received the request and the current time userToken carried Timestamp to compare time, once the time interval exceeds Duration , it is considered out. Therefore, the timeout period is controlled by client , and the degree of freedom is relatively large.

Code demo

Since ckman is go language, so I also use go for demonstration here. (In fact, as long as the above specifications are met, the same effect can be achieved using any programming language).

First of all, we ckman configuration profile in public, on how to generate RSA public, there are many online tutorials here is not to go into details.

( ps : The public and private keys in the sample code are from github.com/housepower/ckman/common/rsa_test.go )

server:
  id: 1
  port: 8808
  https: false
  pprof: true
  session_timeout: 3600
  public_key: MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDNw9cbOh1JRVNf/pQiRRMoa4TSmgZeq9zyK+Z5qE0Ak1XcmFzRg1m667ZAgfl/gEiwMGtbKyiPBGeHP5Gw3z5ENIHg7WGKTE0yRM/U/FMnktjly2xzjf7HUl/IA7PFYq5KBVBNPhjwzuFxpmsJL+fhhuYB75uDL0axYwcm7WHdewIDAQAB

Then start ckman to keep the service running.

Then you can write the code for the client

For convenience, I directly quote the package of ckman go get github.com/housepower/ckman on the console. Of course, if you feel that ckman is too large, you can also implement a set of rsa encryption algorithm by private key, please refer to github.com/housepower/ckman/common/rsa.go .

The code example is as follows:

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
    "time"

    "github.com/housepower/ckman/common"
)

const PRI_KEY string = "MIICXgIBAAKBgQDNw9cbOh1JRVNf/pQiRRMoa4TSmgZeq9zyK+Z5qE0Ak1XcmFzRg1m667ZAgfl/gEiwMGtbKyiPBGeHP5Gw3z5ENIHg7WGKTE0yRM/U/FMnktjly2xzjf7HUl/IA7PFYq5KBVBNPhjwzuFxpmsJL+fhhuYB75uDL0axYwcm7WHdewIDAQABAoGBAIKxMz1t6hAR4mUEc95YdVSlBhYmEomrK4j97UO0bERDULPuanYAscuRz46lf21Gc+TEvEuJ3BcKux8id00aXpcbbNhqIDyUMvET4MjdisgXhxay/dzc6jRBYQdhMrLT0NYfQSbULdXA3CGQhti4nChazn708ag6slvjGtsC4O9BAkEA9c/ZmbKisBb3GweWP/IhYB+GO5Qsby0KkF582NgnGIjnpGirniO2jyNSXO72QerTfG4JXqofGkH7AmlO0bkX0QJBANZLFBzoRIJr8x32dsKnd/V/7k2OgNbrUGwFJrJOGCSClPF7yM3xjN0lg3EjKW4AZP75pr//vOLOYTQDHyeNv4sCQQCoUlzyJ2XJ6N/q7WYQgbAjD1MuxwcqVhBuzZT2NAWJgm4EofwqvM/M8mX651NPzgploT/fR+UmaNoGS7BCYlmRAkAFAY3/uuFW1qTAT3CozXa88ncjsq+J1cd0Lo6f3bksqSxHk+e1/+2VgPnYG8Us/69cUYK2u4ezGLUmnOgOaX5PAkEA6wwIjYGDQRYIEVD4oJyNtdL7FFso63lon3LMySxLgi/KZGS4N8+FYJQVIzWWCrdk3Z1mXw4wuOQkE4pDy8xx+w=="

func main() {
    //构建userToken结构体
    userTokenModel := common.UserTokenModel{
        Duration:           3600,                        //超时时间，这里设的是1个小时
        RandomPaddingValue: "ckmandemo",                 //随机值，eoi产品中使用的，ckman中用不到
        UserId:             123456,                      //用户id，eoi产品中可以设置不同的用户权限，ckman中用不到
        Timestamp:          time.Now().UnixNano() / 1e6, //当前时间，单位为毫秒
    }

    //将userToken结构体使用私钥加密
    uenc, _ := json.Marshal(userTokenModel)
    var rsaEncrypt common.RSAEncryption
    userToken, err := rsaEncrypt.Encode(uenc, PRI_KEY)
    if err != nil {
        fmt.Println("rsa encode error:", err)
        return
    }

    //创建http客户端
    client := http.Client{}

    //示例中我们调用/api/v1/ck/get 接口去获取ipv4集群的节点状态
    url := "http://localhost:8808/api/v1/ck/get/ipv4"
    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        fmt.Println("http request failed:", err)
        return
    }

    req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
    req.Header.Set("Content-type", "application/json")
    req.Header.Add("userToken", string(userToken)) //将userToken添加到header中，可以看到，这里并没有登录，也没有在header中添加token
    response, err := client.Do(req)
    if err != nil {
        fmt.Println("http error:", err)
        return
    }
    if response.StatusCode == 200 {
        defer response.Body.Close()
        body, err := ioutil.ReadAll(response.Body)
        if err != nil {
            fmt.Println("response error:", err)
            return
        }
        //打印返回结果
        fmt.Println(string(body))
    } else {
        fmt.Println("response:", response.Status)
        return
    }
}

The results of the operation are as follows:

chenyc@YenchangChan:ckmanDemo$go run main.go 
{"retCode":0,"retMsg":"ok","entity":{"status":"green","version":"21.3.9.83","nodes":[{"ip":"192.168.21.51","hostname":"node1","status":"green","shardNumber":1,"replicaNumber":1,"disk":"36.95GB/49.98GB"},{"ip":"192.168.21.52","hostname":"node2","status":"green","shardNumber":1,"replicaNumber":2,"disk":"28.15GB/49.98GB"},{"ip":"192.168.21.53","hostname":"node3","status":"green","shardNumber":2,"replicaNumber":1,"disk":"28.71GB/49.98GB"},{"ip":"192.168.21.54","hostname":"node4","status":"green","shardNumber":2,"replicaNumber":2,"disk":"23.27GB/49.98GB"}],"mode":"deploy","needPassword":false}}

This is consistent with the result of our web-side visit:

If we use F12 turn on the debug mode, we will find that the JSON returned by the interface is consistent with the JSON output by the code.

The above is a simple demonstration of the GET request. If it is POST or other requests, the same operation is required, so I will not demonstrate it here. ckman swagger , please refer to the 06100268855bd0 document.

To open the swagger document, you need to add the swagger_enable configuration item in the configuration file:

server:
  id: 1
  port: 8808
  https: false
  pprof: true
  swagger_enable: true    #配置此项即可使用swagger文档
  session_timeout: 3600
  public_key: MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDNw9cbOh1JRVNf/pQiRRMoa4TSmgZeq9zyK+Z5qE0Ak1XcmFzRg1m667ZAgfl/gEiwMGtbKyiPBGeHP5Gw3z5ENIHg7WGKTE0yRM/U/FMnktjly2xzjf7HUl/IA7PFYq5KBVBNPhjwzuFxpmsJL+fhhuYB75uDL0axYwcm7WHdewIDAQAB

After restarting, you can log in to http://localhost:8808/swagger/index.html to view. The interface is as follows:

Summarize

This article briefly introduces how to use code to operate ckman in actual development. You can token RSA public and private keys. This helps to ckman the call of 06100268855cbb with the company’s own products, making ckman not only It is only an independent management tool, and it also plays the role of a server server, which greatly expands the usage scenarios ckman