Detailed analysis of application parameter configuration cases in Kubernetes

✏️ About the author:
Zong Yufen, Zilliz test development engineer, master of computer technology engineering from Huazhong University of Science and Technology. Currently focusing on the quality assurance work of Milvus database, including but not limited to interface testing, SDK testing, Benchmark testing, etc. A test and development kid who likes to locate problems and loves to explore chaos engineering theory and fault drill practice.

How to modify Milvus Advanced Configuration

background

When using Milvus, we may wish to modify the default parameters to meet the needs of different scenarios. Not long ago, Milvus users shared how to modify the configuration file when deploying with docker-compose (click to read: technology sharing | How to configure Milvus 2.0 parameters ). This article will briefly introduce how to modify the configuration file when deploying Milvus with Kubernetes Configuration parameters.

Different parameter configurations can choose different modification schemes. All Milvus configuration files are located under the /milvus/configs/ path. When using Kubernetes to install the Milvus application, we will add the Milvus Chart warehouse. After adding it, use the command helm show values milvus/milvus view the parameter items that Chart supports configuration. If these parameter items contain the parameters we want to modify, you can --values or --set . For specific usage details, please refer to Milvus Helm Chart or Helm ; if the parameters we want to modify are not included, You can consider the small methods described below:

The configuration file management of Milvus in Kubernetes is realized through the ConfigMap resource object. If we need to modify the parameters other than the Milvus Chart configurable options, we need to update the ConfigMap object corresponding to the Chart Release first, and then modify the deployment resource file of the corresponding Pod. Next, take the modification of the rootcoord.dmlChannelNum parameter in the /milvus/configs/advanced/root_coord.yaml file as an example. Follow the two steps of modifying the properties of the ConfigMap object and then modifying the properties of the Deployment object, and modify its value from 256 to 128 .

It should be noted that this method only modifies the configuration of the Milvus application that has been deployed. If you need to modify the configuration in /milvus/configs/advanced/*.yaml during deployment or before deployment, we need to redevelop the Milvus Helm Chart.

Modify the ConfigMap manifest file

The Release running in Kubernetes corresponds to the ConfigMap object milvus-chaos data attribute only contains the configuration of the milvus.yaml file. In the same way, we need to rootcoord.dmlChannelNum where the parameter 06169432177057 is located in the data rootcoord.dmlChannelNum to 128 at the same time.

kind: ConfigMap
apiVersion: v1
metadata:
  name: milvus-chaos
  ...
data:
  milvus.yaml: >
    ......
  root_coord.yaml: |
    rootcoord:
      dmlChannelNum: 128
      maxPartitionNum: 4096
      minSegmentSizeToEnableIndex: 1024
      timeout: 3600 # time out, 5 seconds
      timeTickInterval: 200 # ms

Modify the Deployment manifest file

The data stored in the ConfigMap object can be configMap , and then the configuration data can be injected into the Pod to be used by the containerized application running in the Pod. If we want the Pod to access the new configuration file, we need to modify those Pod templates that will load the root_coord.yaml configuration. Specifically, add a mount statement under spec.template.spec.containers.volumeMounts Take the deployment resource list of the rootcoord pod as an example. From the spec.template.spec.volumes you can see that the top level of the Pod declares a milvus-config named milvus-config, the type is configMap of Volume , and the rootcoord container in the Pod declares to milvus-chaos the milvus.yaml file of the volume 061694321770dd. Load it under the path /milvus/configs/milvus.yaml. In the same way, we only need to mount the root_coord.yaml file to the /milvus/configs/advanced/root_coord.yaml path so that the container can access it.

spec:
  replicas: 1
  selector:
    ......
  template:
    metadata:
      ...
    spec:
      volumes:
        - name: milvus-config
          configMap:
            name: milvus-chaos
            defaultMode: 420
      containers:
        - name: rootcoord
          image: 'milvusdb/milvus-dev:master-20210906-86afde4'
          args:
            ...
          ports:
            ...
          resources: {}
          volumeMounts:
            - name: milvus-config
              readOnly: true
              mountPath: /milvus/configs/milvus.yaml
              subPath: milvus.yaml
            - name: milvus-config
              readOnly: true
              mountPath: /milvus/configs/advanced/root_coord.yaml
              subPath: root_coord.yaml
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler

Validation results

After completing the above two steps of modification, the Pod remounts the ConfigMap volume, and after the modification of the ConfigMap properties is detected, the Pod will be updated on a rolling basis. When the new Pod enters the Running state again, we can enter the Pod to verify whether the modification is successful. The specific commands are as follows. You can see that rootcoord.dmlChannelNum in the /milvus/configs/advanced/root_coord.yaml file has been updated to 128.

$ kctl exec -ti milvus-chaos-rootcoord-6f56794f5b-xp2zs -- sh

# cd configs/advanced
# pwd
/milvus/configs/advanced
# ls
channel.yaml  common.yaml  data_coord.yaml  data_node.yaml  etcd.yaml  proxy.yaml  query_node.yaml  root_coord.yaml

# cat root_coord.yaml
rootcoord:
  dmlChannelNum: 128
  maxPartitionNum: 4096
  minSegmentSizeToEnableIndex: 1024
  timeout: 3600 # time out, 5 seconds
  timeTickInterval: 200 # ms
# exit

So far, the method of modifying Milvus configuration has been introduced. In the later versions of Milvus, we will place the configuration parameters that users care about in a file, and support configuration updates through Helm Chart. Before the new version is born, I hope that the temporary modification plan introduced in this document can be helpful to everyone!