Enterprise-level cloud native application delivery and management series - Helm debugging and maintenance (2)

Hello everyone, my name is Zhang Jintao.

In the last article "Enterprise-level cloud native application delivery and management series - Helm foundation (1)" , I mainly introduced
The birth and development of Helm, including the situation of various versions of Helm and the development of the community.

In addition, Helm's architecture, concepts, plugins, and its basic usage are introduced.

In this section, I will assume that the reader already has a certain Helm and Kubernetes foundation.
Introduce some high-level features and usage of Helm,
If you have unclear concepts, you can read my history article.

Prepare

Here we use the helm create command to create a Helm chart. After the execution is complete, a new directory will be created in the current directory, which contains a template pre-created by Helm.
The rest of this article will be based on this Helm chart.

 tao@moelove:~$ helm create moelove
Creating moelove                
tao@moelove:~$ ls                    
moelove
tao@moelove:~$ tree
.
└── moelove
    ├── Chart.yaml
    ├── charts
    ├── templates
    │   ├── NOTES.txt
    │   ├── _helpers.tpl
    │   ├── deployment.yaml
    │   ├── hpa.yaml
    │   ├── ingress.yaml
    │   ├── service.yaml
    │   ├── serviceaccount.yaml
    │   └── tests
    │       └── test-connection.yaml
    └── values.yaml

4 directories, 10 files

Debug

When we go to create/maintain, or use Helm chart for application deployment, sometimes we may encounter some errors.
So how to debug Helm charts? This is a problem that many people encounter.

Helm charts are maintained via YAML, which is indentation/syntax sensitive.
If your indentation or syntax is wrong, it will cause an error.
The easiest way to check is to use helm lint to check.

For example, we make the following changes:

 diff --git a/values.yaml b/values.yaml
index 4a8b237..696a77d 100644
--- a/values.yaml
+++ b/values.yaml
@@ -5,7 +5,7 @@
 replicaCount: 1
 
 image:
-  repository: nginx
+ repository: nginx
   pullPolicy: IfNotPresent
   # Overrides the image tag whose default is the chart appVersion.
   tag: ""

As you can see, we got the indentation of image.repository wrong.
At this time, when installing, you will see the following error:

 tao@moelove:~$ helm install foo .
Error: INSTALLATION FAILED: cannot load values.yaml: error converting YAML to JSON: yaml: line 9: mapping values are not allowed in this context

I suggest you can use helm lint to check first and avoid some low-level errors. as follows:

 tao@moelove:~$ helm lint .
==> Linting .
[INFO] Chart.yaml: icon is recommended
[ERROR] values.yaml: unable to parse YAML: error converting YAML to JSON: yaml: line 9: mapping values are not allowed in this context
[ERROR] templates/: cannot load values.yaml: error converting YAML to JSON: yaml: line 9: mapping values are not allowed in this context
[ERROR] : unable to load chart
        cannot load values.yaml: error converting YAML to JSON: yaml: line 9: mapping values are not allowed in this context

Error: 1 chart(s) linted, 1 chart(s) failed

We reverted this content to its original state and made the following changes:

 diff --git a/values.yaml b/values.yaml
index 4a8b237..c86c0be 100644
--- a/values.yaml
+++ b/values.yaml
@@ -2,7 +2,7 @@
 # This is a YAML-formatted file.
 # Declare variables to be passed into your templates.
 
-replicaCount: 1
+replicaCount: "this should not be string"
 
 image:
   repository: nginx

Modify replicaCount into a string. At this time, we use helm lint which cannot be checked.

 tao@moelove:~$ helm lint .
==> Linting .
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed

Because the type cannot be checked from the YAML syntax, and this is also limited by our specific business logic (Kubernetes).

At this time, if you install it, you will get the following error:

 tao@moelove:~$ helm install foo .
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.replicas): invalid type for io.k8s.api.apps.v1.DeploymentSpec.replicas: got "string", expected "integer"

As you can see, you can get feedback directly for this type of error.

But there is another situation, that is, the grammar rules and types are normal, but they do not meet the actual expectations of the business.

For example, we make the following changes:

 diff --git a/values.yaml b/values.yaml
index 4a8b237..8feedd6 100644
--- a/values.yaml                                                                                    
+++ b/values.yaml                                                                                    
@@ -8,7 +8,7 @@ image:
   repository: nginx
   pullPolicy: IfNotPresent
   # Overrides the image tag whose default is the chart appVersion.
-  tag: ""
+  tag: "1.20"
  
 imagePullSecrets: []
 nameOverride: ""

At this point, the installation can be successful, but maybe the image we expect to install is 1.20-alpine .
In this case, the above two methods are ineffective.

We can debug by helm install --dry-run --debug command.
Of course, if you want to debug via helm template it's fine too.

The main difference between the two is that if the --debug parameter is added, more detailed information can be output.
Including the final used Values information and so on.

This is my recommended approach and is suitable for use when you are developing/debugging Helm charts.

Helm unit tests

When it comes to unit testing of Helm charts, you may have questions, YAML also needs to write unit tests?

Yes. If you are the maintainer of a Helm chart, it is a good idea to write unit tests to better ensure that most of the content is as expected.

If you want to write unit tests for Helm charts, I have three tool recommendations.

• quintush/helm-unittest is a fork from helm-unittest/helm-unittest , but it is more active and contains a lot of features and fixes, plus it works well with Helm 3;
• conftest This is a tool based on Open Policy Agent (OPA), which completes configuration verification by writing policy files using Rego. I now use it in the Apache APISIX Ingress controller project to assist users with upgrade checks;
• terratest is a general-purpose testing framework developed in Go that supports tests with multiple configurations, including Helm, AWS/Docker, etc.;

Among them, my favorite is conftest, because I prefer to write Rego, and friends who are interested in OPA can refer to my previous article "Open Policy Agent (OPA) Getting Started Practice" .
The simplest is helm-unittest, which is more focused.

Here we use helm-unittest as an example.

Install

helm-unittest can be installed as Helm Plugin by executing the following command:

 tao@moelove:~$ helm plugin install https://github.com/quintush/helm-unittest
Support linux-amd64
...
Installed plugin: unittest

test

Just create a tests directory in the chart directory and write test files in it.

 tao@moelove:~$ cat tests/deployment_test.yaml
suite: test deployment
templates:
  - deployment.yaml
tests:
  - it: should work
    set:
      image.tag: latest
    asserts:
      - isKind:
          of: Deployment
      - matchRegex:
          path: metadata.name
          pattern: -moelove$
      - equal:
          path: spec.template.spec.containers[0].image
          value: nginx:latest

I can run it in the following way. Note that the -3 parameter must be added to be compatible with Helm v3.

 tao@moelove:~$ helm unittest -3 .

### Chart [ moelove ] .

 PASS  test deployment  tests/deployment_test.yaml

Charts:      1 passed, 1 total
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshot:    0 passed, 0 total
Time:        5.932023ms

We can tweak this case to see what happens if the test fails:

 tao@moelove:~$ cat tests/deployment_test.yaml 
suite: test deployment
templates:
  - deployment.yaml
tests:
  - it: should work
    set:
      image.tag: alpine
    asserts:
      - isKind:
          of: Deployment
      - matchRegex:
          path: metadata.name
          pattern: -moelove$
      - equal:
          path: spec.template.spec.containers[0].image
          value: nginx:latest

Here we set --- image.tag to alpine , but we didn't match it in the assertion, so after executing the test we see a failure.

 tao@moelove:~$ helm unittest -3 .

### Chart [ moelove ] .

 FAIL  test deployment  tests/deployment_test.yaml
        - should work

                - asserts[2] `equal` fail
                        Template:       moelove/templates/deployment.yaml
                        DocumentIndex:  0
                        Path:   spec.template.spec.containers[0].image
                        Expected to equal:
                                nginx:latest
                        Actual:
                                nginx:alpine
                        Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,2 +1,2 @@
                                -nginx:latest
                                +nginx:alpine


Charts:      1 failed, 0 passed, 1 total
Test Suites: 1 failed, 0 passed, 1 total
Tests:       1 failed, 0 passed, 1 total
Snapshot:    0 passed, 0 total
Time:        5.22252ms

Error: plugin "unittest" exited with error

We can clearly see the cause and location of the specific failure.

If you are maintaining a Helm chart and need to ensure that it is delivered with high quality, adding unit tests to it is a good idea.
As for the choice of tools, it mainly depends on personal preference. helm-unittest only needs to write YAML, while the other two tools write Rego and Go respectively.

Summarize

In this article, we mainly focus on the topic of debugging and maintaining Helm charts.
Introduces some built-in inspection tools for charts in Helm, and also introduces how to use helm-unittest to write unit tests for Helm charts.

Mastering these is very useful when maintaining, delivering, and using Helm charts. If this article is helpful to you, please like, forward, and leave a message for discussion.

If you are interested in this topic, I will update some advanced content of Helm in the future, so stay tuned!

---

Welcome to subscribe my article public account [MoeLove]