kubebuilder combat seven: webhook

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: Classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

Overview of this article

• This article is the seventh article in the "kubebuilder actual combat" series. In the previous article, we completed the design, development, deployment, and verification process of an Operator. In order to keep the whole process concise and not bloated, we deliberately skipped an important one in actual combat. Knowledge point: <font color="red">webhook</font>, now is the time to learn it, this is a very important function;
• This article consists of the following parts:
• Introduce webhook;
• Combine the previous elasticweb project to design a scene using webhook;
• Ready to work
• Generate webhook
• Development (configuration)
• Development (coding)
• deploy
• Verify Defaulter (add default value)
• Validation Validator (validity check)

About webhook

• Most readers who are familiar with java development know the filter (Servlet Filter), as shown in the figure below, the external request will first reach the filter and do some unified operations, such as transcoding and verification, and then the real business logic will process the request:

• The webhook in the Operator has a similar function to the above filters. The external changes to the CRD resources will be processed by the webhook in advance before being processed by the Controller. The process is as follows, which is from "Getting Started with Kubernetes | Operator and Operator Framework" :

• Let’s take a look at what webhook does. As shown in the figure below, kubernetes official blog clearly points out that webhook can do two things: mutating and validating.
  
• kubebuilder provides us with tools to generate basic files and codes for webhooks, which are similar to tools for making APIs, which greatly simplifies the workload. We only need to focus on business realization;
• Based on the webhook and controller made by kubebuilder, if they are the same resource, then <font color="red">they are in the same process</font>;

Design actual combat scenarios

• In order to make the actual combat meaningful, let's increase the demand for the previous elasticweb project and let the webhook play a practical role;

1. If the user forgets to enter the total QPS, the system webhook is responsible for setting the default value <font color="red">1300</font>, the operation is as follows:

2. In order to protect the system, set the upper limit <font color="red">1000</font> for the QPS of a single pod. If the singlePodQPS value input from outside exceeds 1000, <font color="blue"> will fail to create a resource object</font> >, as shown in the figure below:

Source download

• The complete source code in this actual combat can be downloaded on GitHub. The address and link information are shown in the following table ( https://github.com/zq2599/blog_demos):

name	link	Remarks
Project homepage	https://github.com/zq2599/blog_demos	The project's homepage on GitHub
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	[e-mail]:zq2599/blog_demos.git	The warehouse address of the source code of the project, ssh protocol

• There are multiple folders in this git project, and kubebuilder-related applications are under the <font color="blue">kubebuilder</font> folder, as shown in the red box below:

• There are multiple subfolders under the kubebuilder folder. The corresponding source code of this article is in the <font color="blue">elasticweb</font> directory, as shown in the red box in the following figure:

Ready to work

• Similar to the controller, webhook can run in the kubernetes environment or outside of the kubernetes environment;
• If the webhook runs outside the kubernetes environment, it is a bit troublesome. You need to put the certificate in the environment. The default address is:

/tmp/k8s-webhook-server/serving-certs/tls.{crt,key}

• In order to save trouble and to be closer to the usage in the production environment, the next actual combat method is to <font color="red">deploy the webhook in the kubernetes environment</font>
• In order to make the webhook run in the kubernetes environment, we need to do some preparations <font color="blue">install the cert manager</font>, perform the following operations:

• kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.yaml

• After the above operations are completed, many resources will be created, such as namespace, rbac, pod, etc. Take pod as an example:

[root@hedy ~]# kubectl get pods --all-namespaces
NAMESPACE        NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager     cert-manager-6588898cb4-nvnz8              1/1     Running   1          5d14h
cert-manager     cert-manager-cainjector-7bcbdbd99f-q645r   1/1     Running   1          5d14h
cert-manager     cert-manager-webhook-5fd9f9dd86-98tm9      1/1     Running   1          5d14h
...

• After the operation is completed, the preparatory work is over, and the actual combat can begin;

Generate webhook

• Enter the elasticweb project and execute the following command to create a webhook:

kubebuilder create webhook \
--group elasticweb \
--version v1 \
--kind ElasticWeb \
--defaulting \
--programmatic-validation

• After the above command is executed, first check the <font color="blue">main.go</font> file, as shown in the red box 1 in the figure below, a piece of code is automatically added to make the webhook take effect:

• The <font color="blue">elasticweb_webhook.go</font> in the red box 2 above is the new file, the content is as follows:

package v1

import (
    "k8s.io/apimachinery/pkg/runtime"
    ctrl "sigs.k8s.io/controller-runtime"
    logf "sigs.k8s.io/controller-runtime/pkg/log"
    "sigs.k8s.io/controller-runtime/pkg/webhook"
)

// log is for logging in this package.
var elasticweblog = logf.Log.WithName("elasticweb-resource")

func (r *ElasticWeb) SetupWebhookWithManager(mgr ctrl.Manager) error {
    return ctrl.NewWebhookManagedBy(mgr).
        For(r).
        Complete()
}

// EDIT THIS FILE!  THIS IS SCAFFOLDING FOR YOU TO OWN!

// +kubebuilder:webhook:path=/mutate-elasticweb-com-bolingcavalry-v1-elasticweb,mutating=true,failurePolicy=fail,groups=elasticweb.com.bolingcavalry,resources=elasticwebs,verbs=create;update,versions=v1,name=melasticweb.kb.io

var _ webhook.Defaulter = &ElasticWeb{}

// Default implements webhook.Defaulter so a webhook will be registered for the type
func (r *ElasticWeb) Default() {
    elasticweblog.Info("default", "name", r.Name)

    // TODO(user): fill in your defaulting logic.
}

// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation.
// +kubebuilder:webhook:verbs=create;update,path=/validate-elasticweb-com-bolingcavalry-v1-elasticweb,mutating=false,failurePolicy=fail,groups=elasticweb.com.bolingcavalry,resources=elasticwebs,versions=v1,name=velasticweb.kb.io

var _ webhook.Validator = &ElasticWeb{}

// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *ElasticWeb) ValidateCreate() error {
    elasticweblog.Info("validate create", "name", r.Name)

    // TODO(user): fill in your validation logic upon object creation.
    return nil
}

// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *ElasticWeb) ValidateUpdate(old runtime.Object) error {
    elasticweblog.Info("validate update", "name", r.Name)

    // TODO(user): fill in your validation logic upon object update.
    return nil
}

// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (r *ElasticWeb) ValidateDelete() error {
    elasticweblog.Info("validate delete", "name", r.Name)

    // TODO(user): fill in your validation logic upon object deletion.
    return nil
}

• There are two points to note in the above code. The first part is related to filling in the default values, as shown in the following figure:

• The second part is related to verification, as shown in the figure below:

• The business requirements we want to achieve are achieved by modifying the content of the above <font color="blue">elasticweb_webhook.go</font>, but the code will be written later, and the configuration will be changed first;

Development (configuration)

• Open the file <font color="blue">config/default/kustomization.yaml</font>, the contents in the four red boxes in the figure below are originally annotated, now please delete all the annotation symbols to make it effective:

• Still the file <font color="blue">config/default/kustomization.yaml</font>, the content below the node <font color="red">vars</font>, originally all annotated, now please put them all After the release, the effect is as shown in the figure below:

• The configuration has been completed and can be coded;

Development (coding)

• Open the file <font color="blue">elasticweb_webhook.go</font>
• New dependency:

apierrors "k8s.io/apimachinery/pkg/api/errors"

• Find the <font color="blue">Default</font> method and change it to the following content. It can be seen that the code is very simple, judging whether TotalQPS exists, and if it does not exist, write the default value, and add two lines of log:

func (r *ElasticWeb) Default() {
    elasticweblog.Info("default", "name", r.Name)

    // TODO(user): fill in your defaulting logic.
    // 如果创建的时候没有输入总QPS，就设置个默认值
    if r.Spec.TotalQPS == nil {
        r.Spec.TotalQPS = new(int32)
        *r.Spec.TotalQPS = 1300
        elasticweblog.Info("a. TotalQPS is nil, set default value now", "TotalQPS", *r.Spec.TotalQPS)
    } else {
        elasticweblog.Info("b. TotalQPS exists", "TotalQPS", *r.Spec.TotalQPS)
    }
}

• Next, we develop the verification function. We encapsulate the verification function into a validateElasticWeb method, and then call it once when adding and modifying it. As shown below, it can be seen that the final call is <font color="blue">apierrors.NewInvalid</font > The error instance is generated, and this method accepts multiple errors, so it is necessary to prepare the slice as an input parameter. Of course, if multiple parameter verification fails, you can put them into the slice:

func (r *ElasticWeb) validateElasticWeb() error {
    var allErrs field.ErrorList

    if *r.Spec.SinglePodQPS > 1000 {
        elasticweblog.Info("c. Invalid SinglePodQPS")

        err := field.Invalid(field.NewPath("spec").Child("singlePodQPS"),
            *r.Spec.SinglePodQPS,
            "d. must be less than 1000")

        allErrs = append(allErrs, err)

        return apierrors.NewInvalid(
            schema.GroupKind{Group: "elasticweb.com.bolingcavalry", Kind: "ElasticWeb"},
            r.Name,
            allErrs)
    } else {
        elasticweblog.Info("e. SinglePodQPS is valid")
        return nil
    }
}

• Then find the method that is called when adding and modifying the resource object, and call validateElasticWeb in it:

// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *ElasticWeb) ValidateCreate() error {
    elasticweblog.Info("validate create", "name", r.Name)

    // TODO(user): fill in your validation logic upon object creation.

    return r.validateElasticWeb()
}

// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *ElasticWeb) ValidateUpdate(old runtime.Object) error {
    elasticweblog.Info("validate update", "name", r.Name)

    // TODO(user): fill in your validation logic upon object update.
    return r.validateElasticWeb()
}

• The coding is complete, it can be seen that it is very simple. Next, let's clean up the things left over from the previous actual combat, and then start new deployment and verification;

Clean up

• If you follow the "kubebuilder actual combat" series all the way down, the system should accumulate the leftover content at this time, you can complete the cleanup by the following steps:
• Delete elasticweb resource object:

kubectl delete -f config/samples/elasticweb_v1_elasticweb.yaml

2. Delete controller

kustomize build config/default | kubectl delete -f -

3. Delete CRD

make uninstall

• Now everything is ready, webhook can be deployed;

deploy

1. Deploy CRD

make install

2. Build the image and push it to the warehouse (I finally had enough of the turtle speed of <font color="blue">hub.docker.com</font> and changed it to Alibaba Cloud Image Warehouse):

make docker-build docker-push IMG=registry.cn-hangzhou.aliyuncs.com/bolingcavalry/elasticweb:001

3. Deploy the controller integrated with the webhook function:

make deploy IMG=registry.cn-hangzhou.aliyuncs.com/bolingcavalry/elasticweb:001

4. Check the pod to confirm that the startup is successful:

zhaoqin@zhaoqindeMBP-2 ~ % kubectl get pods --all-namespaces
NAMESPACE           NAME                                             READY   STATUS    RESTARTS   AGE
cert-manager        cert-manager-6588898cb4-nvnz8                    1/1     Running   1          5d21h
cert-manager        cert-manager-cainjector-7bcbdbd99f-q645r         1/1     Running   1          5d21h
cert-manager        cert-manager-webhook-5fd9f9dd86-98tm9            1/1     Running   1          5d21h
elasticweb-system   elasticweb-controller-manager-7dcbfd4675-898gb   2/2     Running   0          20s

Verify Defaulter (add default value)

• Modify the file <font color="blue">config/samples/elasticweb_v1_elasticweb.yaml</font>, the modified content is as follows, it can be seen that the <font color="blue">totalQPS</font> field has been commented out:

apiVersion: v1
kind: Namespace
metadata:
  name: dev
  labels:
    name: dev
---
apiVersion: elasticweb.com.bolingcavalry/v1
kind: ElasticWeb
metadata:
  namespace: dev
  name: elasticweb-sample
spec:
  # Add fields here
  image: tomcat:8.0.18-jre8
  port: 30003
  singlePodQPS: 500
  # totalQPS: 600

• Create an elasticweb resource object:

kubectl apply -f config/samples/elasticweb_v1_elasticweb.yaml

• At this time, the QPS of a single pod is 500. If the webhook code takes effect, the total QPS is 1300, and the number of corresponding pods should be 3. Next, let’s see if it meets expectations;
• First, see if resource objects such as elasticweb, deployment, pod, etc. are normal, as shown below, all are in line with expectations:

zhaoqin@zhaoqindeMBP-2 ~ % kubectl get elasticweb -n dev                                                                 
NAME                AGE
elasticweb-sample   89s
zhaoqin@zhaoqindeMBP-2 ~ % kubectl get deployments -n dev
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
elasticweb-sample   3/3     3            3           98s
zhaoqin@zhaoqindeMBP-2 ~ % kubectl get service -n dev    
NAME                TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
elasticweb-sample   NodePort   10.105.125.125   <none>        8080:30003/TCP   106s
zhaoqin@zhaoqindeMBP-2 ~ % kubectl get pod -n dev    
NAME                                 READY   STATUS    RESTARTS   AGE
elasticweb-sample-56fc5848b7-5tkxw   1/1     Running   0          113s
elasticweb-sample-56fc5848b7-blkzg   1/1     Running   0          113s
elasticweb-sample-56fc5848b7-pd7jg   1/1     Running   0          113s

• Use the <font color="blue">kubectl describe</font> command to view the details of the elasticweb resource object, as shown below, the TotalQPS field is set to 1300 by the webhook, and the RealQPS is also calculated correctly:

zhaoqin@zhaoqindeMBP-2 ~ % kubectl describe elasticweb elasticweb-sample -n dev
Name:         elasticweb-sample
Namespace:    dev
Labels:       <none>
Annotations:  <none>
API Version:  elasticweb.com.bolingcavalry/v1
Kind:         ElasticWeb
Metadata:
  Creation Timestamp:  2021-02-27T16:07:34Z
  Generation:          2
  Managed Fields:
    API Version:  elasticweb.com.bolingcavalry/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:image:
        f:port:
        f:singlePodQPS:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2021-02-27T16:07:34Z
    API Version:  elasticweb.com.bolingcavalry/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:realQPS:
    Manager:         manager
    Operation:       Update
    Time:            2021-02-27T16:07:34Z
  Resource Version:  687628
  UID:               703de111-d859-4cd2-b3c4-1d201fb7bd7d
Spec:
  Image:           tomcat:8.0.18-jre8
  Port:            30003
  Single Pod QPS:  500
  Total QPS:       1300
Status:
  Real QPS:  1500
Events:      <none>

• Let’s take a look at the controller’s log. The webhook part is as expected, as shown in the red box below. If the TotalQPS field is found to be empty, it will be set to the default value, and the value of SinglePodQPS did not exceed 1000 at the time of detection:

• Finally, don’t forget to verify whether the web service is normal with a browser. My full address here is: http://192.168.50.75:30003/
• So far, we have completed the Defaulter verification of the webhook, and then verify the Validator

Validator

• Next, it’s time to verify the parameter verification function of the webhook, first verify the logic of the modification;
• Edit the file config/samples/update_single_pod_qps.yaml, the values are as follows:

spec:
  singlePodQPS: 1100

• Use the patch command to make it effective:

kubectl patch elasticweb elasticweb-sample \
-n dev \
--type merge \
--patch "$(cat config/samples/update_single_pod_qps.yaml)"

• At this time, the console will output an error message:

Error from server (ElasticWeb.elasticweb.com.bolingcavalry "elasticweb-sample" is invalid: spec.singlePodQPS: Invalid value: 1100: d. must be less than 1000): admission webhook "velasticweb.kb.io" denied the request: ElasticWeb.elasticweb.com.bolingcavalry "elasticweb-sample" is invalid: spec.singlePodQPS: Invalid value: 1100: d. must be less than 1000

• Then use the <font color="blue">kubectl describe</font> command to view the details of the elasticweb resource object, as shown in the red box in the following figure, which is still 500. It can be seen that the webhook has taken effect and prevented the error from occurring:

• Look at the controller log again, as shown in the red box in the following figure, which corresponds to the code:

• Next, try the verification function of webhook when it is added;
• To clean up the elastic resource object created earlier, execute the command:

kubectl delete -f config/samples/elasticweb_v1_elasticweb.yaml

• Modify the file as shown in the red box in the following figure. Let's change the value of singlePodQPS to more than <font color="red">1000</font> to see if webhook can detect this error and prevent the creation of resource objects:

• Execute the following command to start creating elasticweb resource object:

kubectl apply -f config/samples/elasticweb_v1_elasticweb.yaml

• The console prompts the following information, including the error description written in our code, which proves that the creation of the elasticweb resource object failed, and proves that the Validator function of webhook has taken effect:

namespace/dev created
Error from server (ElasticWeb.elasticweb.com.bolingcavalry "elasticweb-sample" is invalid: spec.singlePodQPS: Invalid value: 1500: d. must be less than 1000): error when creating "config/samples/elasticweb_v1_elasticweb.yaml": admission webhook "velasticweb.kb.io" denied the request: ElasticWeb.elasticweb.com.bolingcavalry "elasticweb-sample" is invalid: spec.singlePodQPS: Invalid value: 1500: d. must be less than 1000

• If you are not at ease, execute the kubectl get command to check and find that it is empty:

zhaoqin@zhaoqindeMBP-2 elasticweb % kubectl get elasticweb -n dev       
No resources found in dev namespace.
zhaoqin@zhaoqindeMBP-2 elasticweb % kubectl get deployments -n dev
No resources found in dev namespace.
zhaoqin@zhaoqindeMBP-2 elasticweb % kubectl get service -n dev
No resources found in dev namespace.
zhaoqin@zhaoqindeMBP-2 elasticweb % kubectl get pod -n dev
No resources found in dev namespace.

• Also look at the controller log, as shown in the red box in the following figure, which is in line with expectations:

• At this point, we have completed the development, deployment, and verification of the operator's webhook. The entire elasticweb is considered to have complete basic functions, and hope to provide a reference for your operator development;

You are not alone, Xinchen and original are with you all the way

1. Java series
2. Spring series
3. Docker series
4. kubernetes series
5. database + middleware series
6. DevOps series

Welcome to pay attention to the public account: programmer Xin Chen

Search "Programmer Xin Chen" on WeChat, I am Xin Chen, and I look forward to traveling the Java world with you...
https://github.com/zq2599/blog_demos