Understand Kubernetes admission control (Admission Controller)

Hi everyone, this is Zhang Jintao.

In my previous article "Container Image Security in the Cloud Native Era" (series), I mentioned the core component of the Kubernetes cluster - kube-apiserver, which allows various components from end users or clusters to interact with it. To communicate (for example, query, create, modify, or delete Kubernetes resources).

In this article, we will focus on a very important part of the kube-apiserver request processing process-the admission controller (Admission Controller)

What is the admission controller of K8s

Request processing flow in K8s

Before talking about what the K8s admission controller is, let's review the process of handling specific requests in the Kubernetes API.

Figure 1, Kubernetes API processing request process (from API Handler to etcd persistence process)

As shown in the figure above, each API request is received by kube-apiserver and finally persisted to ETCD, which is the request processing flow of Kubernetes API.

It mainly contains the following parts:

• API Handler - Mainly responsible for providing services and receiving requests.

For its internal implementation, the request will first come to FullHandlerChain (it is DefaultBuildHandlerChain ) is a director object

type director struct {
    name               string
    goRestfulContainer *restful.Container
    nonGoRestfulMux    *mux.PathRecorderMux
}

director is initialized according to the configuration. If the goRestfulContainer of the WebServices of 061a593c7bcae2 is /apis , or the request prefix matches the RootPath, then enter the Restful processing link.

• Authentication - authentication process.

After the TLS connection is established, the authentication process will be carried out. If the request for authentication fails, the request will be rejected and a 401 error code will be returned; if the authentication is successful, it will proceed to the authentication part. There are many client authentication methods currently supported, such as: x509 client certificate, Bearer Token, authentication based on username and password, OpenID authentication, etc. Since these content is not the focus of this article, we will skip it for now, and interested friends can leave a message in the comment area for discussion.

• Authorization - authentication process.

For Kubernetes, it supports multiple authentication modes, such as ABAC mode, RBAC mode, and Webhook mode. When we create a cluster, we can directly pass parameters to kube-apiserver for configuration, which will not be repeated here.

• Mutating Admission - Refers to the admission controller that can be used for changing operations, which will be described in detail below.

• Object Schema Validation - Schema validation of resource objects.

• Validating Admission - Refers to the admission controller that can be used for verification operations, which will be described in detail below.

• ETCD - ETCD implements persistent storage of resources.

The above is a request processing flow, in which Mutating Admission and Validating Admission are our protagonists today. Let's take a look at it in detail.

What is an admission controller (Admission Controller)

The admission controller refers to some codes or functions that can be used to change or verify operations request is authenticated and authorized

The admission control process is divided into two stages:

• The first stage is to run the Mutating Admission controller. It can modify the objects it accepts, which leads to another role of it, making changes to related resources as part of the request processing;

• The second stage is to run the Validating Admission controller. It can only perform verification and cannot modify any resource data;

It should be noted that some controllers can be both a change admission controller and a verification admission controller. If the admission controller at any stage rejects the request, the entire request will be rejected immediately and an error will be returned to the end user.

Why do you need an admission controller

We mainly understand why we need access controllers from two perspectives:

• From a security perspective

• • We need to clarify whether the source of the image deployed in the Kubernetes cluster is credible to avoid being attacked;
  • Under normal circumstances, try not to use root users in Pod, or try not to open privileged containers, etc.;

• From a governance perspective

• • For example, to distinguish businesses/services by label, you can verify whether the service already has a corresponding label through the admission controller;
  • For example, add resource quota restrictions to avoid resource oversold situations;

Admission controller

Considering that these requirements are more useful & indeed more needed, Kubernetes has implemented many built-in admission controllers. You can refer to the official document for a detailed list: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#what-does-each-admission-controller-do

These built-in admission controllers are built together with kube-apiserver in the form of plug-ins, and you can enable and disable them. For example, use the following parameters to control:

➜  bin ./kube-apiserver --help |grep admission-plugins    
      --admission-control strings              Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.)
      --disable-admission-plugins strings      admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
      --enable-admission-plugins strings       admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

There are two special admission controllers among so many admission controllers, namely MutatingAdmissionWebhook and ValidatingAdmissionWebhook . They do not actually implement the corresponding strategy, but provide an extensible way for kube-apiserver. Users can MutatingAdmissionWebhook and ValidatingAdmissionWebhook , and this way does not need to compile kube-spiserver Or restart, which is completely dynamic and very convenient.

Let's take a look at it in detail.

Dynamic admission controller

MutatingAdmissionWebhook that uses the 061a593c7bd000 and ValidatingAdmissionWebhook mentioned above for runtime configuration is the dynamic admission controller.

It is an HTTP callback mechanism used to receive and process admission requests, and it is a Web service. There are currently two types of access webhooks:

• validating admission webhook
• mutating admission webhook

The mutating admission webhook will be called first, and resources can be modified during this process.

If we need to ensure the final state of the object to perform certain operations, we should consider using validating admission webhook , because the request that reaches this stage will not be modified.

Conditions of Use

• Ensure that the Kubernetes cluster version is at least v1.16 (to use admissionregistration.k8s.io/v1 API) or admissionregistration.k8s.io/v1beta1 (to use 061a593c7bd0ee API);

• Ensure that the MutatingAdmissionWebhook and ValidatingAdmissionWebhook access controllers have been enabled;

• Ensure that the admissionregistration.k8s.io/v1beta1 or admissionregistration.k8s.io/v1 API is enabled;

What is admission webhook

It is actually an ordinary HTTP Server, and what needs to be processed is AdmissionReview type 061a593c7bd1d1. Let's look at an example. For example, we need to check the access Ingress

func (ia *IngressAdmission) HandleAdmission(obj runtime.Object) (runtime.Object, error) {

    review, isV1 := obj.(*admissionv1.AdmissionReview)
    if !isV1 {
        return nil, fmt.Errorf("request is not of type AdmissionReview v1")
    }

    if !apiequality.Semantic.DeepEqual(review.Request.Kind, ingressResource) {
        return nil, fmt.Errorf("rejecting admission review because the request does not contain an Ingress resource but %s with name %s in namespace %s",
            review.Request.Kind.String(), review.Request.Name, review.Request.Namespace)
    }

    status := &admissionv1.AdmissionResponse{}
    status.UID = review.Request.UID

    ingress := networking.Ingress{}

    codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
        Pretty: true,
    })
    codec.Decode(review.Request.Object.Raw, nil, nil)
    _, _, err := codec.Decode(review.Request.Object.Raw, nil, &ingress)
    if err != nil {
        klog.ErrorS(err, "failed to decode ingress")
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    if err := ia.Checker.CheckIngress(&ingress); err != nil {
        klog.ErrorS(err, "invalid ingress configuration", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    klog.InfoS("successfully validated configuration, accepting", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
    status.Allowed = true
    review.Response = status

    return review, nil
}

Core processing logic is actually processing the request Webhook send when AdmissionReview , it will contain the resources we object to be verified. Then we verify or modify the resource object according to actual needs.

There are a few points to note here:

• The processing of Mutating Webhook is serial, while the processing of Validating Webhook is parallel;
• Although the processing of Mutating Webhook is serial, the order is not guaranteed;
• Pay attention to the processing of Mutating Webhook to be idempotent, so as to prevent the result from not meeting expectations;
• When request processing, pay attention to processing all API versions of the resource object;

How to deploy admission webhook

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1/ingresses

Configure webhook specific connection information and trigger rules in webhooks rules can specify the specific behavior of which resources take effect.

Summarize

This article mainly introduces the admission controller in Kubernetes. By default, some have been compiled with kube-apiserver in the form of plug-ins. In addition, we can also write dynamic admission controllers by ourselves to complete related requirements.

Of course, there are already many ready-made tools in the K8s ecosystem that can help us accomplish these things. In many cases, we don't need to develop corresponding services by ourselves. In the follow-up, I will share with you some of the current mainstream tools that can be used for Mutating and Validating access control. Welcome to follow.

---

Welcome to subscribe to my article public account【MoeLove】