Operator3-设计一个operator二-owns的使用

头像
对你无可奈何
    29
    发布于
    北京
     

    背景：

    上一节（Operator3-设计一个operator）做完发现一个问题 我创建了jan 应用jan-sample，子资源包括deployment，service.ingress,pod(其中pod是deployment管理的)
    image.png
    手动删除Pod.由于Deployment rc控制器。Pod资源可以自动重建。但是我删除deployment能不能自动重建呢？正常的deployment service ingress子资源的生命周期，我应该是靠jan应用去维系的，试一试：

    [zhangpeng@zhangpeng jan]$ kubectl delete deployment jan-sample
deployment.apps "jan-sample" deleted
[zhangpeng@zhangpeng jan]$ kubectl get deployment
No resources found in default namespace.

    image.png
    到这里才发现没有考虑周全......删除deployment资源并不能重建，正常创建应用应该要考虑一下jan资源下面资源的重建.搜了一下别人写的operator貌似的可以加一下Owns，尝试一下！

    Deployment Ingress Service关于Owns的使用

    Deployment

    func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&janv1.Jan{}).
        Owns(&appsv1.Deployment{}).
        Complete(r)
}

    Deployment delete尝试

    make run develop-operator项目，并尝试delete deployment jan-sample查看是否重建：

    [zhangpeng@zhangpeng develop-operator]$ kubectl get Jan
[zhangpeng@zhangpeng develop-operator]$ kubectl get all

    image.png

    [zhangpeng@zhangpeng develop-operator]$ kubectl delete deployment jan-sample
[zhangpeng@zhangpeng develop-operator]$ kubectl get deployment

    image.png
    恩发现deployment应用可以自动重建了！

    image.png

    Ingress and Service资源

    简单添加一下Owns？

    but其他资源是否可以呢？是不是也偷懒一下添加Owns？

    func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&janv1.Jan{}).
        Owns(&appsv1.Deployment{}).
        Owns(&corev1.Service{}).
        Owns(&v1.Ingress{}).
        Complete(r)
}

    尝试了一下不生效的，但是这种方式思路是对的至于为什么不生效呢？
    deploy声明了&appv1.Deployment,但是service,ingress是没有创建变量声明的！
    image.png

    拆分改造代码

    继续改造Jan operator使其支持service ingress子资源的误删除创建：
    image.png
    把这边拆分一下？
    image.png

    image.png
    image.png
    image.png
    jan_controller.go

    /*
Copyright 2022 zhang peng.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package jan

import (
    "context"
    "encoding/json"
    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    v1 "k8s.io/api/networking/v1"
    "k8s.io/apimachinery/pkg/api/errors"
    "k8s.io/apimachinery/pkg/runtime"
    utilruntime "k8s.io/apimachinery/pkg/util/runtime"
    "reflect"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/log"
    "sigs.k8s.io/controller-runtime/pkg/reconcile"

    janv1 "develop-operator/apis/jan/v1"
)

// JanReconciler reconciles a Jan object
type JanReconciler struct {
    client.Client
    Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=apps,resources=deployments/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=networking,resources=ingresses,verbs=get;list;watch;create;update;patch;delete

// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Jan object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile
func (r *JanReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    defer utilruntime.HandleCrash()
    _ = log.FromContext(ctx)
    instance := &janv1.Jan{}
    err := r.Client.Get(context.TODO(), req.NamespacedName, instance)
    if err != nil {
        if errors.IsNotFound(err) {
            // Request object not found, could have been deleted after reconcile request.
            // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
            // Return and don't requeue
            return reconcile.Result{}, nil
        }
        // Error reading the object - requeue the request.
        return reconcile.Result{}, err
    }
    if instance.DeletionTimestamp != nil {
        return reconcile.Result{}, err
    }

    // 如果不存在，则创建关联资源
    // 如果存在，判断是否需要更新
    //   如果需要更新，则直接更新
    //   如果不需要更新，则正常返回

    deploy := &appsv1.Deployment{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, deploy); err != nil && errors.IsNotFound(err) {
        // 创建关联资源
        // 1. 创建 Deploy
        deploy := NewJan(instance)
        if err := r.Client.Create(context.TODO(), deploy); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(instance.Spec)

        if instance.Annotations != nil {
            instance.Annotations["spec"] = string(data)
        } else {
            instance.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), instance); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    Service := &corev1.Service{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, Service); err != nil && errors.IsNotFound(err) {

        // 2. 创建 Service
        service := NewService(instance)
        if err := r.Client.Create(context.TODO(), service); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(service.Spec)

        if service.Annotations != nil {
            service.Annotations["spec"] = string(data)
        } else {
            service.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), service); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    Ingress := &v1.Ingress{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, Ingress); err != nil && errors.IsNotFound(err) {

        // 2. 创建 Service
        ingress := NewIngress(instance)
        if err := r.Client.Create(context.TODO(), ingress); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(ingress.Spec)

        if ingress.Annotations != nil {
            ingress.Annotations["spec"] = string(data)
        } else {
            ingress.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), ingress); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    oldspec := janv1.JanSpec{}
    if err := json.Unmarshal([]byte(instance.Annotations["spec"]), &oldspec); err != nil {
        return reconcile.Result{}, err
    }

    if !reflect.DeepEqual(instance.Spec, oldspec) {
        data, _ := json.Marshal(instance.Spec)

        if instance.Annotations != nil {
            instance.Annotations["spec"] = string(data)
        } else {
            instance.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), instance); err != nil {
            return reconcile.Result{}, nil
        }
        // 更新关联资源
        newDeploy := NewJan(instance)
        oldDeploy := &appsv1.Deployment{}
        if err := r.Client.Get(context.TODO(), req.NamespacedName, oldDeploy); err != nil {
            return reconcile.Result{}, err
        }
        oldDeploy.Spec = newDeploy.Spec
        if err := r.Client.Update(context.TODO(), oldDeploy); err != nil {
            return reconcile.Result{}, err
        }

        newService := NewService(instance)
        oldService := &corev1.Service{}
        if err := r.Client.Get(context.TODO(), req.NamespacedName, oldService); err != nil {
            return reconcile.Result{}, err
        }
        oldService.Spec = newService.Spec
        if err := r.Client.Update(context.TODO(), oldService); err != nil {
            return reconcile.Result{}, err
        }
        return reconcile.Result{}, nil
    }
    newStatus := janv1.JanStatus{
        Replicas:      *instance.Spec.Replicas,
        ReadyReplicas: instance.Status.Replicas,
    }

    if newStatus.Replicas == newStatus.ReadyReplicas {
        newStatus.Phase = janv1.Running
    } else {
        newStatus.Phase = janv1.NotReady
    }
    if !reflect.DeepEqual(instance.Status, newStatus) {
        instance.Status = newStatus
        log.FromContext(ctx).Info("update game status", "name", instance.Name)
        err = r.Client.Status().Update(ctx, instance)
        if err != nil {
            return reconcile.Result{}, err
        }
    }
    return reconcile.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&janv1.Jan{}).
        Owns(&appsv1.Deployment{}).
        Owns(&corev1.Service{}).
        Owns(&v1.Ingress{}).
        Complete(r)
}

    make run尝试一下：
    image.png
    注意：make run之前默认删除jan应用！

    [zhangpeng@zhangpeng develop-operator]$ kubectl delete svc jan-sample
[zhangpeng@zhangpeng develop-operator]$ kubectl get svc

    en service的自动恢复生效了
    image.png
    然后试一试ingress

    [zhangpeng@zhangpeng develop-operator]$ kubectl get ingress
[zhangpeng@zhangpeng develop-operator]$ kubectl delete ingress jan-sample
[zhangpeng@zhangpeng develop-operator]$ kubectl get ingress

    image.png
    继续发现问题:
    en,我修改一下jan_v1_jan.yaml中host ww1.zhangpeng.com修改为ww11.zhangpeng.com,but ingress的相关信息没有及时更新啊？
    image.png
    继续模仿一下上面的service oldservice newservice新增 newIngress oldIngress ：
    image.png
    重新make run
    ingress相关信息得到了修改
    image.png

    最终代码：

    jan_controller.go

    /*
Copyright 2022 zhang peng.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package jan

import (
    "context"
    "encoding/json"
    appsv1 "k8s.io/api/apps/v1"
    corev1 "k8s.io/api/core/v1"
    v1 "k8s.io/api/networking/v1"
    "k8s.io/apimachinery/pkg/api/errors"
    "k8s.io/apimachinery/pkg/runtime"
    utilruntime "k8s.io/apimachinery/pkg/util/runtime"
    "reflect"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/client"
    "sigs.k8s.io/controller-runtime/pkg/log"
    "sigs.k8s.io/controller-runtime/pkg/reconcile"

    janv1 "develop-operator/apis/jan/v1"
)

// JanReconciler reconciles a Jan object
type JanReconciler struct {
    client.Client
    Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=apps,resources=deployments/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=networking,resources=ingresses,verbs=get;list;watch;create;update;patch;delete

// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Jan object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile
func (r *JanReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    defer utilruntime.HandleCrash()
    _ = log.FromContext(ctx)
    instance := &janv1.Jan{}
    err := r.Client.Get(context.TODO(), req.NamespacedName, instance)
    if err != nil {
        if errors.IsNotFound(err) {
            // Request object not found, could have been deleted after reconcile request.
            // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
            // Return and don't requeue
            return reconcile.Result{}, nil
        }
        // Error reading the object - requeue the request.
        return reconcile.Result{}, err
    }
    if instance.DeletionTimestamp != nil {
        return reconcile.Result{}, err
    }

    // 如果不存在，则创建关联资源
    // 如果存在，判断是否需要更新
    //   如果需要更新，则直接更新
    //   如果不需要更新，则正常返回

    deploy := &appsv1.Deployment{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, deploy); err != nil && errors.IsNotFound(err) {
        // 创建关联资源
        // 1. 创建 Deploy
        deploy := NewJan(instance)
        if err := r.Client.Create(context.TODO(), deploy); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(instance.Spec)

        if instance.Annotations != nil {
            instance.Annotations["spec"] = string(data)
        } else {
            instance.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), instance); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    Service := &corev1.Service{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, Service); err != nil && errors.IsNotFound(err) {

        // 2. 创建 Service
        service := NewService(instance)
        if err := r.Client.Create(context.TODO(), service); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(service.Spec)

        if service.Annotations != nil {
            service.Annotations["spec"] = string(data)
        } else {
            service.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), service); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    Ingress := &v1.Ingress{}

    if err := r.Client.Get(context.TODO(), req.NamespacedName, Ingress); err != nil && errors.IsNotFound(err) {

        // 2. 创建 Ingress
        ingress := NewIngress(instance)
        if err := r.Client.Create(context.TODO(), ingress); err != nil {
            return reconcile.Result{}, err
        }
        // 4. 关联 Annotations
        data, _ := json.Marshal(ingress.Spec)

        if ingress.Annotations != nil {
            ingress.Annotations["spec"] = string(data)
        } else {
            ingress.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), ingress); err != nil {
            return reconcile.Result{}, nil
        }
        return reconcile.Result{}, nil
    }
    oldspec := janv1.JanSpec{}
    if err := json.Unmarshal([]byte(instance.Annotations["spec"]), &oldspec); err != nil {
        return reconcile.Result{}, err
    }

    if !reflect.DeepEqual(instance.Spec, oldspec) {
        data, _ := json.Marshal(instance.Spec)

        if instance.Annotations != nil {
            instance.Annotations["spec"] = string(data)
        } else {
            instance.Annotations = map[string]string{"spec": string(data)}
        }
        if err := r.Client.Update(context.TODO(), instance); err != nil {
            return reconcile.Result{}, nil
        }
        // 更新关联资源
        newDeploy := NewJan(instance)
        oldDeploy := &appsv1.Deployment{}
        if err := r.Client.Get(context.TODO(), req.NamespacedName, oldDeploy); err != nil {
            return reconcile.Result{}, err
        }
        oldDeploy.Spec = newDeploy.Spec
        if err := r.Client.Update(context.TODO(), oldDeploy); err != nil {
            return reconcile.Result{}, err
        }

        newService := NewService(instance)
        oldService := &corev1.Service{}
        if err := r.Client.Get(context.TODO(), req.NamespacedName, oldService); err != nil {
            return reconcile.Result{}, err
        }
        oldService.Spec = newService.Spec
        if err := r.Client.Update(context.TODO(), oldService); err != nil {
            return reconcile.Result{}, err
        }
        newIngress := NewIngress(instance)
        oldIngress := &v1.Ingress{}
        if err := r.Client.Get(context.TODO(), req.NamespacedName, oldIngress); err != nil {
            return reconcile.Result{}, err
        }
        oldIngress.Spec = newIngress.Spec
        if err := r.Client.Update(context.TODO(), oldIngress); err != nil {
            return reconcile.Result{}, err
        }
        return reconcile.Result{}, nil
    }
    newStatus := janv1.JanStatus{
        Replicas:      *instance.Spec.Replicas,
        ReadyReplicas: instance.Status.Replicas,
    }

    if newStatus.Replicas == newStatus.ReadyReplicas {
        newStatus.Phase = janv1.Running
    } else {
        newStatus.Phase = janv1.NotReady
    }
    if !reflect.DeepEqual(instance.Status, newStatus) {
        instance.Status = newStatus
        log.FromContext(ctx).Info("update game status", "name", instance.Name)
        err = r.Client.Status().Update(ctx, instance)
        if err != nil {
            return reconcile.Result{}, err
        }
    }
    return reconcile.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&janv1.Jan{}).
        Owns(&appsv1.Deployment{}).
        Owns(&corev1.Service{}).
        Owns(&v1.Ingress{}).
        Complete(r)
}

    总结

    1. owns的一般使用
    2. 将 deployment service ingress或者其他资源作为operator应用的子资源，进行生命周期管理
    3. 下一步想处理一下 make run 控制台的输出，输出一些有用的信息
    kuberneteskubebuilder云原生
    本作品系原创，采用《署名-非商业性使用-禁止演绎 4.0 国际》许可协议
    Kubernetes实践
    avatar
    29 声望
    9 粉丝
    0 条评论
    得票最新
    推荐阅读
    Ubuntu20.4 docker运行stable diffusion webui
    环境前提系统环境ubuntu20.04 {代码...} nvida cuda显卡驱动默认已经安装成功 {代码...} 安装配置dockerupgrade系统更新系统依赖： {代码...} 卸载之前的Docker环境确认是否之前安装过docker并卸载： {代码...} ...

    对你无可奈何阅读 84

    Helm3-安装RabbitMQ
    最近在使用k8s搭建微服务时，发现需要手动修改yaml文件里面的pod name、pod image、svc name、ingress tls等等，非常麻烦，但是有了helm之后情况就不一样了，helm是k8s的包管理器，类似ubuntu的apt-get，centos的...

    Awbeci阅读 9.6k

    Apache APISIX 结合 Authing 实现集中式身份认证管理
    Apache APISIX 是一个动态、实时、高性能的 API 网关，提供负载均衡、动态上游、灰度发布、服务熔断、身份认证、可观测性等丰富的流量管理功能。Apache APISIX 不仅支持插件动态变更和热插拔，而且拥有众多实用的...

    API7_技术团队赞 1阅读 2.5k

    Kubernetes Gateway API 深入解读和落地指南
    Kubernetes Gateway API 是 Kubernetes 1.18 版本引入的一种新的 API 规范，是 Kubernetes 官方正在开发的新的 API，Ingress 是 Kubernetes 已有的 API。Gateway API 会成为 Ingress 的下一代替代方案。Gateway A...

    Rainbond赞 2阅读 444

    Jvm调优与微服务资源分配
    在没有接触微服务之前，我们的java程序一般都部署在WebLogic、Tomcat这类应用服务器上，这些应用服务器本身也是基于Jvm虚拟机的。一般我们统一对应用服务器做Jvm参数调优（分配多大内存，线程池限制等），而不用...

    KerryWu阅读 6.1k

    Apache APISIX 助力便利充电创领者小电，实现云原生方案
    原文链接业务背景小电作为国内共享充电宝服务平台，目前还处于初创阶段。从运维体系、测试环境等方面来讲，当下产品的业务主要面临了以下几个问题：VM 传统模式部署，利用率低且不易扩展开发测试资源抢占多套独立...

    API7_技术团队赞 1阅读 1.6k

    K8S-使用Helm安装RabbitMQ和Redis的总结
    记得去年2021上半年的时候自学了k8s并且使用helm安装了rabbitmq和redis，可以在开发、测试和生产环境上用起来，但是下半年之后就没有用，再拾起来的时候发现好多知识点都忘了，这篇文章就是总结使用helm安装rabbm...

    Awbeci赞 1阅读 1.9k

    avatar
    29 声望
    9 粉丝
    宣传栏