从HelloWorld看Knative Serving代码实现

摘要： Knative Serving以Kubernetes和Istio为基础，支持无服务器应用程序和函数的部署并提供服务。我们从部署一个HelloWorld示例入手来分析Knative Serving的代码细节。

概念先知
官方给出的这几个资源的关系图还是比较清晰的：

1.Service: 自动管理工作负载整个生命周期。负责创建route,configuration以及每个service更新的revision。通过Service可以指定路由流量使用最新的revision，还是固定的revision。
2.Route：负责映射网络端点到一个或多个revision。可以通过多种方式管理流量。包括灰度流量和重命名路由。
3.Configuration:负责保持deployment的期望状态，提供了代码和配置之间清晰的分离，并遵循应用开发的12要素。修改一次Configuration产生一个revision。
4.Revision：Revision资源是对工作负载进行的每个修改的代码和配置的时间点快照。Revision是不可变对象，可以长期保留。

看一个简单的示例
我们开始运行官方hello-world示例，看看会发生什么事情：

apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
name: helloworld-go
namespace: default
spec:
runLatest: // RunLatest defines a simple Service. It will automatically configure a route that keeps the latest ready revision from the supplied configuration running.

configuration:
  revisionTemplate:
    spec:
      container:
        image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
        env:
        - name: TARGET
          value: "Go Sample v1"

查看 knative-ingressgateway：

kubectl get svc knative-ingressgateway -n istio-system

查看服务访问：DOMAIN

kubectl get ksvc helloworld-go --output=custom-columns=NAME:.metadata.name,DOMAIN:.status.domain

这里直接使用cluster ip即可访问

curl -H "Host: helloworld-go.default.example.com" http://10.96.199.35

目前看一下服务是部署ok的。那我们看一下k8s里面创建了哪些资源：

我们可以发现通过Serving,在k8s中创建了2个service和1个deployment：

那么究竟Serving中做了哪些处理，接下来我们分析一下Serving源代码

源代码分析
Main
先看一下各个组件的控制器启动代码，这个比较好找，在/cmd/controller/main.go中。
依次启动configuration、revision、route、labeler、service和clusteringress控制器。

...
controllers := []*controller.Impl{

    configuration.NewController(
        opt,
        configurationInformer,
        revisionInformer,
    ),
    revision.NewController(
        opt,
        revisionInformer,
        kpaInformer,
        imageInformer,
        deploymentInformer,
        coreServiceInformer,
        endpointsInformer,
        configMapInformer,
        buildInformerFactory,
    ),
    route.NewController(
        opt,
        routeInformer,
        configurationInformer,
        revisionInformer,
        coreServiceInformer,
        clusterIngressInformer,
    ),
    labeler.NewRouteToConfigurationController(
        opt,
        routeInformer,
        configurationInformer,
        revisionInformer,
    ),
    service.NewController(
        opt,
        serviceInformer,
        configurationInformer,
        routeInformer,
    ),
    clusteringress.NewController(
        opt,
        clusterIngressInformer,
        virtualServiceInformer,
    ),
}

...
Service
首先我们要从Service来看，因为我们一开始的输入就是Service资源。在/pkg/reconciler/v1alpha1/service/service.go。
比较简单，就是根据Service创建Configuration和Route资源

func (c Reconciler) reconcile(ctx context.Context, service v1alpha1.Service) error {

...
configName := resourcenames.Configuration(service)
config, err := c.configurationLister.Configurations(service.Namespace).Get(configName)
if errors.IsNotFound(err) {
    config, err = c.createConfiguration(service)
...
routeName := resourcenames.Route(service)
route, err := c.routeLister.Routes(service.Namespace).Get(routeName)
if errors.IsNotFound(err) {
    route, err = c.createRoute(service)
...

}
Route
/pkg/reconciler/v1alpha1/route/route.go
看一下Route中reconcile做了哪些处理：
1.判断是否有Ready的Revision可进行traffic
2.设置目标流量的Revision（runLatest:使用最新的版本;pinned：固定版本，不过已弃用;release:通过允许在两个修订版之间拆分流量，逐步扩大到新修订版，用于替换pinned。manual：手动模式，目前来看并未实现）
3.创建ClusterIngress:Route不直接依赖于VirtualService[https://istio.io/docs/referen...] ,而是依赖一个中间资源ClusterIngress，它可以针对不同的网络平台进行不同的协调。目前实现是基于istio网络平台。
4.创建k8s service:这个Service主要为Istio路由提供域名访问。

func (c Reconciler) reconcile(ctx context.Context, r v1alpha1.Route) error {

....
// 基于是否有Ready的Revision
traffic, err := c.configureTraffic(ctx, r)
if traffic == nil || err != nil {
    // Traffic targets aren't ready, no need to configure child resources.
    return err
}

logger.Info("Updating targeted revisions.")
// In all cases we will add annotations to the referred targets.  This is so that when they become
// routable we can know (through a listener) and attempt traffic configuration again.
if err := c.reconcileTargetRevisions(ctx, traffic, r); err != nil {
    return err
}

// Update the information that makes us Addressable.
r.Status.Domain = routeDomain(ctx, r)
r.Status.DeprecatedDomainInternal = resourcenames.K8sServiceFullname(r)
r.Status.Address = &duckv1alpha1.Addressable{
    Hostname: resourcenames.K8sServiceFullname(r),
}

// Add the finalizer before creating the ClusterIngress so that we can be sure it gets cleaned up.
if err := c.ensureFinalizer(r); err != nil {
    return err
}

logger.Info("Creating ClusterIngress.")
desired := resources.MakeClusterIngress(r, traffic, ingressClassForRoute(ctx, r))
clusterIngress, err := c.reconcileClusterIngress(ctx, r, desired)
if err != nil {
    return err
}
r.Status.PropagateClusterIngressStatus(clusterIngress.Status)

logger.Info("Creating/Updating placeholder k8s services")
if err := c.reconcilePlaceholderService(ctx, r, clusterIngress); err != nil {
    return err
}

r.Status.ObservedGeneration = r.Generation
logger.Info("Route successfully synced")
return nil

}
看一下helloworld-go生成的Route资源文件：

apiVersion: serving.knative.dev/v1alpha1
kind: Route
metadata:
name: helloworld-go
namespace: default
...
spec:
generation: 1
traffic:

• configurationName: helloworld-go
  percent: 100

status:
...
domain: helloworld-go.default.example.com
domainInternal: helloworld-go.default.svc.cluster.local
traffic:

• percent: 100 # 所有的流量通过这个revision
  revisionName: helloworld-go-00001 # 使用helloworld-go-00001 revision

这里可以看到通过helloworld-go配置, 找到了已经ready的helloworld-go-00001(Revision)。

Configuration
/pkg/reconciler/v1alpha1/configuration/configuration.go
1.获取当前Configuration对应的Revision, 若不存在则创建。
2.为Configuration设置最新的Revision
3.根据Revision是否readiness，设置Configuration的状态LatestReadyRevisionName

func (c Reconciler) reconcile(ctx context.Context, config v1alpha1.Configuration) error {

...
// First, fetch the revision that should exist for the current generation.
lcr, err := c.latestCreatedRevision(config)
if errors.IsNotFound(err) {
    lcr, err = c.createRevision(ctx, config)
...    
revName := lcr.Name
// Second, set this to be the latest revision that we have created.
config.Status.SetLatestCreatedRevisionName(revName)
config.Status.ObservedGeneration = config.Generation

// Last, determine whether we should set LatestReadyRevisionName to our
// LatestCreatedRevision based on its readiness.
rc := lcr.Status.GetCondition(v1alpha1.RevisionConditionReady)
switch {
case rc == nil || rc.Status == corev1.ConditionUnknown:
    logger.Infof("Revision %q of configuration %q is not ready", revName, config.Name)

case rc.Status == corev1.ConditionTrue:
    logger.Infof("Revision %q of configuration %q is ready", revName, config.Name)

    created, ready := config.Status.LatestCreatedRevisionName, config.Status.LatestReadyRevisionName
    if ready == "" {
        // Surface an event for the first revision becoming ready.
        c.Recorder.Event(config, corev1.EventTypeNormal, "ConfigurationReady",
            "Configuration becomes ready")
    }
    // Update the LatestReadyRevisionName and surface an event for the transition.
    config.Status.SetLatestReadyRevisionName(lcr.Name)
    if created != ready {
        c.Recorder.Eventf(config, corev1.EventTypeNormal, "LatestReadyUpdate",
            "LatestReadyRevisionName updated to %q", lcr.Name)
    }

...
}
看一下helloworld-go生成的Configuration资源文件：

apiVersion: serving.knative.dev/v1alpha1
kind: Configuration
metadata:
name: helloworld-go
namespace: default
...
spec:
generation: 1
revisionTemplate:

metadata:
  creationTimestamp: null
spec:
  container:
    env:
    - name: TARGET
      value: Go Sample v1
    image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
    name: ""
    resources: {}
  timeoutSeconds: 300

status:
...
latestCreatedRevisionName: helloworld-go-00001
latestReadyRevisionName: helloworld-go-00001
observedGeneration: 1
我们可以发现LatestReadyRevisionName设置了helloworld-go-00001(Revision)。

Revision
/pkg/reconciler/v1alpha1/revision/revision.go
1.获取build进度
2.设置镜像摘要
3.创建deployment
4.创建k8s service：根据Revision构建服务访问Service
5.创建fluentd configmap
6.创建KPA
感觉这段代码写的很优雅，函数执行过程写的很清晰，值得借鉴。另外我们也可以发现，目前knative只支持deployment的工作负载

func (c Reconciler) reconcile(ctx context.Context, rev v1alpha1.Revision) error {

...
if err := c.reconcileBuild(ctx, rev); err != nil {
    return err
}

bc := rev.Status.GetCondition(v1alpha1.RevisionConditionBuildSucceeded)
if bc == nil || bc.Status == corev1.ConditionTrue {
    // There is no build, or the build completed successfully.

    phases := []struct {
        name string
        f    func(context.Context, *v1alpha1.Revision) error
    }{{
        name: "image digest",
        f:    c.reconcileDigest,
    }, {
        name: "user deployment",
        f:    c.reconcileDeployment,
    }, {
        name: "user k8s service",
        f:    c.reconcileService,
    }, {
        // Ensures our namespace has the configuration for the fluentd sidecar.
        name: "fluentd configmap",
        f:    c.reconcileFluentdConfigMap,
    }, {
        name: "KPA",
        f:    c.reconcileKPA,
    }}

    for _, phase := range phases {
        if err := phase.f(ctx, rev); err != nil {
            logger.Errorf("Failed to reconcile %s: %v", phase.name, zap.Error(err))
            return err
        }
    }
}

...
}
最后我们看一下生成的Revision资源：

apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
name: helloworld-go
namespace: default
...
spec:
generation: 1
runLatest:

configuration:
  revisionTemplate:
    spec:
      container:
        env:
        - name: TARGET
          value: Go Sample v1
        image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
      timeoutSeconds: 300

status:
address:

hostname: helloworld-go.default.svc.cluster.local

...
domain: helloworld-go.default.example.com
domainInternal: helloworld-go.default.svc.cluster.local
latestCreatedRevisionName: helloworld-go-00001
latestReadyRevisionName: helloworld-go-00001
observedGeneration: 1
traffic:

• percent: 100
  revisionName: helloworld-go-00001

这里我们可以看到访问域名helloworld-go.default.svc.cluster.local，以及当前revision的流量配比(100%)
这样我们分析完之后，现在打开Serving这个黑盒：

最后
这里只是基于简单的例子，分析了主要的业务流程处理代码。对于activator(如何唤醒业务容器)，autoscaler（Pod如何自动缩为0）等代码实现有兴趣的同学可以一起交流。

参考
https://github.com/knative/do...