k8s与CICD--将drone部署到kubernetes中，实现agent动态收缩

前言

本文主要讲如何把drone部署到k8s集群当中，本身drone这种基于容器的pipeline方式，和k8s是相当契合的。这样的好处有：

• k8s集群守护drone-server 和drone-agent。
• 可以利用rpc特性，根据agent负载压力来动态调整agent的数量。当然即使不动态调整，我们手动调整一下复制集的数目也是相当简单的。
• 部署到k8s集群以后，可以利用k8s已有的日志系统和监控系统。

其实在接下来的文章系列中，我们会不断侧重于k8s相关。依旧是直接先上yaml文件，先来一个直观的感受。

相关yaml文件

ConfigMap在此处可以理解为drone应用的配置文件。这里有关于server和agent一系列设置。不过在k8s中大家需要注意的是：更新configmap以后，对于挂载该configmap的应用，配置内容并不能立即生效，大约需要10s。

apiVersion: v1
kind: ConfigMap
metadata:
  name: drone-config
  namespace: devops
data:

  #######################
  # Drone Server Config #
  #######################

  # server host name
  server.host: drone.xxx.com
  # start the server in debug mode
  server.debug: "false"
  # open user registration
  server.open: "true"
  # database driver, defaul as sqlite3
  server.database.driver: sqlite3
  # database driver configuration string
  server.database.datasource: drone.sqlite

  # remote parameters (Gogs)
  server.remote.gogs: "true"
  server.remote.gogs.url: "http://gogs.xxx.com"
  server.remote.gogs.private.mode: "true"

  ######################
  # Drone Agent Config #
  ######################
  agent.debug: "false"
  agent.debug.pretty: "false"
  agent.max.procs: "1"
  agent.healthcheck: "true"

Secret文件，主要是存放一些秘钥之类的。不过这里也是有坑的，这个secret用于server和angent通信，设置不对就会构建项目一直处于pending状态。切记k8s中，secret需要base64。

echo -n "yourpassword" | base64
eW91cnBhc3N3b3Jk

apiVersion: v1
kind: Secret
metadata:
  name: drone-secrets
  namespace: devops
data:
  server.secret: eW91cnBhc3N3b3Jk

接下来就是drone-server的Deployment和Service和Ingress。此处为了简单，用了sqlite数据库，真正生产环境建议用mysql或是pgsql。即使用sqlite，也应该挂载到ceph中，保证数据的安全。这里直接hostpath。k8s中，应该做到存储和计算的分离。

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: drone-server
  namespace: devops
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: drone-server
    spec:
      nodeSelector:
        net-type: external
      containers:
      - image: drone/drone:latest
        imagePullPolicy: Always
        name: drone-server
        ports:
        - containerPort: 8000
          protocol: TCP
        - containerPort: 9000
          protocol: TCP
        volumeMounts:
          # Persist our configs in an SQLite DB in here
          - name: drone-server-sqlite-db
            mountPath: /var/lib/drone
        resources:
          requests:
            cpu: 40m
            memory: 32Mi
        env:
        - name: DRONE_HOST
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.host
        - name: DRONE_OPEN
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.open
        - name: DRONE_DATABASE_DRIVER
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.database.driver
        - name: DRONE_DATABASE_DATASOURCE
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.database.datasource
        - name: DRONE_SECRET
          valueFrom:
            secretKeyRef:
              name: drone-secrets
              key: server.secret
        - name: DRONE_GOGS
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.remote.gogs
        - name: DRONE_GOGS_URL
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.remote.gogs.url
        - name: DRONE_GOGS_PRIVATE_MODE
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.remote.gogs.private.mode
        - name: DRONE_DEBUG
          valueFrom:
            configMapKeyRef:
              name: drone-config
              key: server.debug
      volumes:
        - name: drone-server-sqlite-db
          hostPath:
            path: /var/lib/drone

apiVersion: v1
kind: Service
metadata:
  name: drone-service
  namespace: devops
spec:
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 8000
  - name: grpc
    protocol: TCP
    port: 9000
    targetPort: 9000
  selector:
    app: drone-server

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: drone-ingress
  namespace: devops
spec:
  rules:
  - host: drone.xxx.com
    http:
      paths:
      - backend:
          serviceName: drone-service
          servicePort: 80
        path: /

下面就是agent的部署文件了，replicas: 1 该项可以设置agent的数量，扩容起来特别方便。server和agent通过grpc的方式进行通信，主要端口是9000。

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: drone-agent
  namespace: devops
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: drone-agent
    spec:
      nodeSelector:
        net-type: external
      containers:
      - image: drone/agent:latest
        imagePullPolicy: Always
        name: drone-agent
        volumeMounts:
          # Enables Docker in Docker
          - name: docker-socket
            mountPath: /var/run/docker.sock
        resources:
          requests:
            cpu: 100m
            memory: 64Mi
        livenessProbe:
          httpGet:
            path: /healthz
            port: 3000
          initialDelaySeconds: 3
          periodSeconds: 3
        env:
        - name: DRONE_SERVER
          value: drone-service:9000
        # issue: https://github.com/drone/drone/issues/2048
        - name: DOCKER_API_VERSION
          value: "1.24"
        - name: DRONE_SECRET
          valueFrom:
            secretKeyRef:
              name: drone-secrets
              key: server.secret
      volumes:
        - name: docker-socket
          hostPath:
            path: /var/run/docker.sock

所有都部署到devops命名空间下，这个namespace已经建好了。当然如果没有的话，需要提前创建。

效果图

总结

项目github地址，这里有该系列的所有文件。