Microservices have everything from code to k8s deployment (14. Deployment environment construction)

1 Overview

After the project is developed, we need to deploy it. Next, we will build a deployment environment based on gitlab + jenkins + harbor + k8s

• gitlab: put code, you can do ci
• jenkins: do cd release project
• harbor: mirror repository
• k8s: run service

We only run services inside k8s. As for middleware (mysql, redis, es, etc.), it will be deployed outside k8s. If you use cloud services online, you can use cloud services directly. If you build it yourself, it is best to run it in k8s outside. Since I am demonstrating locally, I will use the middleware of the previous development environment for the middleware here. Don't worry about this. It mainly demonstrates how to deploy the go-zero service to k8s.

k8s deployment will not be introduced here. If there is no k8s environment, you can use rancher or kubeadm to build it yourself. If not, buy a cloud service k8s that pays on time.

So we need to configure as follows:

server nickname	effect	IP
deploy-server.com	Deploy gitlab, jenkins, harbor (pre-installed docker, docker-compose)	192.168.1.180
srv-data.com	Deploy mysql, redis, es, etc., simulate an independent environment, and connect to this server internally in k8s	192.168.1.181
nginx-gateway.com	Gateway, independent from outside the k8s cluster	192.168.1.182
k8s cluster	K8s cluster	192.168.1.183

2. gitlab

2.1 Deploy gitlab

Create folder

 $ mkdir gitlab && cd gitlab
$ vim docker-compose.yml

docker-compose.yml

 version: '3'

services:
    gitlab:
      image: 'twang2218/gitlab-ce-zh'
      container_name: 'gitlab'
      restart: always
      hostname: '192.168.1.180' #部署机器的ip,非容器ip(因为是本地不是线上所以用ip，线上的话可以用域名)
      environment:
        TZ: 'Asia/Shanghai'
        GITLAB_OMNIBUS_CONFIG: |
         external_url 'http://192.168.1.180'  #使用这个地址访问gitlab web ui(因为是本地不是线上所以用ip，线上的话可以用域名)
         gitlab_rails['gitlab_shell_ssh_port'] = 2222 #ssh clone代码地址
         unicorn['port'] = 8888 #gitlab一个内部端口
      ports:
        - '80:80'        #web 80 端口
       #- '443:443'      #web 443 端口,本次未使用就不开放了
        - '2222:22'      #ssh 检出代码 端口
      volumes:
        - ./etc:/etc/gitlab             #Gitlab配置文件目录
        - ./data:/var/opt/gitlab  #Gitlab数据目录
        - ./logs:/var/log/gitlab   #Gitlab日志目录

implement

 $  docker-compose up -d

This execution time may be a little long, you may as well go to make a cup of coffee and take a break~~

2.2 Visit gitlab

Visit http://192.168.1.103 (ie http://"ip/domain name in docker-compose")

Setting a new password for the first time: 12345678

The account is root by default

2.3 Create a project

2.4 Configure ssh public key

Click the arrow down the avatar position, "Settings"

Configure your own public key and click "Add key" (the public key will not be generated by your own search, so I won't go into details here)

2.5 Upload project

Click on the project again, go back to the project you just created, upload the go-zero-looklook project to this repository ssh://git@192.168.1.180:2222/root/go-zero-looklook.git, here we go The gitlab build is over.

[Note] This time I will not do the gitlab-runner demo, and I will add it later if I have time.

3. Harbor

3.1 Deploy harbor

Download harbor https://github.com/goharbor/harbor/releases/download/v2.2.0/harbor-offline-installer-v2.2.0.tgz , it will be faster to download offline offline installation

After downloading and unzipping, enter the harbor folder

 $ cd harbor && cp harbor.yml.tmpl harbor.yml

We open harbor.yml and modify it as follows

 hostname: 192.168.1.180                                                                       #修改为本机ip，不能使用localhost、127.0.0.1

http:
  port: 8077                                                                                              #改一下http端口8077

#https:                                                                                                      #暂时将https注释掉，我们先不通过https只通过http
#  port: 443
#  certificate: /your/certificate/path
#  private_key: /your/private/key/path

data_volume: /root/harbor/data          #修改一下数据目录位置

log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /root/harbor/log     #修改一下日志目录位置

Just run "sudo ./install.sh" and wait for a while.

3.2 Visit harbor

Browser input http://192.168.1.180:8077

Account: admin

Password: Harbor12345 (recorded in harbor.yml, default is Harbor12345)

Landed successfully

3.3 Create a new private project

View the push command

 $ docker push 192.168.1.180:8077/go-zero-looklook/REPOSITORY[:TAG]

3.4 Support http

By default, https is used for pulling and pushing to the mirror warehouse. Since we do not have https here, we need to use http, so we need to execute the following on the deploy-server.com server

 $ echo '{"insecure-registries":["192.168.1.180:8077"] }' >> /etc/docker/daemon.json

At this point, our harbor construction is complete.

4. Jenkins

4.1 Deploying jenkins

Create folder

 $ mkdir jenkins && cd jenkins
$ vim docker-compose.yml

docker-compose.yml

 version: '3'
services:
  jenkins:
    image: 'jenkins/jenkins:lts'
    container_name: jenkins
    restart: always
    environment:
      - TZ=Asia/Shanghai
    user: root
    ports:
      - '8989:8080'
      - '50000:50000'
    volumes:
      - './jenkins_home:/var/jenkins_home'
      - '/var/run/docker.sock:/var/run/docker.sock'
      - '/usr/bin/docker:/usr/bin/docker'
      - '/root/port.sh:/root/port.sh'

[Note] The content of /root/port.sh is as follows

 #!/bin/sh

case $1 in
"identity-api") echo 1001
;;
"identity-rpc") echo 1101
;;
"usercenter-api") echo 1002
;;
"usercenter-rpc") echo 1102
;;
"message-mq") echo 1207
;;
"mqueue-rpc") echo 1106
;;
"order-api") echo 1004
;;
"order-mq") echo 1204
;;
"order-rpc") echo 1104
;;
"payment-api") echo 1005
;;
"payment-rpc") echo 1105
;;
"travel-api") echo 1003
;;
"travel-rpc") echo 1103
esac

implement

 $ docker-compose up -d

This time is not too slow, you can go have another cup of coffee

4.2 Mounting Tool

1) Copy the goctl into the jenkins container

 $ docker cp $GOPATH/bin/goctl jenkins:/usr/local/bin
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ goctl -v     #验证成功
goctl version 1.3.0-20220201 linux/amd64

2) Copy the kubectl file into the jenkins container

 $ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
$ sudo chmod a+x kubectl
$ docker cp kubectl jenkins:/usr/local/bin
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3" .....

3) Copy the k8s configuration .kube/config to the jenkins container

 $ docker cp ~/.kube jenkins:/root/ #前提是家目录下的.kube文件夹中存在k8s的config配置
$ docker exec -it jenkins /bin/sh #进入jenkins 容器
$ kubectl ge ns
default              Active   43m
kube-node-lease      Active   43m
kube-public          Active   43m
kube-system          Active   43m
local-path-storage   Active   43m

[Note] The above four steps can also be directly entered into the mirror image. I am just demonstrating here and leave it to you to deal with it yourself.

4.3 Accessing jenkins

http://192.168.1.180:8989

Don't panic when the above picture appears on the first visit, let you wait for a while, it is preparing, and it will automatically jump to the login page when it is ready.

The following interface appears, indicating that it is ready, because our directory is mounted, we check the local jenkins_home/secrets/initialAdminPassword password, and enter the next step.

Select "Install Recommended Plugins"

Then wait for the plugin installation to complete

4.4 Create User

Account: root/root

4.5 Deployment completed

At this point, the jenkins deployment is complete

4.6 Adding Credentials

Click on the left menu "Manage Jenkins"

Click on "Manage Credentials"

Click the triangle next to "Global" and then click "Add Credentials"

Enter the "Add Credentials" page, select "SSH Username with private key" for the type, and use the private key method, Username is a logo of gitlab, and add a pipeline later. You know that this logo is self-defined on behalf of the credentials of gitlab. Private Key that is, the private key configured in gitlab (the private key corresponding to the public key we configured in gitlab before, here is our own private key), our certificate is used by jenkins to go to gitlab When the code is free of Mila

OK.

4.7 Add harbor warehouse configuration

Enter the home page, click on the left menu Manage Jenkins -> Configure System

Scroll down to the 全局属性 entry and add the relevant information about the docker private warehouse, as shown in the figure docker用户名 , docker用户密码 , docker私有仓库地址

Click "Save"

4.8 Configuring git

Enter Manage Jenkins -> Global Tool Configureation , find the Git entry, and fill in the path where the git executable file of the machine where jenkins is located. Need to be managed (as shown below)

Configure the Git Parameter plugin required by pipline

Click "System Configuration" -> "Plugin Management"

Then click "Optional Plugins", enter "Git Parameter" in the search, as shown below

After the installation is complete, it can be restarted, and the jenkins construction is completed.

5. k8s

The deployment of k8s will not be introduced here. Use kubeadm, rancher, and kind to install it yourself, or buy a cloud container service by volume. In short, it is good to have a k8s cluster.

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.