How to install Rancher HA on RKE2 using domestic resources

About the author
Yuan Zhen, Technical Support Manager of SUSE Rancher, is responsible for the after-sales technical support team for subscription customers and provides technical support services for subscription customers. Since 2016, he has been in touch with container and Kubernetes technologies. He has in-depth research on automated operation and maintenance, Devops, Kubernetes, prometheus and other cloud-native related technologies. He has rich practical experience in SRE operation and maintenance system construction and SRE operation and maintenance system architecture design.

Rancher has always advocated "simple is beautiful", and the installation and use have brought simplicity to the fullest. However, using Rancher and RKE2 through the domestic environment may lead to poor experience due to network problems:

• Both Rancher and RKE2 rely on container images to start, and pulling images through dockerhub in China is sometimes very slow.
• RKE2 uses github resources to install scripts and binary files by default, which is difficult to download in China.

In order to solve the above problems, Rancher has mirrored the resources in China and provided them to the domestic environment. This article will introduce how to use domestic resources to install Rancher HA on RKE2.

Environment Description

• Operating System: SUSE Linux Enterprise Server 15 SP3
• Kubernetes distribution: RKE2 Kubernetes-v1.21.9+rke2r1
• Rancher version: 2.6.3
• Node information

• rancher2-6-node01 Server node (controller, etcd, worker)
• rancher2-6-node02 Server node (controller, etcd, worker)
• rancher2-6-node03 Server node (controller, etcd, worker)

• Helm 3.8.0

Deploy RKE2 Kubernetes cluster

• This chapter will introduce how to deploy a high-availability cluster of RKE2 for use in the deployment environment of Rancher 2.6;
• The Rancher China team regularly synchronizes the RKE2 image on dockerhub to the domestic Alibaba Cloud image repository (registry.cn-hangzhou.aliyuncs.com), so it is recommended to use the Alibaba Cloud image repository address when installing RKE2 in China;
• RKE2 can specify the mirror warehouse address through the parameter system-default-registry.

creates the first server node

Normally, RKE2 uses the /etc/rancher/rke2/config.yaml file as the default configuration file. However, in cluster deployment mode, you need to specify the server address, token, and tls-san parameters. You can create a configuration file first:

mkdir -p /etc/rancher/rke2 ##创建目录
vim /etc/rancher/rke2/config.yaml ##编辑配置文件

token: rke2-create-token ##自定义token
tls-san: 172.16.200.1 ##tls-san参数
system-default-registry: "registry.cn-hangzhou.aliyuncs.com" ##阿里云镜像库地址

More configuration options can be found in the official documentation. After the configuration file is created, use the following command to execute the script to install rke2-server. Since the Rancher 2.6.3 version as of the date of this article does not support running on the kubernetes 1.22 version, v1 is used here. .21.9+rke2r1 kubernetes version:

curl -sfL http://rancher-mirror.rancher.cn/rke2/install.sh |
INSTALL_RKE2_MIRROR=cn INSTALL_RKE2_VERSION=v1.21.9+rke2r1 sh -

Execute the following command to start rke2 and set it to start automatically at boot (the first startup needs to download files such as images, which takes a certain amount of time):

systemctl start rke2-server && systemctl enable rke2-server

The default kubectl tool and kubeconfig file paths are as follows:

kubectl: /var/lib/rancher/rke2/bin/kubectl
kubeconfig: /etc/rancher/rke2/rke2.yaml
crictl: /var/lib/rancher/rke2/bin/crictl
ctr: /var/lib/rancher/rke2/bin/ctr

The above tools can be softly linked to /usr/bin for convenience, for example:

ln -s /var/lib/rancher/rke2/bin/kubectl /usr/bin/kubect

Execute the following command to view the cluster status:

kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get node

NAME STATUS ROLES AGE VERSION
rancher2-6-node01 Ready control-plane,etcd,master 106s v1.21.9+rke2r1

Add another server node

In the previous step, we have successfully started the first server node, now we need to add the remaining two nodes to the cluster to form a highly available cluster.

Before adding the second server node, you need to manually create an RKE2 configuration file:

mkdir -p /etc/rancher/rke2 ##创建目录
vim /etc/rancher/rke2/config.yaml ##编辑配置文件

server: https://172.16.200.1:9345 ##添加首个节点的server地址；
token: <token for server node> ##填写第一个server节点的token，通过在第一个节点查看/var/lib/rancher/rke2/server/token文件获得；
tls-san: 172.16.200.1 ##tls-san参数；
system-default-registry:"registry.cn-hangzhou.aliyuncs.com" ##阿里云镜像库地址；

Execute the following command to execute the script to install rke2-server:

curl -sfL http://rancher-mirror.rancher.cn/rke2/install.sh |
 INSTALL_RKE2_MIRROR=cn INSTALL_RKE2_VERSION=v1.21.9+rke2r1 sh -

Execute the following command to start rke2 and set it to start automatically at boot (the first startup needs to download files such as images, which takes a certain amount of time):

systemctl start rke2-server && systemctl enable rke2-server

Similarly, before adding the third server node, you need to manually create an RKE2 configuration file:

mkdir -p /etc/rancher/rke2 ##创建目录
vim /etc/rancher/rke2/config.yaml ##编辑配置文件

server: https://172.16.200.1:9345 ##添加首个节点的server地址；
token: <token for server node> ##填写第一个server节点的token，通过在第一个节点查看/var/lib/rancher/rke2/server/token文件获得；
tls-san: 172.16.200.1 ##tls-san参数；
system-default-registry: "registry.cn-hangzhou.aliyuncs.com" ##阿里云镜像库地址；

Execute the following command to execute the script to install rke2-server:

curl -sfL http://rancher-mirror.rancher.cn/rke2/install.sh |
INSTALL_RKE2_MIRROR=cn INSTALL_RKE2_VERSION=v1.21.9+rke2r1 sh - 

Execute the following command to start rke2 and set it to start automatically at boot (the first startup needs to download files such as images, which takes a certain amount of time):

systemctl start rke2-server && systemctl enable rke2-server

Verify cluster

Execute the following command to view the cluster status:

kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get node

NAME                              STATUS   ROLES                                             AGE VERSION
rancher2-6-node01   Ready      control-plane,etcd,master     18m v1.21.9+rke2r1
rancher2-6-node02   Ready      control-plane,etcd,master     7m29s v1.21.9+rke2r1
rancher2-6-node03   Ready      control-plane,etcd,master     3m31s v1.21.9+rke2r1

So far, we have successfully created and run the kubernetes high-availability cluster created by RKE2, but it should be noted that the official best practice of RKE2 indicates that the high-availability cluster should be accessed by a unified LB entry, which is the real high-availability. This article mainly introduces the deployment of Rancher 2.6 on the RKE2 cluster. For more details on the use and deployment of RKE2, please refer to RKE2 official document ( https://docs.rancher.cn/docs/rke2/_index/ ) or official public account article .

RKE2 uses containerd as Runtime by default. If you want to query the containers running on the host, you can use the following command:

crictl --config /var/lib/rancher/rke2/agent/etc/crictl.yaml ps

deploy Rancher 2.6.3

• This chapter will introduce how to deploy Rancher 2.6.3 version in RKE2 cluster;
• The Rancher China team regularly synchronizes the Rancher image on dockerhub to the domestic Alibaba Cloud image repository (registry.cn-hangzhou.aliyuncs.com), so it is recommended to use the Alibaba Cloud image repository address when installing Rancher in China;
• You can use the helm chart options rancherImage and systemDefaultRegistry to set up and use the Alibaba Cloud image repository.

Install Rancher 2.6.3

Add Rancher helm repo source:

helm repo add rancher-latest http://rancher-mirror.oss-cn-beijing.aliyuncs.com/server-charts/latest

"rancher-latest" has been added to your repositories

The domestic source address is used here, and the foreign address is https://releases.rancher.com/server-charts/< ;CHART_REPO> For more domestic acceleration information, please check the official public account [ How to elegantly domestically Using Rancher ]

Create a Namespace for Rancher:

kubeconfig=/etc/rancher/rke2/rke2.yaml

kubectl --kubeconfig=$kubeconfig create namespace cattle-system

Create an Ingress certificate:

kubeconfig=/etc/rancher/rke2/rke2.yaml

kubectl --kubeconfig=$kubeconfig \
          -n cattle-system create secret \
          tls tls-rancher-ingress \
          --cert=./tls.pem \
          --key=./tls.key

helm install Rancher Server:

helm --kubeconfig=$kubeconfig install rancher rancher-latest/rancher \
          --namespace cattle-system \
          --set hostname=rancher26.itlsp.com \
          --set rancherImage=registry.cn-hangzhou.aliyuncs.com/rancher/rancher \
          --set ingress.tls.source=secret \
          --set systemDefaultRegistry=registry.cn-hangzhou.aliyuncs.com \
          --set rancherImageTag=v2.6.3

## hostname为上一步创建证书中的hostname；
## systemDefaultRegistry和rancherImage中的镜像库可指定；
## 以上命令为权威证书安装命令，如果是自签名证书参考以下命令

## 创建自签名Ingress证书
kubectl --kubeconfig=$kubeconfig \
           -n cattle-system create \
           secret tls tls-rancher-ingress \
           --cert=./tls.crt \
           --key=./tls.key
## 创建自签名证书CA
kubectl --kubeconfig=$kubeconfig \
           -n cattle-system \
           create secret generic tls-ca \
           --from-file=cacerts.pem

##helm安装Rancher server
helm --kubeconfig=$kubeconfig install rancher rancher-latest/rancher \
         --namespace cattle-system \
         --set hostname=<修改为自己的域名> \
         --set rancherImage=<镜像库地址>/cnrancher/rancher \
         --set privateCA=true \
         --set ingress.tls.source=secret \
         --set systemDefaultRegistry=<镜像库地址> \
         --set rancherImageTag=v2.6.3

Execute the following command to view the Rancher Server Pod status:

kubectl --kubeconfig=$kubeconfig -n cattle-system get pod | grep rancher

rancher-57f57f775-4bjdj 1/1 Running 1 5m58s
rancher-57f57f775-fdh6c 1/1 Running 0 5m58s
rancher-57f57f775-fnsxk 1/1 Running 0 5m58s

congratulations! At this step, you should be able to access the new Rancher 2.6.3 version through the browser. So far, the new Rancher 2.6.3 version has been running on the more secure, more stable and more efficient RKE2 platform.

Summary

• RKE2 is SUSE Rancher's next-generation Kubernetes distribution, known for higher security (it has another name: RKE Government); FIPS 140-2 enabled, and fully scanned for CIS versions 1.5 and 1.6 ;
• In addition, RKE2 does not use docker as Runtime by default, but uses a lower-level containerd; in response to the question of where kubernetes will go after abandoning docker, this approach has greatly stabilized the confidence of domestic users;
• RKE2 runs core components such as kube-apiserver, etcd, kube-controller-manager as static Pods, and is managed by the binary running kubelet, which enhances the fault self-healing ability and overall stability, and also reduces the difficulty of deployment;
• The installation of Rancher and RKE2 in this article both use the Alibaba Cloud image warehouse registry.cn-hangzhou.aliyuncs.com, and the script to install RKE2 also uses domestic resources, which can greatly improve the experience of domestic users.