A key step for single-node Rancher offline installation

About the Author

Wang Hailong, SUSE/Rancher China Community Technical Manager, responsible for the maintenance and operation of the Rancher China Technical Community. He has 7 years of experience in the field of cloud computing, and has experienced technological changes from OpenStack to Kubernetes. He has rich operation and maintenance and practical experience no matter the underlying operating system Linux, virtualized KVM or Docker container technology.

Preface

Rancher starting from v2.5.x, single-node Rancher Server has built-in K3s as a local cluster. In addition to supporting the operation of Rancher Server, this local cluster will also run fleet, rancher-webhook, gitjob, coredns and other components. The following figure shows the components that are started by default in the Rancher v2.5.8 built-in K3s cluster and the required images:

Among them, docker.io/rancher/coredns-coredns and docker.io/rancher/pause are built-in images of Rancher Server. Other images need to be downloaded online from dockerhub after Rancher Server starts.

If your environment is offline, and Rancher Server is started by docker run, some logs of mirror download failure will be reported:

E0511 08:39:56.060906      28 pod_workers.go:191] Error syncing pod d0f83155-f023-4ff6-9164-11b6b63ef4d6 ("helm-operation-t8mtl_cattle-
system(d0f83155-f023-4ff6-9164-11b6b63ef4d6)"), skipping: [failed to "StartContainer" for "helm" with ImagePullBackOff: "Back-
off pulling image \"rancher/shell:v0.1.6\"", failed to "StartContainer" for "proxy" with ImagePullBackOff: "Back-off pulling image \"rancher/shell:v0.1.6\""]

Official website documentation single node offline installation of Rancher Server ( http://docs.rancher.cn/docs/rancher2.5/installation/other-installation-methods/air-gap/install-rancher/_index/# single node installation ) The chapter points out that you can use the CATTLE_SYSTEM_DEFAULT_REGISTRY parameter to specify the mirror required to obtain the Rancher Server from the private mirror repository, but if your private mirror repository is TLS enabled or certified, CATTLE_SYSTEM_DEFAULT_REGISTRY cannot meet your needs.

For the above scenarios, we can /etc/rancher/k3s/registries.yaml built-in k3s cluster to the host, and then modify the registries.yaml to achieve more complex scenarios such as enabling TLS or authentication. For k3s configuration private warehouse, please refer to k3s official website ( http://docs.rancher.cn/docs/k3s/installation/private-registry/_index/ ).

Single node Rancher Server configuration private mirror warehouse

The following chapters will guide you how to install a single-node Rancher Server in an offline environment through a private mirror warehouse. According to the different types of mirror warehouses, it is divided into the following scenarios:

• The private warehouse is a certificate (https) issued by an SSL certificate authority, with certification
• The private warehouse is a self-signed certificate (https), with certification
• The private warehouse does not use SSL (using http) and has authentication

private warehouse is a certificate (https) issued by an SSL certificate authority, with certification

The private warehouse has uploaded the mirrors required by Rancher Server in advance. For how to synchronize the mirrors to the private mirror warehouse, please refer to the official document ( http://docs.rancher.cn/docs/rancher2.5/installation/other-installation-methods /air-gap/populate-private-registry/_index ).

1. Configure K3s registries.yaml

root@ip-172-31-14-159:~# cat /opt/registries.yaml
mirrors:
  # 私有仓库域名
  harbor.kingsd.top:
    endpoint:
      - "https://harbor.kingsd.top"
configs:
  "harbor.kingsd.top":
    auth:
      username: admin  # 这是私有镜像仓库的用户名
      password: Password  # 这是私有镜像仓库的密码

2. Start the single node Rancher Server

root@ip-172-31-14-159:~# docker run -itd --privileged \
>     --restart=unless-stopped \
>     -p 80:80 -p 443:443 \
>     -e CATTLE_SYSTEM_DEFAULT_REGISTRY=harbor.kingsd.top \ # 设置私有仓库域名
>     -v /opt/registries.yaml:/etc/rancher/k3s/registries.yaml \ # 将宿主机`registries.yaml`映射到容器内
>     rancher/rancher:v2.5.8

3. Verification

After waiting for Rancher Server to start successfully, we can exec into the container to confirm whether the required components are started:

root@3fd636aa513e:/var/lib/rancher# kubectl get pods -A
NAMESPACE                 NAME                                READY   STATUS      RESTARTS   AGE
cattle-system             helm-operation-5w49g                0/2     Completed   0          3m21s
cattle-system             helm-operation-gclkp                0/2     Completed   0          3m3s
cattle-system             helm-operation-jt948                0/2     Completed   0          3m13s
cattle-system             helm-operation-l76g6                0/2     Completed   0          3m41s
cattle-system             helm-operation-zmm6f                0/2     Completed   0          3m28s
cattle-system             rancher-webhook-58b8d9f6c6-bxmns    1/1     Running     0          3m24s
fleet-system              fleet-agent-7c7d457b6d-x4kvf        1/1     Running     0          2m33s
fleet-system              fleet-controller-5ddfd96f5c-lxbnt   1/1     Running     0          3m10s
fleet-system              gitjob-7b4ddfcbf7-6hx52             1/1     Running     0          3m10s
kube-system               coredns-66c464876b-hfjqk            1/1     Running     0          4m1s
rancher-operator-system   rancher-operator-5cbfb5d6d7-kq29z   1/1     Running     0          3m32s

root@3fd636aa513e:/var/lib/rancher# k3s crictl images
IMAGE                                        TAG                 IMAGE ID            SIZE
docker.io/rancher/coredns-coredns            1.6.9               4e797b3234604       43.3MB
docker.io/rancher/pause                      3.1                 da86e6ba6ca19       746kB
harbor.kingsd.top/rancher/fleet-agent        v0.3.5              2a0c55d5db357       55.9MB
harbor.kingsd.top/rancher/fleet              v0.3.5              981b7123a405f       23.9MB
harbor.kingsd.top/rancher/gitjob             v0.1.15             dac9b6c58fe07       24.8MB
harbor.kingsd.top/rancher/rancher-operator   v0.1.4              c18d03bea7c6f       14.5MB
harbor.kingsd.top/rancher/rancher-webhook    v0.1.0              aec2ca2e747d7       12.6MB
harbor.kingsd.top/rancher/shell              v0.1.6              2e550736e6a49       46.8MB

As you can see, except for the built-in rancher/coredns-coredns and rancher/pause mirrors of Rancher Server, all other mirrors are pulled from harbor.kingsd.top.

private warehouse is a self-signed certificate (https), with certification

The main difference between Rancher Server's private repositories of self-signed certificates and SSL certificate authority private repositories is that when connecting to the private repositories of self-signed certificates, you need to specify the self-signed certificate of the private mirror repository in the registries.yaml of k3s.

1. Configure K3s registries.yaml

root@ip-172-31-14-159:~# cat /opt/registries.yaml
mirrors:
  # 私有仓库域名
  harbor.kingsd.top:
    endpoint:
      - "https://harbor.kingsd.top"
configs:
  "harbor.kingsd.top":
    auth:
      username: admin  # 这是私有镜像仓库的用户名
      password: Password  # 这是私有镜像仓库的密码
    tls:
      ca_file: /opt/certs/ca.crt # 镜像仓库中使用的ca文件的路径。
      cert_file: /opt/certs/harbor.kingsd.top.cert  # 镜像仓库中使用的cert文件的路径。
      key_file: /opt/certs/harbor.kingsd.top.key # 镜像仓库中使用的key文件的路径。

2. Start the single node Rancher Server

root@ip-172-31-14-159:~# docker run -itd --privileged \
>     --restart=unless-stopped \
>     -p 80:80 -p 443:443 \
>     -e CATTLE_SYSTEM_DEFAULT_REGISTRY=harbor.kingsd.top \   # 设置私有仓库域名
>     -v /opt/registries.yaml:/etc/rancher/k3s/registries.yaml \  # 将宿主机`registries.yaml`映射到容器内
>     -v /opt/certs:/opt/certs \  # 将证书映射到容器内
>     rancher/rancher:v2.5.8

3. Verification

After waiting for Rancher Server to start successfully, we can exec into the container to confirm whether the required components are started:

root@381b5d2c26d9:/var/lib/rancher# kubectl get pods -A
NAMESPACE                 NAME                                READY   STATUS      RESTARTS   AGE
cattle-system             helm-operation-b5mvm                0/2     Completed   0          6m27s
cattle-system             helm-operation-cdlc8                0/2     Completed   0          5m57s
cattle-system             helm-operation-hcxmj                0/2     Completed   0          5m47s
cattle-system             helm-operation-vqz9z                0/2     Completed   0          6m4s
cattle-system             helm-operation-wqgz9                0/2     Completed   0          6m12s
cattle-system             rancher-webhook-58b8d9f6c6-z68ps    1/1     Running     0          6m18s
fleet-system              fleet-agent-7c7d457b6d-zznvd        1/1     Running     0          5m25s
fleet-system              fleet-controller-5ddfd96f5c-kcqmq   1/1     Running     0          5m53s
fleet-system              gitjob-7b4ddfcbf7-99l46             1/1     Running     0          5m53s
kube-system               coredns-66c464876b-mflfv            1/1     Running     0          6m49s
rancher-operator-system   rancher-operator-5cbfb5d6d7-prsqh   1/1     Running     0          6m9s

root@381b5d2c26d9:/var/lib/rancher# k3s crictl images
IMAGE                                        TAG                 IMAGE ID            SIZE
docker.io/rancher/coredns-coredns            1.6.9               4e797b3234604       43.3MB
docker.io/rancher/pause                      3.1                 da86e6ba6ca19       746kB
harbor.kingsd.top/rancher/fleet-agent        v0.3.5              2a0c55d5db357       55.9MB
harbor.kingsd.top/rancher/fleet              v0.3.5              981b7123a405f       23.9MB
harbor.kingsd.top/rancher/gitjob             v0.1.15             dac9b6c58fe07       24.8MB
harbor.kingsd.top/rancher/rancher-operator   v0.1.4              c18d03bea7c6f       14.5MB
harbor.kingsd.top/rancher/rancher-webhook    v0.1.0              aec2ca2e747d7       12.6MB
harbor.kingsd.top/rancher/shell              v0.1.6              2e550736e6a49       46.8MB

As you can see, except for the built-in rancher/coredns-coredns and rancher/pause mirrors of Rancher Server, all other mirrors are pulled from harbor.kingsd.top.

private warehouse does not use SSL (using http), with certification

For HTTP private warehouses, you only need to modify the mirrors.endpoint configuration in registries.yaml to start with http.

1. Configure K3s registries.yaml

root@ip-172-31-14-159:~# cat cat /opt/registries.yaml
cat: cat: No such file or directory
mirrors:
  # 私有仓库IP
  3.96.56.137:
    endpoint:
      - "http://3.96.56.137"
configs:
  "3.96.56.137":
    auth:
      username: admin  # 这是私有镜像仓库的用户名
      password: Password  # 这是私有镜像仓库的密码

2. Start the single node Rancher Server

root@ip-172-31-14-159:~# docker run -itd --privileged \
>     --restart=unless-stopped \
>     -p 80:80 -p 443:443 \
>     -e CATTLE_SYSTEM_DEFAULT_REGISTRY=3.96.56.137 \  # 设置私有仓库IP
>     -v /opt/registries.yaml:/etc/rancher/k3s/registries.yaml \  # 将宿主机`registries.yaml`映射到容器内
>     rancher/rancher:v2.5.8

3. Verification

root@cb018bb70446:/var/lib/rancher# kubectl get pods -A
NAMESPACE                 NAME                                READY   STATUS      RESTARTS   AGE
cattle-system             helm-operation-44tb7                0/2     Completed   0          77s
cattle-system             helm-operation-cwpvz                0/2     Completed   0          66s
cattle-system             helm-operation-f898m                0/2     Completed   0          58s
cattle-system             helm-operation-fc4tj                0/2     Completed   0          51s
cattle-system             helm-operation-qq4kz                0/2     Completed   0          42s
cattle-system             rancher-webhook-c49756c7f-rjwdj     1/1     Running     0          63s
fleet-system              fleet-agent-55865c8959-rz8p2        1/1     Running     0          21s
fleet-system              fleet-controller-797ff98bfd-xj48k   1/1     Running     0          47s
fleet-system              gitjob-58bdfc4c69-mp84z             1/1     Running     0          47s
kube-system               coredns-66c464876b-dbm8v            1/1     Running     0          96s
rancher-operator-system   rancher-operator-578b4c64d4-4ptq9   1/1     Running     0          69s

root@cb018bb70446:/var/lib/rancher# k3s crictl images
IMAGE                                  TAG                 IMAGE ID            SIZE
3.96.56.137/rancher/fleet-agent        v0.3.5              2a0c55d5db357       55.9MB
3.96.56.137/rancher/fleet              v0.3.5              981b7123a405f       23.9MB
3.96.56.137/rancher/gitjob             v0.1.15             dac9b6c58fe07       24.8MB
3.96.56.137/rancher/rancher-operator   v0.1.4              c18d03bea7c6f       14.5MB
3.96.56.137/rancher/rancher-webhook    v0.1.0              aec2ca2e747d7       12.6MB
3.96.56.137/rancher/shell              v0.1.6              2e550736e6a49       46.8MB
docker.io/rancher/coredns-coredns      1.6.9               4e797b3234604       43.3MB
docker.io/rancher/pause                3.1                 da86e6ba6ca19       746kB

As you can see, except for the built-in rancher/coredns-coredns and rancher/pause mirrors of Rancher Server, the other mirrors are all pulled from 3.96.56.137.

postscript

The single node Rancher Server connecting to the private warehouse is actually the built-in K3s cluster connecting to the private warehouse. For more k3s private mirror warehouse configuration, please refer to the k3s official website ( http://docs.rancher.cn/docs/k3s/installation/private-registry /_index ).