Manage Amazon EKS Anywhere Clusters with KubeSphere

Amazon EKS Anywhere introduction

Recently, Amazon Cloud Technology has open Amazon EKS Anywhere . This project provides Amazon EKS with a new deployment model to easily create and manage Kubernetes clusters using virtual machines in an on-premises data center, and leverage the benefits of Amazon EKS Distro to bring consistent Kubernetes operations management to your data center experience. It can perform full lifecycle management of multiple Kubernetes clusters and flexibly deliver Amazon EKS container services to users.

Amazon EKS can help you automate cluster management, reduce costs, and eliminate the tedious work of using multiple open source or third-party tools to operate Kubernetes clusters. You can also build your own management tools to create Amazon EKS Distro clusters, configure operating environments, and update software.

With Amazon EKS Anywhere, you also get Amazon Cloud Technology's default optimizations for cluster configurations, machine operating systems, and third-party integrations that let you focus on business innovation rather than complex component compatibility or on-premises and cloud environments Deployment compatibility between. Additionally, you can utilize the Amazon EKS console to view all of your Kubernetes clusters deployed through Amazon EKS Anywhere.

Amazon EKS Anywhere currently only supports VMware vSphere, and will support bare metal deployments in the future.

KubeSphere introduction

KubeSphere is a container platform built on Kubernetes, completely open source, and provides multi-cloud and multi-cluster management, DevOps, observability, microservice governance, edge computing, application management and other functions through a minimal interface interaction, providing a powerful network With storage management capabilities, it can help enterprises to complete the rapid construction, deployment, and operation and maintenance of container architectures on heterogeneous infrastructures such as public cloud, virtualization, and physical machines, and realize continuous application delivery and full life cycle management.

The federated cluster management function of KubeSphere can manage multiple heterogeneous infrastructures Kubernetes, Amazon EKS and product families in a unified manner. When deploying an application, users can distribute multiple copies of the application to multiple different Kubernetes resource pools according to business requirements. , and unified operation and maintenance management through KubeSphere, from the upper layer to achieve cross-regional, cross-cluster high availability.

In January 2021, KubeSphere officially settled in Amazon Cloud Technology Quick Start, providing global users with the ability to deploy cloud native applications with one-click deployment of Amazon EKS and KubeSphere. Through KubeSphere's rich application management functions, it helps users accelerate cloud native landing.

What would be the effect of combining KubeSphere and Amazon EKS Anywhere? This article will lead you to practice.

This series of articles will be divided into two parts. This article introduces Amazon EKS Anywhere and how to install Amazon EKS Anywhere and KubeSphere; next time we have the opportunity to introduce how to use KubeSphere to unify Amazon EKS Anywhere and Amazon EKS.

Creating an Amazon EKS Anywhere cluster

Premise

Running Amazon EKS Anywhere requires:

• A vSphere 6.7+ environment running vCenter
• Ability to deploy 8-10 virtual machines
• The vSphere environment has a running DHCP service serving the created workload
• A network that can be made available to the cluster in a vSphere environment
• Import the OVA template into vSphere and make it into a template
• Have vCenter administrator privileges
• A management server for executing commands

Management server requirements:

• CPU Architecture: amd64
• CPU 4 cores
• Memory 16G
• Disk 100G
• Operating SystemMac OS (10.15) / Ubuntu (20.04.2 LTS)
• Ability to access vCenter via https/443
• The user must be in the docker user group or the root user to ensure that they have permission to operate docker
• have write access to the file
• Ability to access the following URLs:

1. public.ecr.aws
2. anywhere-assets.eks.amazonaws.com (download binaries, configuration manifests and OVA templates)
3. distro.eks.amazonaws.com (download Amazon EKS Distro)
4. d2glxqk2uabbnd.cloudfront.net (as a repository for Amazon EKS Anywhere and Amazon EKS Distro ECR)
5. api.github.com (not required if GitOps is not enabled)

An Amazon EKS Anywhere cluster needs to create 6 to 10 virtual machines, resource requirements per node:

• CPU 2 cores
• Memory 8G
• Disk 100G

download OVA mirror

First, you need to download the OVA image, and select the corresponding OVA image according to different operating systems and different Kubernetes versions:

• Ubuntu with Kubernetes 1.21
• Ubuntu with Kubernetes 1.20

import template

Templates can be imported using the vCenter web interface. We will not describe the importing templates in vCenter in detail here. Please refer to the document xxx.

After the template is added, you need to create a tag named "os:ubuntu" and a tag named "eksdRelease:kubernetes-1-20-eks-6".

Then check the two tags created and click Assign.

Setting up the management server

The management server is used to execute various operation commands, and docker and kubelet need to be installed first.

$ sudo apt update
$ sudo apt install -y docker.io
$ sudo usermod -a -G docker $USER
$ wget https://distro.eks.amazonaws.com/kubernetes-1-19/releases/4/artifacts/kubernetes/v1.19.8/bin/linux/amd64/kubectl
$ mkdir -p $HOME/bin
$ chmod +x kubectl
$ mv kubectl $HOME/bin/
$ echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc

*Swipe left to see more

After the installation is complete, you need to log out and log in again to obtain the correct user group permissions and PATH environment.

Install the latest version of eksctl (the minimum version of eksctl required by Amazon EKS Anywhere is 0.66.0).

$ curl "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" \
    --silent --location \
    | tar xz -C /tmp
$ sudo mv /tmp/eksctl /usr/local/bin/

*Swipe left to see more

Install eksctl-anywhere.

$ export Amazon EKSA_RELEASE="0.5.0" OS="$(uname -s | tr A-Z a-z)"
$ curl "https://anywhere-assets.eks.amazonaws.com/releases/eks-a/1/artifacts/eks-a/v${Amazon EKSA_RELEASE}/${OS}/eksctl-anywhere-v${Amazon EKSA_RELEASE}-${OS}-amd64.tar.gz" \
    --silent --location \
    | tar xz ./eksctl-anywhere
$ sudo mv ./eksctl-anywhere /usr/local/bin/

*Swipe left to see more

Create an authentication file used by the cluster for ssh login authentication to Amazon EKS Anywhere cluster nodes.

$ ssh-keygen -f eks-a
$ cat eks-a.pub
  ssh-rsa AAAAB3NzxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxoE= root@localhost

*Swipe left to see more

Get the sha1 thumbprint of the vCenter certification file.

# 安装 govc
$ wget -c https://github.com/vmware/govmomi/releases/download/v0.25.0/govc_Linux_x86_64.tar.gz -O - | tar -xz
$ chmod +x ./govc
$ mv ./govc /usr/local/bin
$ cd ..
$ rm -rf govc/
$ govc version -l
# 设置 vCenter 地址和管理员的用户密码
$ export GOVC_URL="xxx"
$ export VSPHERE_USERNAME=xxx
$ export VSPHERE_PASSWORD=xxx
# 获取 vCenter 认证文件 的 thumbprint
$ govc about.cert -k=true -thumbprint
$ xxx.xxx.xxx.xxx aa:bb:cc:xx:xx:xx:xx:cc:bb:aa

*Swipe left to see more

Create a cluster configuration file

Cluster names must be lowercase and do not contain uppercase or special characters.

$ CLUSTER_NAME=prod
$ eksctl anywhere generate clusterconfig $CLUSTER_NAME \
   --provider vsphere > eksa-cluster.yaml

*Swipe left to see more

The following is an annotated template reference, please delete the Chinese comments when using.

an Amazon EKS Anywhere

Before creating a cluster, you also need to set up a vSphere user and password.

# 设置你的 vCenter 管理员用户密码
$ export Amazon EKSA_VSPHERE_USERNAME='<Username>'
$ export Amazon EKSA_VSPHERE_PASSWORD='<Passwd>'

*Swipe left to see more

Finally create the cluster.

$ eks-a create cluster -f ${CLUSTER_NAME}.yaml

*Swipe left to see more

During the installation you will see the following output.

During the creation process, Amazon EKS Anywhere will create a .kubeconfig file in the cluster name directory (eg $PWD/$CLUSTER_NAME/), which you can use to view cluster status and deploy workloads through kubectl.

p$ export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
$ kubectl get ns

*Swipe left to see more

If your cluster is created on a local computer, you can save the KUBECONFIG file to the cluster management server, and you can also save the KUBECONFIG file on the management server to the local computer to manage the cluster.

Deploy KubeSphere v3.2.0

Preparing for persistent storage

The Amazon EKS Anywhere cluster uses Amazon Cloud CSI as the default storage by default, here we replace it with OpenEBS as the default storage.

$ kubectl apply -f https://openebs.github.io/charts/openebs-operator.yaml

*Swipe left to see more

Check the Pods under the OpenEBS namespace and wait for all Pods to start up.

$ kubectl get pods -n openebs

*Swipe left to see more

After the Pods in the OpenEBS namespace are all started, set OpenEBS as the default storage.

$ kubectl patch storageclass openebs-hostpath -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class": "true"}}}'

*Swipe left to see more

on Amazon EKS Anywhere cluster

(1) Execute the following command on the eks-a management node/management server to deploy KubeSphere v3.2.0

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/kubesphere-installer.yaml
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/cluster-configuration.yaml

*Swipe left to see more

(2) View the KubeSphere installation log

$ kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

*Swipe left to see more

(3) When you see "Welcome to KubeSphere!", the installation is complete.

#####################################################
###              Welcome to KubeSphere!           ###
#####################################################

*Swipe left to see more

Author of this article

Zhang

Amazon Cloud Technology Senior Architect

Responsible for the construction of Amazon cloud technology partner related solutions and partner ecological cooperation. Together with partners, according to customer needs, analyze the challenges and future directions they encounter at the technical architecture level, and design and implement architectural solutions based on the Amazon cloud technology platform and partner products. He has worked as a solution architect in well-known foreign companies and domestic leading companies. He has rich experience in cloud and network fields, and has a deep understanding of public cloud services and architecture.

Yao

KubeSphere Senior Software Engineer

Responsible for providing solutions, core members of the open source project KubeEye.

Yang

KubeSphere preacher

KubeSphere Open Source Community

Scan the QR code above to register now