玩转AWS（七）：EBS CSI driver for EKS

本文记录使用AWS CLI创建Amazon EBS CSI插件的配置过程、出现的问题及解决方案。

# ${YOUR_CLUSTER_NAME} 你的EKS集群的名字
# ${YOUR_REGION_NAME} 你的集群所在区域的代号
# ${YOUR_ACCOUNT_ID}  你的IAM用户账户ID

# 获取OpenID Connect 提供商 URL，也可以在EKS集群概述界面看到
aws eks describe-cluster --name ${YOUR_CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text
    
https://oidc.eks.${YOUR_REGION_NAME}.amazonaws.com/id/XXX

# 编辑policy文件
cat aws-ebs-csi-driver-trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${YOUR_ACCOUNT_ID}:oidc-provider/oidc.eks.region-code.amazonaws.com/id/XXX"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.region-code.amazonaws.com/id/XXX:aud": "sts.amazonaws.com",
          "oidc.eks.region-code.amazonaws.com/id/XXX:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa"
        }
      }
    }
  ]
}

aws iam create-role \
  --role-name AmazonEKS_EBS_CSI_DriverRole \
  --assume-role-policy-document file://"aws-ebs-csi-driver-trust-policy.json"

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
  --role-name AmazonEKS_EBS_CSI_DriverRole


aws eks create-addon --cluster-name ${YOUR_CLUSTER_NAME} --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::${YOUR_ACCOUNT_ID}:role/AmazonEKS_EBS_CSI_DriverRole

部署示例应用
git clone https://github.com/kubernetes-sigs/aws-ebs-csi-driver.git
cd aws-ebs-csi-driver/examples/kubernetes/dynamic-provisioning/
kubectl apply -f manifests/
kubectl get pods --watch

问题及修复

# 发现pod Pending
kubectl describe pvc ebs-claim

Warning  ProvisioningFailed  59s                 ebs.csi.aws.com_ebs-csi-controller-55c5976b48-vjks8_3cf9ed57-43a2-4754-af70-8d303436ddd5  failed to provision volume with StorageClass "ebs-sc": rpc error: code = Internal desc = Could not create volume "pvc-4fcadb3c-93a7-442d-b6a3-624e93b3d9a9": could not create volume in EC2: WebIdentityErr: failed to retrieve credentials
caused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity
          status code: 403, request id: 471eca6b-de6e-469d-abb3-0f6a596ab2bb

kubectl get serviceaccount -n kube-system |grep ebs
发现没有创建serviceaccount

先删除CloudFormation stack
CloudFormation\stack\eksctl-xxx-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa

kubectl get all -l app.kubernetes.io/name=aws-ebs-csi-driver -n kube-system

kubectl delete deploy ebs-csi-controller -n kube-system
kubectl delete daemonset ebs-csi-node -n kube-system
kubectl delete daemonset ebs-csi-node-windows -n kube-system

重新创建iamserviceaccount
eksctl create iamserviceaccount     --name ebs-csi-controller-sa     --namespace kube-system     --cluster ${YOUR_CLUSTER_NAME}     --role-name AmazonEKS_EBS_CSI_DriverRole     --role-only     --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy     --approve

重新部署aws-ebs-csi-driver
aws eks create-addon --cluster-name ${YOUR_CLUSTER_NAME} --addon-name aws-ebs-csi-driver   --service-account-role-arn arn:aws:iam::${YOUR_ACCOUNT_ID} :role/AmazonEKS_EBS_CSI_DriverRole

eksctl get addon --name aws-ebs-csi-driver --cluster ${YOUR_CLUSTER_NAME}

再次查看已恢复正常

[root@ip-172-31-31-169 storage]# kubectl get all -l app.kubernetes.io/name=aws-ebs-csi-driver -n kube-system
NAME                                      READY   STATUS    RESTARTS   AGE
pod/ebs-csi-controller-55c5976b48-5vkmm   6/6     Running   0          2m55s
pod/ebs-csi-controller-55c5976b48-9gpbm   6/6     Running   0          2m55s
pod/ebs-csi-node-2mjkh                    3/3     Running   0          2m55s
pod/ebs-csi-node-6j4bl                    3/3     Running   0          2m55s
pod/ebs-csi-node-tlm5b                    3/3     Running   0          2m55s

NAME                                  DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR              AGE
daemonset.apps/ebs-csi-node           3         3         3       3            3           kubernetes.io/os=linux     2m55s
daemonset.apps/ebs-csi-node-windows   0         0         0       0            0           kubernetes.io/os=windows   2m56s

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ebs-csi-controller   2/2     2            2           2m55s

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/ebs-csi-controller-55c5976b48   2         2         2       2m55s
[root@ip-172-31-31-169 storage]# kubectl get pvc
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
ebs-claim   Bound    pvc-4fcadb3c-93a7-442d-b6a3-624e93b3d9a9   4Gi        RWO            ebs-sc         50m
[root@ip-172-31-31-169 storage]# kubectl get po 
NAME   READY   STATUS    RESTARTS   AGE
app    1/1     Running   0          51m

参考文档

https://docs.aws.amazon.com/eks/latest/userguide/csi-iam-role...