A Preliminary Discussion on Istio Microservice Application Deployment Based on Alibaba Cloud ASK

Author: Wang Feilong (Nothing)

At present, Kubernetes has become the de facto standard for container orchestration systems in the industry. The cloud-native application ecosystem (Helm, Istio, Knative, Kubeflow, Spark on Kubernetes, etc.) based on Kubernetes has made Kubernetes a cloud operating system. In this context, serverless containers have become one of the evolutionary directions of the existing Container as a Service. On the one hand, the serverless method fundamentally solves the management complexity of Kubernetes itself, so that users do not need to be trapped in Kubernetes cluster capacity planning and security maintenance. , fault diagnosis; on the other hand, it further releases the capabilities of cloud computing, and realizes the requirements of security, availability, and scalability by the infrastructure.

As an Alibaba Cloud Serverless Kubernetes platform [ 1] , ASK not only has major capabilities such as free operation and maintenance, second-level elasticity, ultra-large Pod capacity, and elastic prediction, but more importantly, it is still a standard Kubernetes platform.

This article will verify ASK's compatibility with standard Kubernetes by trying Istio to deploy microservice applications on ASK. As the leading solution of Service Mesh (service mesh), Istio is sufficiently complex and representative on the one hand, and on the other hand, it also represents the trend of microservice architecture in the cloud-native era and has reference significance.

Let's get started now!

Create a cluster

Before trying Istio, you need to prepare an ASK cluster. Log in to the Alibaba Cloud console, select Products and Services > Container Service for Kubernetes , and select a cluster on the left sidebar to enter the cluster list page. Click Create Cluster in the upper right corner to start creating a cluster, and configure the cluster parameters as follows:

Cluster name: hello-istio
Cluster Specifications: Pro Edition
Region: United States (Silicon Valley)
Payment Type: Pay-As-You-Go
Kubernetes version: 1.20.11-aliyun.1
Private Network: Automatically Created
Service CIDR: 172.21.0.0/20
API Server Access: Standard I (slb.s2.small)
Expose API Server using EIP: yes
Time zone: Aisa/Shanghai (UTC+08:00)
Service Discovery: CoreDNS
Using the log service: create a new project

After confirming the configuration, click Create Cluster to enter and wait for the cluster to be created. Istio relies on DNS services. Here, the CoreDNS component is installed by default when creating a cluster.

title=

After the cluster is created, go to the Cluster List > hello-istio > Details > Cluster Information > Connection Information page, copy the public network access content to the local /tmp/kube/config file, and configure the kubelet with the following commands:

 $ export KUBECONFIG=/tmp/kube/config

Try Istio

Once kubectl is configured, you can start installing and trying Istio on your cluster.

Download Istio

Go to the Istio release page [ 2] to download the installation file for the operating system, or you can download and extract the latest version through the following commands:

 $ curl -L https://istio.io/downloadIstio | sh -

Because my local ~/bin directory has been added to the PATH, here I will copy the extracted Istio directory to the ~/bin directory and build a soft link.

 $ cp istio-1.13.3 ~/bin
$ cd ~/bin
$ ln -s istio-1.13.3/bin/istioctl
$ ls -al ~/bin/
total 28
drwxr-xr-x   5 feilong.wfl staff   160  5  4 22:40 ./
drwxr-xr-x+ 95 feilong.wfl staff  3040  5  8 22:30 ../
drwxr-x---   9 feilong.wfl staff   288  4 15 00:48 istio-1.13.3/
lrwxr-xr-x   1 feilong.wfl staff    25  5  4 22:40 istioctl -> istio-1.13.3/bin/istioctl*

If the output of the istioctl --help command is OK, then istioctl is configured correctly.

Install Istio

This installation uses the demo profile [ 3] , which contains a set of features specially prepared for testing, as well as configuration combinations for user production or performance testing.

 $ istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete

Add a tag to the namespace instructing Istio to automatically inject the Envoy sidecar proxy when deploying the application:

 $ kubectl label namespace default istio-injection=enabled
namespace/default labeled

Deploy the sample app

Deploy the Bookinfo sample application [ 4] :

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created

Check that the Pod is ready:

 $ kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
details-v1-79f774bdb9-t2jhq       2/2     Running   0          2m54s
productpage-v1-6b746f74dc-qc9lg   2/2     Running   0          2m46s
ratings-v1-b6994bb9-tmbh6         2/2     Running   0          2m51s
reviews-v1-545db77b95-xdhp4       2/2     Running   0          2m49s
reviews-v2-7bf8c9648f-4gn6f       2/2     Running   0          2m48s
reviews-v3-84779c7bbc-jfndj       2/2     Running   0          2m48s

To wait and make sure all Pods reach this state: READY has a value of 2/2, and STATUS has a value of Running. Depending on the platform, this process may take several minutes.

Check that the Service is ready:

 $ kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
details       ClusterIP   172.21.11.168   <none>        9080/TCP   59s
kubernetes    ClusterIP   172.21.0.1      <none>        443/TCP    33m
productpage   ClusterIP   172.21.0.124    <none>        9080/TCP   51s
ratings       ClusterIP   172.21.9.7      <none>        9080/TCP   57s
reviews       ClusterIP   172.21.13.223   <none>        9080/TCP   55s

Make sure that the web service is working properly. If the command returns the page title, the application is already running in the cluster.

 $ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -s productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>

open service

Now, the BookInfo application has been deployed, but not accessible to the outside world. To open access, create an Istio Ingress Gateway, which routes a path to services within the mesh.

Associate the application to the Istio gateway:

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/bookinfo-gateway.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created

Make sure there are no issues with the config file:

 $ istioctl analyze
✔ No validation issues found when analyzing namespace: default.

Determine Inbound IP and Port

Use the following commands to set the INGRESS_HOST and INGRESS_PORT variables for the access gateway:

 $ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')

Set the variable GATEWAY_URL and make sure that both the IP address and port are successfully assigned to the variable:

 $ export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT
$ echo "$GATEWAY_URL"
47.88.21.82:80

Verify external access

Run the following command to get the external access address of the Bookinfo application:

 $ echo "http://$GATEWAY_URL/productpage"
http://47.88.21.82:80/productpage

Copy the output address of the above command to a browser and access it to confirm that Bookinfo has achieved external access. Refresh the page and find that the display style of Book Reviews is constantly changing.

title=

View the Dashboard

Dashboards can help understand the structure of the service mesh, display the topology of the network, and analyze the health of the mesh.

First install Kiali and other plugins and wait for the deployment to complete.

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/addons
$ kubectl rollout status deployment/kiali -n istio-system
Waiting for deployment "kiali" rollout to finish: 0 of 1 updated replicas are available...
deployment "kiali" successfully rolled out

Access the Kiali Dashboard.

 $ istioctl dashboard kiali

In the navigation menu on the left, select Graph , and then in the Namespace drop-down list, select default .

The Kiali dashboard shows an overview of the grid and the relationships between the various services of the Bookinfo sample application. It also provides filters to visualize the flow of traffic.

title=

Add default target rule

Before using Istio to control Bookinfo version routing, it is necessary to define the available versions in the target rule [ 5] . Run the following command to create a default target rule for the Bookinfo service:

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/destination-rule-all.yaml
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created

Wait a few seconds for the target rule to take effect. You can view target rules with the following command:

 $ kubectl get destinationrules
NAME          HOST          AGE
details       details       30s
productpage   productpage   32s
ratings       ratings       31s
reviews       reviews       32s

Route all traffic to v1 version

Run the following command to create the v1 version of the Virtual Service that routes all traffic to the microservice:

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/virtual-service-all-v1.yaml
virtualservice.networking.istio.io/productpage created
virtualservice.networking.istio.io/reviews created
virtualservice.networking.istio.io/ratings created
virtualservice.networking.istio.io/details created

You can test the new configuration by refreshing the /productpage page of the Bookinfo application again. Note that no matter how many times you refresh, the comments section of the page will not show rating stars. This is because Istio is currently configured as a reviews service to route all traffic to version reviews:v1, which does not access the star rating service.

Routing based on user identity

Next, we will change the routing configuration to route all traffic from a specific user to a specific service version. All traffic from the user named Jason in the example will be routed to the service review:v2.

Istio does not have any special built-in mechanism for user identity. In this example, the productpage service adds a custom end-user request header to all HTTP requests to the reviews service to achieve this effect.

Run the following command to enable user-based routing:

 $ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
virtualservice.networking.istio.io/reviews created

Make sure the rule has been created:

 $ kubectl get virtualservice reviews -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"reviews","namespace":"default"},"spec":{"hosts":["reviews"],"http":[{"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"reviews","subset":"v2"}}]},{"route":[{"destination":{"host":"reviews","subset":"v1"}}]}]}}
  creationTimestamp: "2022-05-15T16:05:55Z"
  generation: 1
  name: reviews
  namespace: default
  resourceVersion: "1984849"
  uid: f3bd3dcb-d83c-4a75-9511-1fc9308ca05b
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

On the /productpage of the Bookinfo application, log in as user jason. Refresh your browser to see a star rating next to each review.

title=

Principles and Limitations

On the data plane, Istio is responsible for coordinating and controlling all network communication before the microservices through the Sidecar proxy (istio-proxy) injected into the Pod. In order for the sidecar proxy (istio-proxy) to hijack business container traffic, Istio needs to deliver iptables rules to the network where the Pod is located. In the normal installation mode, the iptables rules are issued by the initialization container istio-init injected by Istio into the Pod. Distributing iptables rules to the Pod network requires that the container can use the two high-level capabilities (Capabilities) of NET_ADMIN and NET_RAW. These two high-privilege capabilities in ASK cluster are affected by ASK Pod Security Policy [ 6] and ECI Container Security Policy [ 7] .

The CAPS of ASK Pod Security Policy is *, which means there is no limit.

 $ kubectl get psp
NAME             PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
ack.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *

ECI Container Security Policy allows configuration through container security context. The Pod injection template file in Istio ~/bin/istio-1.13.3/manifests/charts/istio-control/istio-discovery/files/injection-template.yaml contains the following code: Do not enable the Istio CNI plugin [ 8] will add NET_ADMIN and NET_RAW two high-privilege capabilities:

 securityContext:
  allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
  privileged: {{ .Values.global.proxy.privileged }}
  capabilities:
{{- if not .Values.istio_cni.enabled }}
    add:
    - NET_ADMIN
    - NET_RAW
{{- end }}
    drop:
    - ALL

Therefore, in principle, there is no compatibility problem when using Istio in the current ASK cluster.

Summarize

This time I tried high-complexity software such as Istio on ASK, and found no compatibility issues. And from the principle of positive analysis, it is proved that the possibility of compatibility problems is low. Therefore, ASK still has excellent compatibility with native Kubernetes. In the future, we will further explore other Istio functions on the ASK cluster to further verify the compatibility of ASK with native Kubernetes.

Reference link:

[1] Alibaba Cloud Serverless Kubernetes

https://help.aliyun.com/document_detail/127525.html

[2] Istio release page

https://github.com/istio/istio/releases/tag/1.13.3

[3] demo profile**

https://istio.io/latest/docs/setup/additional-setup/config-profiles/

[4] Bookinfo sample application

https://istio.io/latest/en/docs/examples/bookinfo/

[5] Target Rules

Available versions are defined in https://istio.io/latest/docs/concepts/traffic-management/#destination-rules )

[6] ASK Pod Security Policy