Cloud platform SDK strategy

As a cloud native developer, which aspect of development experience do you pay most attention to? Interacting with resources on the cloud in your application is definitely something you care about, whether it's sending a message to the Service Bus or creating a new virtual machine for larger computing needs. You can use the SDK of the cloud platform, such as the Azure SDK, to do all these operations, so how is the Azure SDK written? What is the difference between a cloud platform SDK and a traditional SDK?

Differences between traditional SDKs and cloud platform SDKs

Traditional SDKs, such as .NET SDK or JAVA SDK, are programming interfaces of .NET runtime (.NET CLR) or JAVA runtime (JVM). The "communication protocol" between them is in-process and based on binary of.
Cloud platform SDKs are usually defined based on OpenAPI. OpenAPI is more like a cross-process communication protocol based on RESTFul HTTP APIs. OpenAPI is ubiquitous in the cloud computing world, but OpenAPI has its own limitations.

Problems encountered when using OpenAPI to interact with Azure

OpenAPI is an interface definition of a REST API Service, and it is not an out-of-the-box solution for any programming language. When a developer wants to start using OpenAPI, he usually encounters the following problems:

• exception handling
• Handling of long-running operations
• Paging operation processing
• HTTP Client Lifecycle Management

Building robust code for these processing is not an easy task. In this case, introducing the SDK becomes a natural choice.

How Microsoft generated the Azure SDK from OpenAPI

The design of the Azure SDK follows the general SDK library design guidelines . This guideline ensures a consistent SDK experience between different Azure Services. Starting from March 31, 2022, we will start the SDK library that does not conform to the guideline version. phased out.

To create an SDK for a new Azure Service, the Azure SDK team works closely together, and the common working steps are as follows:

1. The Azure Service team defines the OpenAPI specification (Swagger file) and makes a PR in the REST API specification repository .
2. The Azure SDK team reviews the new OpenAPI specification to make sure there are no breaking changes, syntax errors, etc.
3. The Azure SDK team uses AutoRest to generate client libraries from OpenAPI specifications. AutoRest is an open source client library generation tool created by Microsoft to access Restful web services. AutoRest uses extensions in different languages to generate client libraries for Restful web services for different programming languages.
4. We will review the generated client libraries and make a PR to the Github repository of the corresponding language SDK. We have a great internal engineering platform to test these client libraries to ensure that we do not introduce any regression issues. These tests include auto-generated tests and hand-written tests, but in the future we will rely more on auto-generated test cases.
5. Once all checks are passed, this newly generated client library is merged into the SDK Github Repository for the corresponding language.
6. After the client library is merged into the SDK Github repository of the corresponding language, the corresponding continuous integration and deployment pipeline will be triggered, and if everything is correct, the new client library will be published to the package manager of the corresponding language, such as nuget , npm, etc.
7. All SDK related documents and sample codes will be updated one after another.

AutoRest is the key to the entire client library generation process. AutoRest knows common patterns of REST API calls and exposes these calls as a developer-friendly programming interface in the client library. For example, long-running operations (LROs) and some REST API calls for resource creation, deletion, etc. cannot immediately know the result of the operation. In these cases, the server will return a 201 (Created) or 202 (Accepted) status code with a link to continuously monitor the completion status of the request. This operation behavior is reasonable for a direct RESTFul API call, but is problematic from a programming language point of view, this operation should be turned into an asynchronous operation with a callback function that allows the developer to operate for the LRO success or failure write the corresponding code. OpenAPI, unfortunately, does not have support for this asynchronous syntax. To support this operation, Microsoft has defined a series of AutoRest OpenAPI extensions to enrich the OpenAPI syntax.

Now let's go back to the previous LRO scenario. We can define an x-ms-long-running-operation in OpenAPI. For example, the following definition is an OpenAPI definition for creating/updating an Azure VM (full definition can be found here )

 "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}": {
      "put": {
        "tags": [
          "VirtualMachines"
        ],
        "operationId": "VirtualMachines_CreateOrUpdate",
        "description": "The operation to create or update a virtual machine. Please note some properties can be set only during virtual machine creation.",
        "parameters": [
          {
            "name": "resourceGroupName",
            "in": "path",
            "required": true,
            "type": "string",
            "description": "The name of the resource group."
          },
          {
            "name": "vmName",
            "in": "path",
                        "required": true,
            "type": "string",
            "description": "The name of the virtual machine."
          },
          {
            "name": "parameters",
            "in": "body",
            "required": true,
            "schema": {
              "$ref": "#/definitions/VirtualMachine"
            },
            "description": "Parameters supplied to the Create Virtual Machine operation."
          },
          {
            "$ref": "#/parameters/ApiVersionParameter"
          },
          {
            "$ref": "#/parameters/SubscriptionIdParameter"
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "$ref": "#/definitions/VirtualMachine"
            }
          },
          "201": {
            "description": "Created",
            "schema": {
              "$ref": "#/definitions/VirtualMachine"
            }
          },
          "default": {
            "description": "Error response describing why the operation failed.",
            "schema": {
              "$ref": "#/definitions/CloudError"
            }
          }
        },
        "x-ms-long-running-operation": true
      }
   }

With the above OpenAPI definition, the BeginCreateOrUpdate method call in our generated client library becomes the following, taking the Go language as an example (the complete code can be found here )

 pollerResponse, err := vmClient.BeginCreateOrUpdate(ctx, resourceGroupName, vmName, parameters, nil)
    if err != nil {
        return nil, err
    }
​
    resp, err := pollerResponse.PollUntilDone(ctx, 10*time.Second)
    if err != nil {
        return nil, err
    }
}

The Poller class in the above code comes from the Azure Core project. Azure Core provides the infrastructure of the entire Azure SDK, such as HTTP Client lifecycle management and exception handling. Using Azure Core, we can define more extensions in OpenAPI and turn these extensions into more developer-friendly client library syntax. Common OpenAPI extensions are:

• X-ms-error-response: This extension can determine whether to map some http return status codes to an error
• X-ms-pageable: This extension can map the returned list into a pageable result

For more extensions, see AutoRest Extensions for OpenAPI

We discussed the benefits of using a cloud SDK and Microsoft's process for building a cloud SDK, thank you for reading.

---

Long press to identify the QR code and follow Microsoft China MSDN