Experience sharing of AI projects based on SAP BTP platform

The author has worked in the SAP Chengdu Research Institute for more than fifteen years, and has also participated in the project development and prototype verification of some AI services based on the SAP Business Technology Platform (the Chinese name is SAP Business Technology Platform, hereinafter referred to as SAP BTP). This article shares the experience of these projects on AI, and hopes that AI experts in the community will give them advice.

SAP BTP combines intelligent enterprise applications with database and data management, analytics, integration, and scaling capabilities into a single platform for cloud and hybrid environments, including hundreds of pre-built integrations for SAP and third-party applications. One of them is SAP AI Business Services.

How to consume SAP AI services in Java programs

The background of this project is as follows: develop a Java program, the user can upload a picture, the Java program calls the SAP AI API, the latter will use the pre-trained machine learning model to recognize the picture and return a text format to the user The response information tells the user the recognition result.

The following are the specific implementation steps.

Visit https://api.sap.com , click on API:

Select SAP Leonardo Machine Learning - Functional Services:

Select the image recognition AI service in the Functional Services category, namely Product Image Classification API:

The API model is trained by SAP based on about 50,000 Icecat pictures and can distinguish 29 different product categories. These categories can be found on official documents, such as computer monitors, digital cameras, external storage devices, keyboards, LCD TVs, mobile phone chargers, notebooks and other peripherals and more.

You can open the technical specification page of the Image Recognition API, including the introduction of the API Model Schema, that is, the field names and data types included in the returned response structure after calling the API.

The interface also embeds a small API calling console, you can directly choose to upload a local graphic file in the console, and click the Try it out button to experience the effect of the API.

Use the image below to test:

Seeing that the request is processed successfully in the console, the API judges that the picture has a 97% probability of being a notebook.

After the API test is passed, the next step is to consume it in the Java program. There is a button Download SDK in the upper right corner of the API console. After clicking, the SDK will be downloaded locally. You can see that the SDK is a Gradle-based Java project. So we need to download Maven and Gradle and complete the environment variable configuration.

After the Java development environment is ready, use the import function of Eclipse to import the SDK into Eclipse. After the import is completed, the following figure is shown: the red area is the code that comes with the AI Service SDK, and the blue area is the code manually created by the author to call the API and print the result.

Edit the pom.xml in the root directory and maintain the following dependencies:

com.sap.apibhub.sdk, version 1.0.0

Execute the Maven command mvn install in the project root directory to ensure that the project is successfully built. At this point, you can start writing Java code to use the SAP AI SDK to call the AI API.

The SDK encapsulates the details of the sending of the underlying HTTP request and the parsing of the response, which is very simple to use. The inferenceSyncPost function on line 15 receives a local File object and sends an HTTP request to the SAP AI Service Endpoint.

The API key on line 8, which can be obtained from the API console:

Execute the Java application and print the output in the Eclipse console. AI Service judges that there is a 97% probability that this image is a notbook:

How to Consume SAP AI Services in a Web Application

Log in to the SAP cloud platform and open WebIDE:

Because JavaScript is used to call the remote AI API in the browser environment, in order to avoid cross-domain problems, it is necessary to create a Destination in the cloud platform, which is similar to the Destination created in the ABAP Netweaver transaction code SM59. All HTTP Request and Response All go through this Destination.

The properties are shown in the figure above, and the url is maintained as the corresponding sandbox environment: https://sandbox.api.sap.com/ml

Note down the name of this Destination, sapui5ml-api, because it will be used later in the JavaScript code of the web application.

Maintain the additional property WebIDEnabled as true, so that the Destination can be used in the WebIDE application environment. Click Check Connection to make sure you see a green light, indicating that the connection between the SAP Cloud Platform Destination and the AI Service Endpoint is available.

Open the settings.json file in the web application project in WebIDE, and paste the API Key copied from the API console here:

In the neo-app.json file in the project root directory, maintain the name of the target object of type destination as the same name sapui5ml-api as the Destination created in the SAP cloud platform before:

Run this web application, you can see the following interface:

Do some simple tests for this web application, upload the following image to the web application:

The SAP AI API, identified this image as 74.7% likely to be a laptop, 13.8% likely to be a keyboard, and 11.3% likely to be Tablets.

Click the View JSON button of the web application to see the technical details returned by the AI service.

How to Retrain Machine Learning Models on SAP Cloud Platform

If the image category uploaded by the user through the Java application or Web application described above is not supported by the model pre-trained by the SAP AI service, we can retrain the machine learning model of the AI service by ourselves.

Suppose we want the Product Image Classfication machine learning model to recognize a variety of flowers. First, prepare a lot of pictures of different types of flowers. On the official website of Tensorflow, a zip package for practice has been provided considerately for learners who want to try to retrain the AI model, which contains a large number of pictures of various flowers:

http://download.tensorflow.org/example_images/flower_photos.tgz

The data set used to retrain the AI model must conform to the hierarchical structure shown in the figure below, that is, under the three folders of training, validation and test, each contains subfolders named after product categories, and the data scale ratio is 8 :1:1.

The SAP cloud platform service key contains an IMAGE_RETRAIN_API_URL, which can be used to obtain the online storage url of the data set to be uploaded for retraining the AI model:

Send an HTTP Get request to this url to get the url stored online:

Paste this url into the browser, enter the accessKey and secretKey returned in Postman to log in, and then you can access the online storage in Web mode:

The next step is to upload the local training files to the online storage deployed in AWS.

First, use the command line mc config host to define a remote site named sapjerrys3, and bind the AWS online storage url, accessKey and secret obtained from Postman in the previous step to this site:

Then upload the file using the command line:

mc.exe cp -r C:\Code\MachineLearningStudy\flowersjerry sapjerrys3\data

After about ten minutes, the file upload is complete:

At this point, you can view the uploaded training files in AWS online storage from the browser.

After the retrained dataset is ready, the next step is to submit a model retraining request through a background job. Use Postman to send an HTTP Post request to start a background job for model retraining.

Use the command line to query the status of the background job, and when the status changes to SUCCEEDED, the model training has ended.

After the model is retrained, consume it. The url format is:

https://mlfinternalproduction-image-classifier.cfapps.sap.hana.ondemand.com/api/v2/image/classification/models/<model name>/versions/1

Use a sunflower image:

Send it to the retrained model flowerjerrymodel as a parameter of HTTP Post, and the result shows that AI Service thinks that this picture has about 87% probability of being a sunflower.

Finally, let's look at the specific application of an AI service in the Intelligent Serbice Scenario scenario.

A maintenance engineer accepts a customer's maintenance request, visits a certain equipment, and finds that a part of the equipment is damaged. Suppose the technician, for various reasons, was unable to identify the model of the part empirically. At this time, the technician took out his mobile phone, took a picture of the part, and automatically identified the exact model of the part through the AI API through the SAP intelligent service solution installed on the mobile phone (such as the Java program and Web application described earlier in this article), and returned to a maintenance engineer.

The SAP cloud platform AI service, after receiving the picture uploaded by the maintenance engineer, extracts the feature vector of the picture, and identifies the accurate model through the model trained on the platform based on massive data sets.

The extraction of feature vector, mathematically speaking, is the process of converting the binary stream of the input image into a vector (one-dimensional matrix) through some algorithm.

Take the following figure trapezoid and circle as an example, we divide the figure into 9 areas evenly, and observe the gradient direction of the graphic unit in each area at the center of the figure, we can achieve dimensionality reduction, and represent the two-dimensional image as a one-dimensional matrix .

For programmers who develop applications based on SAP AI services, they do not need to know the specific algorithm of image feature vector extraction. They only need to push the image of the feature vector to be extracted to the AI Service Endpoint through Restful API call, and then the feature vector can be obtained. vector output.

Enter the test console and help documentation about the image feature vector extraction service in SAP AI Service through the following url:

https://api.sap.com/api/img_feature_extraction_api/resource

Upload a local image from the console and click the Execute button:

The output feature vector is obtained:

Now that we have the feature vector, let's go back to the enterprise intelligence service scenario we are discussing. We use another artificial intelligence service related to image processing from SAP AI service: Inference Service for Similarity Scoring.

The input to this image similarity scoring AI API requires two compressed packages. The content of the file in the first compressed package is a series of feature vectors, which come from the image files uploaded by the maintenance engineer. The second compressed package stores the feature vectors of the pictures of parts with models A, B, C...etc.

As shown in the figure above, for the sake of simplicity, I only store two files in the second archive, which store the feature vectors of part model A and model B respectively.

Use Postman to call the API, passing in these two archives:

The obtained results show that the similarity coefficient between the picture uploaded by the maintenance engineer and the model A is almost close to 1, so obviously, the picture represents the model A.

Of course, in addition to directly uploading compressed files containing image feature vectors, it is also a common way to call the API by using the feature vector content contained in these files as input in the form of JSON strings. The format of JSON strings is available on the SAP official website. There are detailed instructions here, so I won't repeat them here.

I hope that the usage scenarios of these SAP AI APIs introduced in this article are helpful to you, thank you for reading.