I only used 3 steps to achieve a realistic 3D scene rendering

How many steps do you need to go through to render 3D models and environmental scenes with both texture and realism?

Obviously, the current 3D model material rendering technology cannot achieve standardized tasks that can be done in a few simple steps to quantify and complete a quality 3D model rendering. Generally, you need:

1. prepares a rich texture and material library : To render the desired material effect, it is necessary to collect the corresponding material appearance and texture material according to the shape details and complexity of the specific model scene, which is time-consuming and complicated by the model The degree is proportional, and most of the materials on the market are paid for, and the cost is high.

2, scratch Scheduling : 3D modeling tools are available to enrich the model parameters to achieve rendering operation, to set and controlled by adjusting the model parameters.

3. The relies on art experience : 3D modelers have gradually become in-demand talents in the industry, which also shows that 3D model production requires skilled and experienced practitioners. With the diversification of models and scenes and the refinement of production, Art experience is directly related to the quality of finished products.

The traditional work method is indeed delicate and slow, but the ability to generate materials against the tedious steps has emerged! material generation capability brought to you today is simple and easy to use, rendering high-quality material effects, only needs 3 steps!

Taking the complex scenes required by the more common 3D games as an example, to render a material for a complex scene composed of multiple objects, only the following simple operations are required:

1. Use an RGB camera to take photos of the required materials.

2. Automatically generate material materials from RGB photos.

3. Material mapping and rendering is completed.

Huawei's 3D modeling service provides material generation capabilities that simplify and speed up the cumbersome material generation and rendering process, based on AI-assisted material generation, improve the efficiency of 3D model appearance creation, and solidify technical art experience and production specifications into materials Preset examples, reuse valuable experience and specifications, improve content production efficiency, and reduce material production costs.

Material generation ability

Provides the ability to convert RGB images to PBR materials. Through the integrated SDK, users only need to take one or more RGB images to generate 4 texture maps with one click, including diffuse map/normal map/specular map/roughness map. AI assists the generation of PBR materials, generates materials from photos, liberates manpower operation costs, and meets consumer-level 3D material applications.

Input picture file specification

• Software and hardware requirements: Ordinary RGB mobile phones, RGB-D\LiDAR is not required, all Android models
• Support type: concrete, marble, rock, gravel, brick, plaster, clay, metal, wood, bark, leather, fabric, paint, plastic, synthetic material, gravel, earth (grass, beach, etc.)
• Input image requirements: input image resolution of 1~4K, no seams, no bright spots, shadows, and reflections in the input image
• Output texture resolution: 1k (1024 1024px) 2k (2048 2048px)
• SDK package size: 88KB
• Accuracy: SSIM>0.9 after rendering

application scenario

The material generation capability of 3D modeling realizes one-click conversion of wood with various textures in reality into PBR material, which has a high degree of reduction and greatly improves the efficiency of scene construction. It is easy to use and effective, and it is widely used in the field of 3D content production.

• E-commerce industry : Use 3D models to display products and provide users with an almost real online shopping experience.
• Exhibition industry : High-value exhibits and cultural relics are displayed in the form of 3D models to facilitate users to carefully observe the details of the exhibits.
• Game industry : Wood needed for floors, tables, walls, etc. in indoor scenes to create an immersive gaming experience.

Development preparation

1. Integrate HMS Core SDK

1.1, add the AppGallery Connect profile of the current application

If you have activated related services in AppGallery Connect, you need to add the "agconnect-services.json" file to your App.

Step 1: Log in to the AppGallery Connect website and click "My Project".

Step 2: Find your project in the project list, and click the application that needs to integrate HMS Core SDK in the project.

Step 3: In the "Application" area of the "Project Settings> General" page, click "agconnect-services.json" to download the configuration file.

Step 4: Copy the "agconnect-services.json" file to the application-level root directory.

1.2, configure the Maven warehouse address of the HMS Core SDK

Step 1: Open the Android Studio project-level "build.gradle" file.

Step 2: Add HUAWEI agcp plugin and Maven code base.

• Configure the Maven repository address of the HMS Core SDK in "buildscript> repositories".
• Configure the Maven repository address of the HMS Core SDK in "allprojects> repositories".
• If the "agconnect-services.json" file is added to the App, you need to add the agcp plugin configuration in "buildscript> dependencies".

buildscript { 
    repositories { 
        google() 
        jcenter() 
        // 配置HMS Core SDK的Maven仓地址。 
        maven {url 'https://developer.huawei.com/repo/'} 
    } 
    dependencies { 
        ... 
        // 增加agcp插件配置。 
        classpath 'com.huawei.agconnect:agcp:1.4.2.300' 
    } 
} 
 
allprojects { 
    repositories { 
        google() 
        jcenter() 
        // 配置HMS Core SDK的Maven仓地址。 
        maven {url 'https://developer.huawei.com/repo/'} 
    } 
} 

Note: The Maven warehouse address cannot be opened and accessed directly in the browser, and can only be configured in the IDE. If you need to add multiple Maven code libraries, please configure Huawei's Maven warehouse address at the end.

1.3, add compilation dependency

Step 1: Open the application-level "build.gradle" file.

Step 2: Add the following compilation dependencies in "dependencies".

• Material generation

dependencies { 
    implementation 'com.huawei.hms:modeling3d-material-generate:{version}' 
}

• 3D object modeling

dependencies { 
    implementation 'com.huawei.hms:modeling3d-object-reconstruct:{version}' 
}

Note: {version} is replaced with the actual Kit dependency version. For the version number index, please refer to the 2 version update instructions. For example: implementation'com.huawei.hms:modeling3d-material-generate:1.0.0.300' and implementation'com.huawei.hms:modeling3d-object-reconstruct:1.0.0.300'.

Step 3: Add agcp plug-in configuration. Please choose according to the actual situation:

• Method 1: Add the following configuration in the next line of the file header declaration.

apply plugin: 'com.huawei.agconnect'

• Method 2: Add the following configuration in plugins.

  plugins { 
    id 'com.android.application' 
    // 添加如下配置 
    id 'com.huawei.agconnect' 
  }

2. Multi-language settings

• If your application does not need to be set to only support certain specific languages, please ignore this step. The application will support all languages supported by HMS Core SDK by default.
• If your application needs to be set to only support certain specific languages, you can configure it through this step.

a. Open the application-level "build.gradle" file.

b. Add "resConfigs" in "android> defaultConfig", configure the languages that need to be supported, and the configuration format is as follows:

android { 
    defaultConfig { 
        ... 
        resConfigs "en", "zh-rCN", "需要支持的其他语言" 
    } 
}  

For the list of languages supported by HMS Core SDK, see Languages Supported by HMS Core SDK.

3. Synchronous engineering

After completing the above configuration, click the gradle synchronization icon in the toolbar to complete the synchronization of the "build.gradle" file, and download the relevant dependencies to the local.

Note: If there is an error, please check whether the network connection is normal, and check whether the "build.gradle" file is correct.

4. Configure the obfuscated script

Before you compile the APK, you need to configure the obfuscated configuration file to avoid obfuscating the HMS Core SDK and causing malfunctions.

Step 1: Open the obfuscation configuration file "proguard-rules.pro" in the application-level root directory, and add the obfuscation configuration script to exclude HMS Core SDK.

-ignorewarnings 
-keepattributes *Annotation* 
-keepattributes Exceptions 
-keepattributes InnerClasses 
-keepattributes Signature 
-keepattributes SourceFile,LineNumberTable 
-keep class com.huawei.hianalytics.**{*;} 
-keep class com.huawei.updatesdk.**{*;} 
-keep class com.huawei.hms.**{*;}

Step 2: If you use AndResGuard, you need to add the AndResGuard permission list in the application-level "build.gradle" file.

"R.string.hms*", 
"R.string.connect_server_fail_prompt_toast", 
"R.string.getting_message_fail_prompt_toast", 
"R.string.no_available_network_prompt_toast", 
"R.string.third_app_*", 
"R.string.upsdk_*", 
"R.layout.hms*", 
"R.layout.upsdk_*", 
"R.drawable.upsdk*", 
"R.color.upsdk*", 
"R.dimen.upsdk*", 
"R.style.upsdk*",  
"R.string.agc*"

5. Add permissions

When invoking the material generation capability, the developer needs to apply for the following permissions in the AndroidManifest.xml file:

<!--  允许程序写入材质贴图文件和读取需要处理的数据 --> 
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> 
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> 
<!--  网络权限 数据上传和材质贴图下载使用 --> 
<uses-permission android:name="android.permission.INTERNET" />

When calling 3D object modeling capabilities, developers need to apply for the following permissions in the AndroidManifest.xml file:

<!--  允许程序写入模型文件和读取需要处理的数据 -->  
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />  
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />  
<!--  网络权限 数据上传和模型下载使用 --> 
<uses-permission android:name="android.permission.INTERNET" /> 

Development steps

Before starting the API development work, complete the development preparations and make sure that the Maven warehouse address of the HMS Core SDK has been configured in the project, and the SDK integration of this service has been completed.

1. To use the cloud-side service capabilities, you need to use the api_key value in "agconnect-services.json", and use api_key or AccessToken to set application authentication information during application initialization. AccessToken has a higher priority.

2. (Recommended) Set the AccessToken through the setAccessToken method, and initialize the setting once when the application starts, and there is no need to set it multiple times.

   MaterialGenApplication.getInstance().setAccessToken("your AccessToken");

   To obtain the Access Token from the api_key in "ageonnect-services.json", see based on OAuth 2.0 open authentication client mode.

• The api_key is set through the setApiKey method, which can be initialized once when the application starts, and there is no need to set it multiple times.

  MaterialGenApplication.getInstance().setApiKey("your api_key");

  When you register an app on AppGallery Connect, your app will be assigned an api_key.

2. Create a new material generation engine and material generation configurator and initialize the material generation engine.

   // 新建材质生成引擎，传入当前context。 
   Modeling3dTextureEngine engine = Modeling3dTextureEngine.getInstance(context); 
   // 新建材质生成配置器。 
   Modeling3dTextureSetting setting = new Modeling3dTextureSetting.Factory() 
   // 设置工作模式为AI模式。 
   .setTextureMode(Modeling3dTextureConstants.AlgorithmMode.AI) 
   .create();

3. Create a new listener callback, used to process the material to generate the upload result.

   Modeling3dTextureUploadListener uploadListener = new Modeling3dTextureUploadListener() { 
       public void onResult(String taskId, Modeling3dTextureUploadResult result, Object ext) { 
     // 获取材质生成上传图片结果。 
     if (result.isComplete()) { 
         // 上传结果处理。 
     } 
       } 
       @Override 
       public void onError(String taskId, int errorCode, String message) { 
     // 上传错误回调函数。 
       } 
       @Override 
       public void onUploadProgress(String taskId, double progress, Object ext) { 
     // 预留接口。 
       } 
   };

4. Upload the collected pictures to the cloud side.

   // 获取材质生成任务ID，传入配置器。 
   Modeling3dTextureInitResult modeling3dTextureInitResult = engine.initTask(setting); 
   String taskId = modeling3dTextureInitResult.getTaskId(); 
   if (taskId == null || taskId.equals("")) { 
       Log.e("", "get taskId error " + modeling3dTextureInitResult.getRetMsg()); 
   } else { 
       // 设置上传监听器。 
       engine.setTextureUploadListener(uploadListener); 
       // 异步上传，传入任务id和图片文件所在文件夹路径。 
       engine.asyncUploadFile(taskId, filePath); 
   }

5. Query the progress of cloud-side material generation.

   // 新建材质生成任务处理实例，传入当前context。 
   Modeling3dTextureTaskUtils taskUtils = Modeling3dTextureTaskUtils.getInstance(context); 
   // 查询材质生成进度。 
   Modeling3dTextureQueryResult queryResult = taskUtils.queryTask(taskId);

6. Create a new listener callback, used to process the material to generate the download result.

   Modeling3dTextureDownloadListener downloadListener = new Modeling3dTextureDownloadListener() { 
       public void onResult(String taskId, Modeling3dTextureDownloadResult result, Object ext) { 
     // 获取材质生成下载贴图结果通知。 
     if (result.isComplete()) { 
         // 下载结果处理。 
     } 
       } 
       @Override 
       public void onError(String taskId, int errorCode, String message) { 
     // 下载错误回调函数。 
       } 
       @Override 
       public void onDownloadProgress(String taskId, double progress, Object ext) { 
     // 预留接口。 
       } 
   };

7. Download the material to generate a map.

   // 设置下载监听器。 
   engine.setTextureDownloadListener(downloadListener); 
   // 下载材质贴图，传入任务id和保存路径。 
   engine.asyncDownloadTexture(taskId, savePath);

8. Call the material generation synchronization interface, you can get the generated texture in real time.

   // 同步接口，传入图片文件路径、保存贴图路径、配置器。 
   int result = engine.syncGenerateTexture(filePath, downloadPath, setting);

9. Delete the material generation task.

   int ret = taskUtils.deleteTask(taskId);

Learn more>>

Visit the official website of Huawei 3D Modeling Service

Huawei's 3D modeling development guidance document

Huawei HMS Core official forum

Huawei 3D modeling open source warehouse address: GitHub , Gitee

To solve integration problems, please go to Stack Overflow

Follow us and learn about the latest technical information of HMS Core