Use CameraX Extensions API to apply special effects to photos

Android CameraX is designed to help you simplify the development of camera applications. With the continuous development of CameraX, camera application developers have shown us their passion and enthusiasm. Many great ideas have been incorporated into the current API, such as the commendable CameraX Extensions API. Recently we adopted the opinions of the developer community and refactored the extensions. Now with the new ExtensionsManager , you can use these extensions with just two lines of code! This article will introduce how to use the Extensions API in your application.

CameraX Extensions

Android devices are equipped with powerful cameras, and manufacturers have invested a lot of energy to incorporate many cutting-edge features or special effects into these camera devices. In the past, these powerful functions could only be provided by the native camera application of the device. Today, with the CameraX Extensions API, third-party developers can access these powerful camera functions through a common and simple interface.

What's covered by CameraX Extensions

CameraX Extensions version 1.0.0 includes some of the most common built-in camera effects:

• BOKEH (Out-of-focus imaging) : When taking photos in portrait mode, make the foreground people clearer.
• HDR (High Dynamic Range) : Use different automatic exposure (AE) configurations when taking pictures to get the best results.
• NIGHT (Night) : Capture the best still images in a low-light environment (usually at night).
• FACE RETOUCH (Face Photo Restoration) : When shooting still images, modify facial skin tone, contours, etc.
• AUTO (Auto) : Automatically adjust the final image according to the surrounding scenery.

Let's take a look at a few sets of photos taken on an Android phone. The special effects provided by the CameraX Extensions API are enabled and disabled when the photos are taken.

BOKEH mode example

△ Picture 1: The photo on the right has BOKEH special effects enabled.

HDR mode example

△ Picture 2: The photo on the right has HDR special effects enabled.

NIGHT mode example

△ Picture 3: The NIGHT special effect is enabled on the photo on the right.

The visual difference is obvious. You can use the CameraX Extensions API to implement these image effects in your own applications.

Now let us see how to integrate CameraX's API into your application.

Extensions API

In the existing CameraX application, you can first introduce the camera-extensions Jetpack library to add CameraX Extensions:

dependencies {
    // 与 Extensions 库版本号相匹配的 CameraX 核心库
    implementation 'androidx.camera:camera-core:1.1.0-alpha08'
    implementation 'androidx.camera:camera-camera2:1.1.0-alpha08'
    implementation 'androidx.camera:camera-lifecycle:1.1.0-alpha08'

    // CameraX Extensions 库
    implementation 'androidx.camera:camera-extensions:1.0.0-alpha28'

    // 其他依赖项
    implementation('androidx.concurrent:concurrent-futures-ktx:1.1.0')
        …
}

Next, integrate Extensions through the following steps:

1. Get the ExtensionsManager instance.
2. Check whether the target device supports the extended mode that needs to be used;
3. Get a CameraSelector with extensions enabled;
4. Use the extended CameraSelector to call bindToLifecycle ).

Get ExtensionsManager instance

The first step is to use the getInstance(Context) ) API of the extension library to obtain an instance of ExtensionsManager This API returns a ListenableFuture, and we can use await() in the Kotlin suspend function to get the result to avoid blocking the main thread. (Note: Use await() on ListenableFuture to introduce androidx.concurrent:concurrent-futures-ktx: 1.1.0 dependency.)

// 创建扩展管理器（使用 Jetpack Concurrent 库）
val extensionsManager =  ExtensionsManager.getInstance(context).await()

Through ExtensionsManager, you can determine whether the device supports a particular extension mode, and get an extension-enabled CameraSelector . Please note the following points:

• ExtensionsManager is a process-wide global resource: only one ExtensionsManager instance exists in a process.
• ExtensionsManager always exists: CameraX provides a valid ExtensionsManager instance regardless of whether the underlying device supports extensions.

Check extended mode availability

Use ExtensionsManager to use the isExtensionAvailable(CameraProvider, CameraSelector, int) ) function to check the availability of the extension: If there are any cameras filtered by CameraSelector on the device that support the queried extension, it returns true, otherwise it returns false.

// 获取相机设备来检查是否支持扩展
val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

// 检查是否支持 BOKEH
 if (extensionsManager.isExtensionAvailable(
    cameraProvider,
    cameraSelector,
    ExtensionMode.BOKEH
   )) {
   ...
}

Get the extended CameraSelector

Once you have confirmed that the device supports this extended mode, you can use the getExtensionEnabledCameraSelector(CameraProvider, CameraSelector, int) ) function to get an extended CameraSelector. This function returns the extended-enabled CameraSelector, which contains all the detailed information about the specified extended mode.

val bokehCameraSelector = extensionsManager
                          .getExtensionEnabledCameraSelector(
                              cameraProvider, cameraSelector, ExtensionMode.BOKEH)

Use the extended CameraSelector to call bindToLifecycle()

The last step is to use bindToLifecycle() ) to bind your use case with the extension-enabled CameraSelector . Using the extension-enabled CameraSelector same as using a normal CameraSelector, for example using DEFAULT_BACK_CAMERA or DEFAULT_FRONT_CAMERA . When using the extended CameraSelector binding use case, CameraX will directly enable the specified extended mode on the camera. For example, when bound to Preview, the extended effect is applied to the preview, or to the image captured by the bound ImageCapture.

// 将开启了 BOKEH 的相机选择器绑定到用例上
val imageCapture = ImageCapture.Builder().build()
val preview = Preview.Builder().build()
cameraProvider.bindToLifecycle(
                lifecycleOwner,
                bokehCameraSelector,
                imageCapture,
                preview
            )

Sample code using Extensions API

The complete code of the Extensions API example is as follows:

fun onCreate() {
    lifecycleScope.launch {
        // 创建 cameraProvider
        val cameraProvider = ProcessCameraProvider.getInstance(context).await() 

        // 创建 extensionsManager（使用 Jetpack Concurrent 库）
        val extensionsManager = 
                ExtensionsManager.getInstance(context).await()

        // 获取相机设备来检查是否支持扩展        
        val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

        // 检查是否支持 BOKEH
        if (extensionsManager.isExtensionAvailable(
                cameraProvider,
                cameraSelector,
                ExtensionMode.BOKEH
            )) {
            // 在启用不同扩展模式之前解除所有用例的绑定
            cameraProvider.unbindAll()

            // 获取启用了 BOKEH 的相机选择器
            val bokehCameraSelector = extensionsManager
                    .getExtensionEnabledCameraSelector(
                cameraProvider,
                cameraSelector,
                ExtensionMode.BOKEH
            )

            // 将开启了 BOKEH 的相机选择器绑定到用例上
            val imageCapture = ImageCapture.Builder().build()
            val preview = Preview.Builder().build()
            cameraProvider.bindToLifecycle(
                lifecycleOwner,
                bokehCameraSelector,
                imageCapture,
                preview
            )
        }
    }
}

Extensions API depends on the core module

The CameraX Extensions API is camera-extensions library, and it depends on the CameraX core modules ( core, camera2 and lifecycle). When using CameraX Extensions, please be sure to use the version in the release package that is the For example, to use camera-extensions:1.0.0-alpha28 , you must include the 1.0.0-alpha08 version of camera-lifecycle , camera-core and camera-camera2 in the application's dependency list because they were released in the same package on August 18, 2021.

Supports extended devices

In order to use the CameraX Extensions API, device manufacturers need to implement CameraX Vendor Extensions interface . You can find a partial list of devices that support the CameraX Extensions API CameraX device page Please note that this is not an exhaustive list. If your device is listed but the availability check returns false, you may need to update your device to the latest ROM version of the manufacturer.

In addition to the list of devices that support extensions, starting from Android 12, you can also check the Android property ro.camerax.extensions.enabled to determine whether the device supports CameraX Extensions.

Remove the old version of Extensions API

The old Extensions API released in August 2019 is now obsolete. This old version of the Extensions API provides an extender class, which requires extension-related configuration to be applied to each Preview and ImageCapture use case. The old extender design may cause developers to forget to enable the extended mode on Preview or ImageCapture, and may cause unexpected behavior.

The new CameraX Extensions library was introduced in 1.0.0-alpha26. The newer Extensions API switches the extension binding from the use case to the target camera, making it more convenient to use. Be sure to migrate to take advantage of the new Extensions API.

We especially thank those excellent Android camera developers and device manufacturers who helped implement the CameraX Extensions API! If you want to know the latest development of CameraX, please join the Android CameraX discussion group .

more information

• CameraX Extensions API Guide
• Extensions API Reference
• Extended test application
• CameraX Release Notes
• Start using CameraX
• CameraX Github sample code

Welcome to click here to submit feedback to us, or share your favorite content or problems found. Your feedback is very important to us, thank you for your support!