Extensions API

CameraX provides an Extensions API to access the following special effects that manufacturers have implemented on various Android devices (Phones, Tablets, or others)

• Auto: Automatically adjusts the final image with the surrounding scenery. For example, the vendor library implementation might perform low light detection and can switch to low light mode or HDR mode to take a picture, or the library might automatically apply a face retouch mode when taking a portrait image. Auto mode leaves mode selection to the vendor library implementation.
• Bokeh: Bokeh mode makes the foreground character appear sharper and blurs the background of a photo. It is generally intended for taking portrait photos of people similar to what would be produced by a camera with a large lens.
• Face Retouch: Retouches face skin tone, geometry and so on when taking still images.
• HDR (High Dynamic Range): HDR mode takes photos that keep a larger range of scene illumination levels visible in the final image. For example, when taking a picture of an object in front of a bright window, both the object and the scene through the window may be visible when using HDR mode, while in normal mode, one or the other might be poorly exposed. As a tradeoff, HDR mode generally takes much longer to capture a single image, has no user control, and might have other artifacts depending on the HDR method used.
• Night: Gets the best still images under low-light situations, typically at night time.

For a device to support those vendor extensions, it must meet the following requirement:

• The effect has library support from the device ROM.
• The ROM library is installed on the current device.
• The device has a version of the operating system that the library requires.

You can enable an extension preferentially: if the extension is both supported by and physically on the device, then it will be enabled; otherwise, it will degrade gracefully.

Vendors aren't required to provide an implementation for every effect and feature. Any capability without a vendor-provided implementation defaults to the CameraX implementation. The default implementation reports that the capability is unavailable and skips enabling it.

Extensions architecture

The following image shows the architecture for extensions with CameraX.

Extensions are separate from the Camera2 core of CameraX. In the diagram, the red arrows indicate the main data flow when users trigger an extension-based feature, such as HDR image capture.

Enable an effect for image capture and preview

Before using the extensions API, retrieve an ExtensionsManager instance using the ExtensionsManager#getInstanceAsync(Context, CameraProvider) method. This will allow you to query the extension availability information. Then retrieve an extension enabled CameraSelector. The extension mode will be applied on the image capture and preview use cases when calling the bindToLifecycle() method with the CameraSelector extension enabled.

To implement the extension for the image capture and preview use cases, refer to the following code sample:

import androidx.camera.extensions.ExtensionMode
import androidx.camera.extensions.ExtensionsManager

fun onCreate() {
    // Create a camera provider
    val cameraProvider : ProcessCameraProvider = ... // Get the provider instance

    lifecycleScope.launch {
        // Create an extensions manager
        val extensionsManager =
                ExtensionsManager.getInstanceAsync(context, cameraProvider).await()

        // Select the camera
        val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

        // Query if extension is available.
        if (extensionsManager.isExtensionAvailable(
                cameraSelector,
                ExtensionMode.BOKEH
            )
        ) {
            // Unbind all use cases before enabling different extension modes.
            cameraProvider.unbindAll()

            // Retrieve extension enabled camera selector
            val bokehCameraSelector = extensionsManager.getExtensionEnabledCameraSelector(
                cameraSelector,
                ExtensionMode.BOKEH
            )

            // Bind image capture and preview use cases with the extension enabled camera selector.
            val imageCapture = ImageCapture.Builder().build()
            val preview = Preview.Builder().build()
            cameraProvider.bindToLifecycle(
                lifecycleOwner,
                bokehCameraSelector,
                imageCapture,
                preview
            )
        }
    }
}

import androidx.camera.extensions.ExtensionMode;
import androidx.camera.extensions.ExtensionsManager;

void onCreate() {
    // Create a camera provider
    ProcessCameraProvider cameraProvider = ... // Get the provider instance

    // Call the getInstanceAsync function to retrieve a ListenableFuture object
    ListenableFuture future =
            ExtensionsManager.getInstanceAsync(context, cameraProvider);

    // Obtain the ExtensionsManager instance from the returned ListenableFuture object
    future.addListener(() -> {
        try {
            ExtensionsManager extensionsManager = future.get();

            // Select the camera
            CameraSelector cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA;

            // Query if extension is available.
            if (extensionsManager.isExtensionAvailable(cameraSelector, ExtensionMode.BOKEH)) {
                // Unbind all use cases before enabling different extension modes.
                cameraProvider.unbindAll();

                // Retrieve extension enabled camera selector
                CameraSelector bokehCameraSelector = extensionsManager
                    .getExtensionEnabledCameraSelector(cameraSelector, ExtensionMode.BOKEH);

                // Bind image capture and preview use cases with the extension enabled camera selector.
                ImageCapture imageCapture = new ImageCapture.Builder().build();
                Preview preview = new Preview.Builder().build();
                cameraProvider.bindToLifecycle(lifecycleOwner, bokehCameraSelector, imageCapture, preview);
            }
        } catch (ExecutionException | InterruptedException e) {
            // This should not happen unless the future is cancelled or the thread is interrupted by
            // applications.
        }
    }, ContextCompact.getMainExecutor(context));
}

Disable the effect

To disable vendor extensions, unbind all use cases and rebind the image capture and preview use cases with a normal camera selector. For example, rebind to the back camera using CameraSelector.DEFAULT_BACK_CAMERA.

Dependencies

The CameraX Extensions API is implemented in the camera-extensions library. The extensions depend on the CameraX core modules (core, camera2, lifecycle).

dependencies {
  def camerax_version = "1.1.0-alpha08"
  implementation "androidx.camera:camera-core:${camerax_version}"
  implementation "androidx.camera:camera-camera2:${camerax_version}"
  implementation "androidx.camera:camera-lifecycle:${camerax_version}"
  //the CameraX Extensions library
  implementation "androidx.camera:camera-extensions:1.0.0-alpha28"
    ...
}

dependencies {
  val camerax_version = "1.1.0-alpha08"
  implementation("androidx.camera:camera-core:${camerax_version}")
  implementation("androidx.camera:camera-camera2:${camerax_version}")
  implementation("androidx.camera:camera-lifecycle:${camerax_version}")
  // the CameraX Extensions library
  implementation("androidx.camera:camera-extensions:1.0.0-alpha28")
    ...
}

Legacy API removal

With the new Extensions API released in 1.0.0-alpha26, the legacy Extensions API released in August 2019 is now deprecated. Starting with version 1.0.0-alpha28, the legacy Extensions API has been removed from the library. Applications using the new Extensions API must now acquire an extension-enabled CameraSelector and use it to bind the use cases.

Applications using the legacy Extensions API should migrate to the new Extensions API to ensure future compatibility with upcoming CameraX releases.

Additional resources

To learn more about CameraX, consult the following additional resources.

Codelab

Code sample

Other references

CameraX Vendor Extensions

CameraX Vendor Extensions Validation Tool