Updates for large-screen devices in Android and Chrome OS

With the continuous innovation of smart terminal hardware, there are more and more types of large-scale devices, such as mobile phones, folding screen devices, tablet computers, ChromeBooks, ChromeBox with external monitors, and Chromebase with integrated screens. The Google team is putting more R&D efforts into the Android framework, Jetpack, and Chrome OS.

△ Android 12L and Jetpack add new APIs and functions to make your APP look more beautiful and more powerful.

Read on for updates on Android and Chrome OS support for large-screen devices!

If you prefer to see this through video, check it out here:

https://www.bilibili.com/video/BV1aL4y1b7nt/?aid=850572148&cid=482028390&page=1

△ Android and Chrome OS updates for large-screen devices

Android 12L

As shown in the figure below, it can be found that users' demand for larger screen space is growing. In 2020 alone, the sales of Android tablets will increase by 100 million units, and the Chrome operating system will increase by more than 92%. There are currently more than 250 million large-screen Android devices in use, so it is necessary for applications to adapt accordingly for such devices. In order to adapt to the growing number of devices and user needs, we have launched Android 12L (hereinafter referred to as 12L) for large-screen devices.

△ Large-screen devices are gradually becoming mainstream 100 million new Android tablets Source: Q2 2021: IDC Single Quarter Personal Computing Device Tracker

We've been working closely with developers to keep abreast of their needs for big-screen development and what's happening in real-time from upstream device manufacturers, improving features and APIs in parallel, and these updates will land in early 2022 to make Android 12 better run on these big screen devices. The 12L includes several developer-specific optimizations, including better multitasking, a redesigned look to make the most of screen real estate, and the addition of a compatibility mode to ensure it works well on phones with smaller screens.

Multitasking

Multitasking has become a daily practice since Android 12, and all apps can run in multi-window mode. But be aware that an app may run in split-screen mode or appear as a window next to another app.

Pay particular attention in the following scenarios:

1. Render interface elements by yourself or require a specific window size;
2. Apps need access to exclusive hardware devices, such as cameras and microphones.

Multi-window mode

△ Multi-window support is easier to access than before

To support multitasking, Android 12L has an updated interface, including an improved taskbar for app switching. We all know that the operation for users to enter the split-screen mode was complicated in the past. The new taskbar simplifies quick switching between apps and makes it easy to return to the home screen.

Navigation buttons

△ Three-button navigation is easier to access than before

On devices with larger screens, the taskbar makes it easy to turn apps into split-screen mode or multi-window mode. This is because all apps can run in split-screen mode or in a separate window regardless of whether they declare a size switchable or not, so it is necessary to update your app to accommodate the size change, while avoiding app restarts or entering compatibility mode. The taskbar also moves the three-button navigation bar to the side of the screen, making it easier for users to operate on large-screen devices.

System interface

△ System interface - modern appearance and texture

Android 12L also brings several UI updates related to the system interface. This includes optimizing the home screen layout, significantly tweaking the look and feel of notifications, and adding pop-ups to make PIN entry easier. You don't need to take any action to automatically adopt the new system look in your app. In addition, we're updating system apps like Settings, Setup Wizard, and more to make better use of screen space.

Improved taskbar

△ The taskbar after the optimized experience - for a better application switching experience

In order to improve the experience of app switching, we have optimized the taskbar. Users can quickly switch applications and return to the home screen. On devices with larger screens, the taskbar can drag apps into split-screen and multi-window modes.

Compatibility Mode

△ Compatibility Mode - stability and visual improvements

If your app is locked in landscape or portrait mode and cannot be resized, the app can be displayed in compatibility mode when the user enters split screen, opens a folded device, or in a multi-window environment like ChromeOS. This ensures that your app looks and functions as originally intended.

We are updating the style and rendering effects of the function and compatibility mode, although these can allow users to continue to use those applications that cannot be changed in size, and can also fit the system, but still can not provide the ideal user experience, and whether to do the application The decision to optimize is in your hands.

Play Store Update

△ Play Store update — showcasing apps for big screens

We've also made some improvements to the Play Store to help users find the best apps for the big screen.

First, we're separating out rating and review features for apps on large screen devices; second, we're optimizing our QA process for the app's variable-size functionality and layout on large screens; , as well as other features for larger screens. These paths allow us to pick and choose apps that provide the best experience for users on their devices. We plan to roll out these improvements in 2022, when you can learn more about this section.

For more information, please refer to the following articles:

• large screen applications Quality Guidelines
• better ratings and reviews for users and developers
• Meet the needs of developers with a better business model

Foldable screen emulator for Android 12L

△ Android 12L/foldable emulator developer.android.google.cn/12L

The 12L feature launch includes a new API and a new SDK level, namely level 32, which is convenient for you to quickly locate new features. Note that the Play Store increases the target SDK requirements every year, only for Android 12, which is SDK 31, and does not force you to upgrade to 32.

Start testing these new features and APIs today and get Android 12L Developer Preview to get your apps ready for the upcoming public release. You can also Android Studio emulator to try out split screen, fold and rotate events. Make sure your app looks good throughout the user journey.

Soon everyone will be able to test these features with Lenovo P12 Pro, Samsung Galaxy Z Fold 3. The 12L feature update is exciting, and we also look forward to adding richer features and more support for large-screen devices in future Android versions. We will continue to work hard to make Android a better operating system for users and developers. Provide better service.

Jetpack WindowManager

Jetpack WindowManager is a library that provides a backwards-compatible API that adapts to window specifications and dimensions, as well as provides details about display features such as folding, hinges, and device configuration. This article will explain the stable APIs available in the library, as well as some new experimental tools in current and future releases to make your app look great on the big screen. The library uses the latest features of 12L, but is also compatible with previous platform versions down to API level 14.

WindowMetrics

△ Build Android interface for any screen size

First introduce WindowMetrics. Android 11 introduces a new set of WindowManager APIs that can give accurate measurements of the current running window of an application. On large-screen devices, your app will most likely not occupy the entire screen due to the increased use of split-screen and other multi-window formats. In the Samsung Galaxy Z Fold series of mobile phones, we found that its split screen usage rate is up to seven times that of other mobile phones. Another example is windows with black borders when large screen phones are in different orientations. It is very important to make the application fully variable in size, and we will discuss this topic at length.

So how to determine the size of the Activity?

class MyActivity : Activity() {

        override fun onConfigurationChanged(newConfig: Configuration) {
            super.onConfigurationChanged(newConfig)
            val windowMetrics = WindowMetricsCalculator.getOrCreate()
                .computeCurrentWindowMetrics(this)
        }
}

The WindowMetrics API returns the exact pixel dimensions of a window in all possible states, and is backward compatible up to SDK level 14. If the user expands the display of the app, it also prompts you for a configurable maximum size so that the developer can choose the appropriate resource to load ahead of time. Remember that WindowMetrics can be changed at runtime, so the recommended time to update the value is when the Activity is initially created and when the configuration is changed using the WindowMetricsCalculator. Please do not use deprecated display-related APIs such as "getRealMetrics" or "getRealSize", otherwise you may get abnormal size values.

WindowSizeClasses

Another key element of being able to visually present an app to users on all device types is providing different layouts. We've heard from everyone's feedback that it can be very difficult to have a clear idea of which screen size to develop for in a complex ecosystem of devices.

△ Window size class in Jetpack WindowManager

Now Jetpack has added the WindowSize class to make this difficult. This feature introduces a unique layout breakpoint that can help you understand how to adapt the interface. The WindowSize class is used when you need to select the appropriate layout for different device types or when you need to respond to window changes in multi-window mode.

Before, in portrait mode, users only operated one app most of the time, but tablets are usually in landscape mode. With foldable devices and different multi-window modes, applications often need to make windows larger or smaller in a single session. So it needs to satisfy as many scenarios as possible. If you'd like to learn about WindowManager Jetpack 1.1 features, check out our tweet " Building an Android interface for any screen size ".

Foldable screen

△ Foldable screen

There is a lot of potential for innovation, especially for foldable devices. As the number of such devices on the market grows, you can go a step further and make your app not only compatible with larger screens, but also adapt to different states of the device when the user folds or unfolds the device while the app is running.

Supporting different modes can bring new interaction methods and new user experiences to the application. For example, today's foldable devices are often placed on a desktop, ideal for watching videos or answering hands-free calls. The device is placed in such a way that one part of the screen is at a comfortable viewing angle and the other part of the screen rests on a level surface, making it ideal for a variety of interactive elements.

FoldingFeature

△ FoldingFeature

Let's see how the API library can help you. You can use FoldingFeature to determine the pose of the device. This class is used to monitor the state of the foldable device and update the surrounding interface if necessary using the feature type, screen orientation, and state update interface.

val windowInfoRepo = windowInfoRepository()
lifecycleScope.launch(Dispatchers.Main) {
    lifecycle.repeatOnLifecycle(Lifecycle.State.STARTED) {
        windowInfoRepo.windowLayoutInfo
            .collect { newLayoutInfo ->
                updateCurrentState(newLayoutInfo)
            }
     }
}

There are two possible states: open and half open. In the flat-open state, the screen is fully expanded into a flat surface, but in some cases the screen is still divided by the hinge instead of being a continuous whole. In the half-open state, the window always contains at least two logical areas. Feature layout information is provided through the WindowInfoRepository. To start or stop listening for events, you can use lifecycle scope to track when the activity is visible. Later, you can update the app layout with the information available in the windowLayoutInfo object.

private fun isTabletopPosture(foldFeature: FoldingFeature) =
    foldFeature.state == FoldingFeature.State.HALF_OPENED &&
        foldFeature.orientation == FoldingFeature.Orientation.HORIZONTAL

FoldingFeature contains information such as hinge screen orientation and state. You can use these values to determine whether the device is in desktop mode or half-open with hinges flat. For more details, check out the documentation for Complete Guide to Folding Device Construction .

Test WindowManager

To keep these new layouts simple and easy to use in the long run, we've also added a new testing API to JetpackWindowManager. A specialized window testing module is also introduced in the library.

val testFeature = FoldingFeature(
   activity: Activity,
   center: Int = -1,
   size: Int = 0,
   state: State = HALF_OPENED,
   orientation: Orientation = VERTICAL,
)

val expected = WindowLayoutInfo(listOf(testFeature))

This example shows the creation of test instances related to the main class. Note that Activity is the only parameter of the FoldingFeature function without a default value. The default configuration of the current testing FoldingFeature is that the horizontal layout in the middle of the screen is half-open.

private val activityRule = ActivityScenarioRule(TestActivity:class.java)
private val publisherRule = WindowLayoutInfoPublisherRule()

@get:Rule
public val testRule: TestRule

init {
  testRule = RuleChain.outerRule(publisherRule).around(activityRule)
}

The WindowLayoutInfoPublisherRule can be used to test the interface and how the interface responds to FoldingFeatures. It can be associated to ActivityScenarioRule. But applying the listing rules is not a complete substitute for end-to-end testing on the device. For example, a real device might update the screen orientation window layout information. But if you use publisherRule, you must update the window size and window layout information yourself.

@Test
public fun testMyFeature() {
  val feature = WindowLayoutInfo(emptyList())
  publisherRule.overrideWindowLayoutInfo(feature)

  activityRule.scenario.onActivity { activity ->
     //应用相关测试
  }
}

After the test rule is set up, you can use the overrideWindowLayoutInfo method to replace the current layout information object.

Activity nested

We recognize that converting an existing legacy codebase to support large screens can be difficult. For applications that have long been developed and used activities for a single screen, switching to a multi-pane layout through Fragments or other tools may require significant refactoring and consume a lot of team resources. To make the transition easier, we introduced ActivityEmbedding , a WindowManager Jetpack feature set that can be used to flexibly organize Activity windows on current mainstream large-screen devices.

Activity shown side by side

△ Activity embedding in Jetpack WindowManager

Its initial interface implementation focused on making the most of large screen real estate by displaying activities side-by-side in a multi-column layout. For example, you can display these lists and details in a separate Activity, but you might want to display these on a larger screen. While I recommend that you refactor your application as a single Activity, it is understandably expensive to do so. This feature allows you to leverage your existing app structure to optimize large screen layouts. And what's most exciting is that adopting this feature requires only minor code adjustments, and in some cases no code adjustments.

△ Small and large screens

Let's take a closer look. This feature was designed with compatibility in mind. Based on the available screen space and the settings you provide, the library can automatically choose the appropriate display type, avoiding the need to fork in-app navigation code to handle large and small screens in different sections. The library also supports screen and window size changes at runtime, and if the user folds or unfolds the device or re-sizes the window in multi-window mode, the presentation will automatically update without any additional action on your part.

Activity stack

△ Activity stack

We also follow the existing ordering of activities in the application, identifying the primary and secondary, two containers, or activity stacks in each partition. The secondary container is always on top of the primary container. If the screen space is small, the activity stack is the same as usual; but if there is enough space, the two stacks can be displayed side by side.

The following example shows what happens if there are multiple activities in the secondary container?

△ Activity stack

They automatically appear within the same boundaries at startup. Existing Activity launch and expected resolution rules also apply.

△ Multiple depth levels

The library also supports multi-level navigation, creating multiple tiles and displaying up to two panes. When a new pane is opened, the previously created pane is moved offscreen. In this example, if the existing tiles show Activities A and B, and you need to display the new Activity C on one side, a second tile for B and C is created. Again, the Z-Order of the container is still considered to be at the top.

△ Screen size changes

Such an order means that the user closes the foldable device, and you can resize and reposition the container while continuing to use the app to maintain the active order. The top activity in the side stack expands automatically, but can be displayed side by side again at any time if the user expands the device.

△ placeholder

This is a different use case that we call "placeholders". Sometimes apps display a top-level navigation list on the home page, with no secondary content to display until the user makes a selection. However, in order to take full advantage of the available space, and for consistency reasons, chunks should be displayed as soon as the app is opened, with most of the auxiliary content left blank. At the same time, if the app is opened on a smaller screen, and after the device is folded, we don't want to show a blank page at the top.

We have added a special option to the library to support the use case of placeholders, let's take a look at how to integrate this feature in the application.

block rules

<SplitPairRule>
     <SplitPairFilter
         window:primaryActivityName=”.ActivityA”
         window:secondaryActivityName=”.AcitvityB” />
</SplitPairRule>

First, declare the library dependencies in the build file; then, create an XML file in assets and provide rule definitions: which activities should be chunked, and the properties of the chunks.

In this example, when B is opened after A, I want to put Activity A and B into a block. To accomplish this, I added a separate chunking pairing rule using the default configuration, and a filter to match the Activity component name. After Activity B is launched from A, the filter is checked and matched, and the library automatically creates a new chunk.

We provide different types of rules for different scenarios, giving you some flexibility. You can find more information on this in our documentation, and you can also use the runtime API to add or remove rules on demand.

configure

Next, we need to inform the library of the rule definition. In this example, we are using the Jetpack AppStartup library. In this example, we use the Jetpack AppStartup library to initialize before other upload components and activities start. To achieve this, we need to add the library dependencies in the build file and add the following entry in the AndroidManifest. Here we specify the initializer class to use.

<provider
    android:name=”androidx.startup.InitializationProvider”
    android:authorities=”${applicationId}.androidx-startup”
    android:exported=”false”
    tools:node=”merge”>
        <meta-data
            android:name=”example.SplitInitializer”
            android:value=”androidx.startup” />
</provider>

Finally, add the implementation of the initializer class. If you already use AppStartup in your application, this structure should be familiar.

class ExampleWindowInitializer : Initializer<SplitController> {
    override fun create(context: Context): SplitController {
        SplitController.initialize(context, R.xml.main_split_config)
        return SplitController.getInstance(context)
    }

    override fun dependencies(): List<Class<out Initializer<*>>> {
        return emptyList()
    }
}

To set up the rules, provide the XML resource ID and definition to SplitController.initialize.

You're done! The library will keep track of the activities launched at different places in your codebase, check the intents used and the activities that launched those intents, and if a matching rule is found, a new chunk will be created and managed by the library.

This feature is available on large screen devices with the latest version of the WindowManager Jetpack extension.

It's available now in the 12L emulator, and will soon be found on real devices like the Samsung GalaxyZ Fold 3. On devices that do not support this feature, the display will still be the same as before, and the activities will still be displayed on top of each other, completely covering each other, so there is no need to worry about display exceptions on devices that are not yet supported.

If you need to know if this functionality is available, a dedicated runtime API is available. We're also experimenting with other ways of interacting with multi-display devices. For the specific implementation code, please refer to WindowManager Jetpack Demo .

rear side screen display mode

△ Rear screen display mode

A cool example is the rear screen display mode that displays a selfie preview while taking a selfie with the high-quality main camera when the device is unfolded. We are developing a set of APIs to support this use case. You can find these features in a future release of the experimental WindowManager Jetpack API.

Chrome OS

△ Chrome OS optimization

For years, Chrome OS has allowed users to install and run Android apps on large-screen devices.

△ PIP

Recently, we've made several improvements to the Android app experience, such as improving picture-in-picture support, adding a low-latency stylus library, and beautifying the interface of apps that weren't designed for large-screen devices. Picture-in-picture now looks better and runs smoother in Chrome OS. Use the standard Android Picture-in-Picture API to get the latest look and functionality at no additional cost.

Let's take a quick look at these APIs next. First, indicate picture-in-picture support in the manifest file:

android:supportsPictureInPicture=”true”

Then start it via enterPictureInPictureMode:

pipButton.setOnClickListener {
    enterPictureInPictureMode(
      new PictureInPictureParams.Builder().build())
}

You can also use onUserLeaveHint to implement functions such as Activity automatically entering picture-in-picture mode when it is placed in the background by the user:

override fun onUserLeaveHint() {
    enterPictureInPictureMode(
      new PictureInPictureParams.Builder().build())
}

Note that interface adjustments may be required in this mode. To make adjustments, find the onPictureInPictureModeChanged callback and modify accordingly:

override fun onPictureInPictureModeChanged(
         isInPictureInPictureMode: Boolean,
         newConfig: Configuration) {
      if (isInPictureInPictureMode) {
          // 画中画功能实现
      } else {
          // 返回常规界面
      }
 }

In May 2021 we announced the Chrome OS Low Latency Stylus API to minimize latency for drawing applications. You can check out the API and demo app now GitHub

Compatibility Mode

△ Compatibility mode

When running an app designed only for a small Android phone in portrait orientation on a large-screen tablet Chromebook or an external monitor, the app's appearance and performance may be unsatisfactory when stretched into full-screen view. Apps can experience a variety of issues, including poor layout, and app crashes that don't handle multi-window or resizing events properly. Like tablets and foldables, Chrome OS now has a compatibility mode where apps designed for small mobile devices can be displayed in phone-sized or tablet-sized windows.

The user can easily change the display mode of the window or enable the window free resize mode on demand, but the interface informs the user that running the app in full large-screen mode may not work as expected. This helps Chrome OS deliver the desired performance and stability, while still allowing users the freedom to interact with apps the way they like.

Ideally your app should not appear in compatibility mode. Next, let’s talk about a few important things to do to avoid apps appearing in compatibility mode in Chrome OS and Android tablets and foldables:

△ Run in open form mode to make full use of screen space

1. Provide suitable large screen layouts for different device types.
2. Test the app to make sure it can handle collapse events, rotate, move into split screen, and freely resize. Jetpack components like ViewModel simplifies maintaining state and provides users with expected results. Be sure to test different layout possibilities on a real device or simulator.
3. Appropriately handle touch, keyboard, mouse, trackpad input as well as more specialized input methods such as stylus, game controller, etc., according to application needs.

For a more in-depth look, head over to our Android Developer Summit to learn how to make layouts look better, fit better, and handle input correctly. Let us know at if your app already looks great on Chrome OS or if you want to know where to start optimizing. For detailed documentation on optimizing for Chrome OS and larger screens, visit chromeos.dev .

Folding devices, tablets, and Chromebooks are growing in popularity. There is still a lot that can be done in app optimization, and we will continue to work hard to introduce new features to further improve the performance of the app.

summary

I believe that everyone has a certain understanding of some new features of 12L for large screen and application design. Welcome to explore the new APIs in 12L by downloading the 12L developer preview

You can try to use JetpackWindowManager optimize the application, adjust the display area according to the precise window size, enable new functions such as attitude detection. In the new version, Activity and test API to simplify the maintenance of large screen layout.

Don't forget to include a great-looking layout for the big screen, and add keyboard, mouse, and other input support. I hope everyone can successfully complete the application transformation for large-screen devices to embrace the new terminal interaction mode.

You are welcome to to submit feedback to us, or share your favorite content and found problems. Your feedback is very important to us, thank you for your support!