Design specifications for applications on folding screens

Android's reach has grown and the experience has gotten better, with more than 250 million large-screen devices now powered by Android, including tablets, foldables, and Chrome OS devices. How to adapt to different screen sizes and ensure a good experience has always been a major problem for developers. Especially with the emergence of emerging products such as foldable devices, adaptation work is becoming more and more urgent. This article will highlight relevant updates from the Material Design guidelines, and provide some advice to help developers build apps according to the principles of adaptive UI to address adaptation issues on tablets and foldable devices.

This article will focus on adaptation in the View system, for more information on how to use Compose to build large-screen applications, see the article " Building an Android Interface for Any Screen Size ".

If you prefer to understand the content of this article through video, please click below:

https://www.bilibili.com/video/BV1S34y1y7b7/?aid=808818730&cid=503331389&page=1

△ Application design specification on folding screen

Design Guidelines

In early 2021, we published the guidance document for large-screen devices on the Material Design website. Android Dev Summit we made some updates to help developers prepare for foldables and more.

-depth understanding of layout

in-depth understanding of layout guide introduces the concept of layout containers, which provides an overall framework to help developers think about how to arrange interface elements such as navigation bars, toolbars, and content on the screen.

△ Three main areas of the layout

The combined section of the guide walks you through how to make the most of screen space for readability, and to arrange important content and action options in different scenarios in a way that respects the user's mental model. Include proper scaling to show more content, such as subtitles and dates in the example, and smaller composition techniques, such as visually grouping content and keeping it relevant in a compact layout.

△ Part of the layout method covered in the combination guide

Take the Fortnightly sample app, which has a well-balanced layout on a tablet, thanks to its adherence to the guidelines for containers. And you can see that Fortnightly uses a Visual Divider to separate the latest news, and on the other side of the screen, it uses whitespace and typography to group news stories in different categories.

△ Fortnightly follows guidelines to separate and group content

Grid system

Many apps now treat the screen as a large canvas or single column, drawing elements horizontally and vertically in relation to each other, and some apps also have an overall margin on one side. This approach may work on small screens, but there are obvious problems when the screen size is larger. A grid system divides your layout into a series of columns, helping you design more expressive layouts in a canonical grid. Using a column grid (as shown in the figure below) in the layout can make the experience of large-screen devices present a more intimate and organized impression, making the device and content more naturally integrated.

△ Column grid

These columns allow you to improve your information hierarchy by dividing the screen into areas that hold related information and actions. As shown in the figure below, there are three areas, which will draw the user's attention to the main information segments or groups of information on the screen in the order in which the designer expects the user to read. Most importantly, column grids provide a logical way to think about how to reflow content as screen sizes grow or shrink, helping you to respond consistently across screen sizes.

△ Use a column grid to divide the screen into three main areas

In this example, the three main areas have been rearranged to maintain the same information hierarchy, but displayed on the small screen in a more user-friendly way.

△ Use column grid to reflow content on different screen sizes

Remembering the grid system helps you choose component behavior, in different layouts, and decide to replace or change components in a way that makes the most sense for the device size and scene. For example, on large screen devices, you can use Navigation rail (left sidebar navigation bar) instead of Bottom navigation (Bottom navigation) , both have the same function and look similar, but the Navigation rail can be more user-friendly layout the pages. Full-screen dialogs on mobile phones can be replaced by Simple dialogs on large screens to maintain the context of the user's current actions.

△ Use simple dialogs (right) instead of full-screen dialogs (left) on large screens

Size Class

Remember, when replacing components, the functional and user-friendly needs of the user must be met first. Finding the right threshold for tuning your interface is an important step in implementing a responsive interface. So we define new breakpoint values, which help to group devices into preset size categories that represent the actual device sizes on the market. They help convert the original size of the app layout into discrete standardized groups from which you can make higher-level interface decisions. For example, almost all standard mobile phones use a combination of a small (Compact) width and a medium (Medium) height in portrait mode, and due to the common use of vertical scrolling, for most applications, the size class according to the width is adapted is enough.

△ Width-based size categories

△ Height-based size class

These dimension classes will appear in the 1.1 Jetpack Window Manager library as a new API. Starting from Android Studio Bumblebee , we have also integrated size categories into the tool in the form of reference devices, and implementing the interface on this basis is conducive to maintaining consistency and making operations easier. And developers don't need to check the actual physical size or screen orientation, or other error-prone flags. As you design and build different size categories, think about how people will hold and touch the devices that these categories represent. Focusing on the shape and size of your device can help you create a more user-friendly experience. For example, on a tablet or large phone, it can be difficult for people to reach the top area of the screen without fully adjusting the grip, so keep important actions and content in easy-to-reach areas.

canonical layout

Canonical layouts provide a set of common layout schemes that are very helpful in designing large-screen applications. The first is a simple combination of list/details, or a list grid view, with a navigation container set/unset on the start side of the screen where the content begins.

△ List/detail layout

Support panels can be used in experiences where people need to focus, such as documentation. Add a panel to the end or bottom of the screen for easy access to tools or contextual controls.

△ Support panel

Feeds are a common pattern in news or social apps, and templates take the form of tiles to entice users to discover more content. The interaction is the same as on a mobile phone - opening an item opens a new page, but the experience is more immersive and designed for large screen sizes.

△ Information flow

The homepage banner prioritizes content at the top of the screen, with supporting elements designed around and below the content, which is a great experience for media-centric apps.

△ Homepage Banner

Specification Layout Practice

Adopting a responsive interface is not just about providing a parallel structure for different screen sizes, but the application needs to be flexible enough to be sized for various needs, such as rotating the device, multi-window mode, and folded and unfolded poses. So during runtime, the application can transition from one size class to another and back again. It's important not to think of size classes as completely separate buckets, and the app also needs to guarantee continuity (i.e. without disrupting the user experience), so app state or data cannot be lost.

△ Responsive interface can adjust content layout according to screen size changes

Imagine, when you resize the browser window, if the browser rewinds a page, or redirects to another page, or modifies the history, the experience is very strange. Therefore, each page should be flexible enough and should be able to maintain state during size transitions, where canonical layouts play an important role. For each page, think about what you can add as the screen size increases. What can be deleted when the screen size gets smaller. Then choose the appropriate strategy. This may mean you need to revisit the navigation map, especially if your current design is mobile-centric.

To build a responsive interface, we should prioritize the placement of persistent elements in the interface, such as navigation elements. Following the Material guidelines, we can provide alternative layouts based on the width's size class, adjusting the navigation to the most convenient location. For example, the bottom navigation view is used for small screens, the Navigation rail is used for medium screens, and the full navigation view is used for large screens. Note that these layouts use the width qualifier "-w" instead of the minimum width qualifier "-sw". The remaining space is used to arrange content, and we can apply canonical layouts in these spaces.

list/details

For list/details, there is a special control named SlidingPaneLayout in AndroidX. Before using it, you need to specify layout_width for its two child elements. During operation, SlidingPaneLayout will judge whether there is enough space to display two panes at the same time:

<SlidingPaneLayout …>
      <FragmentCOntainerView
              android : id=”@+id/list_pane”
              android : layout_width=”300dp”
              android : layout_weight=”1”
              …  />

      <FragmentCOntainerView
              android : id=”@+id/detail_pane”
              android : layout_width=”360dp”
              android : layout_weight=”2”

<SlidingPaneLayout …>

△ SlidingPaneLayout layout example

When the screen space is sufficient, the two panes must reach at least the specified width, and the remaining space can be allocated by layout_weight, as shown on the left; if there is insufficient space, as shown on the right, each pane uses the parent view , the details pane will be slid to the side, or just cover the first pane.

△ Space allocation results in SlidingPaneLayout

viewModel.selectedItemFlow.collect { item ->
// 更新详情窗格的内容
detailPane.showItem(item)
// 将详细信息窗格滑动到视图中
// 如果并排放置两个窗格
// 并不会产生实际效果
slidingPaneLayout.openPane()
}

As shown in the above code, you can control the sliding pane through code, when the user selects an item from the list, we receive the item from the Kotlin stream of the ViewModel, and then update the content of the details pane and slide it by calling openPane into view. In the Trackr application , the effect is shown in the following figure:

For related content on how to use SlidingPaneLayout to implement a dual-pane layout, please refer to the Android developer website: Creating a dual-pane layout , this page also introduces other content, such as integrating the system back button to implement a side-sliding back pane, etc. .

stream

We can display a dataset immersively through the information flow, so RecyclerView is a very suitable choice, we can change its presentation by changing the RecyclerView used by LayoutManager . LinearLayoutManager suitable for smaller widths, but in medium and expanded width scenarios, the page content will be overstretched and deformed. In this case, it may be more suitable to use GridLayoutManager , or StaggeredGridLayoutManager or even FlexBoxLayoutManager .

△ Change the presentation form of RecyclerView by replacing its LayoutManager

Homepage Banner

We can also change the layout of individual items so that some items are taller or wider than others to accentuate their importance and create a more interesting visual effect. In the homepage banner layout, we emphasize a particular element and rearrange other supporting elements around it. Of course we have many ways to do this, but ConstraintLayout is the most flexible because it provides many ways to constrain the size of child elements and their position relative to other child elements. In the following media example application, its first image is limited to a 16:9 aspect ratio, the description pane occupies 60% of the width, and the remaining space is reserved for other elements. Constraints can be changed and even animated with MotionLayout , which is a special ConstraintLayout .

△ Homepage banner example

For backing panels, any layout control from LinearLayout to ConstraintLayout can be used as a container to position the panel. As shown in the image below, we consider one thing, where should the content on the panel be placed when transitioning to small screen sizes. We have many options, such as using a side navigation drawer at the end of the screen, or using a slide-up bottom action bar, or using the options menu, or even hiding the content completely.

foldable devices

Foldable devices are not only equipped with larger screens, they can also adjust the orientation/posture of the device depending on how the device is folded and how the user is using it.

There are currently three common device forms: folded, unfolded, and desktop mode (hover). Also, we'll see other theoretically existing states later, such as book mode.

△ Three common postures of folding equipment

As with other large-screen devices, we need to think more about how users will hold an unfolded device? For example, on a tablet, some areas of the screen are difficult to reach with thumbs, and it is difficult for users to free up their entire hand to manipulate the screen freely. Users can easily reach the bottom corners of the screen, but may not reach the very top of the screen, especially in portrait mode. This means that if you use a component like the Navigation rail, centering or pinning the navigation buttons to the bottom of the screen makes it easier for users to navigate.

△ User operation hotspot in large-screen devices

At the same time, we also need to consider the influence of the hinge position on the interaction. The hinge will bring a noticeable tactile difference, and there will even be a physical separation of the two screens. Therefore, please avoid placing buttons and other important operating items directly on the hinge area. The hinge area is about 48 dp wide on most devices, and also avoid placing interface elements in the hinge area in desktop mode, as in this device mode the user has virtually no access to any functionality in this area.

△ Hinge area

There are two main technical scenarios for designing layouts when a device transitions from folded to unfolded mode. The first is to expand the screen, which uses a simple responsive layout where the app expands the content and fills the screen. Normally we would extend the column grid according to the Material guideline mentioned earlier.

The second is to add another page, which, depending on the app you're building, can take the same approach as the list/details or supplement the main panel with another panel.

△ Scenario 1: Enlarge the screen (left) Scenario 2: Add pages (right)

In both cases, according to material.io's guidelines, you need to create an eight-column grid evenly spaced on both sides of the hinge area, and when adding a navigation container such as the Navigation rail, the start side of the screen is compressed to accommodate the navigation container.

△ Eight-column grid evenly distributed on both sides of the hinge (blue background)

adaptation example

Now let's see how to take advantage of the collapsed state during runtime. Jetpack Window Manager library provides the corresponding API to detect whether the application window is folded. Any Activity can get an instance of WindowInfoRepository . Then, between the two lifecycle states, Started and Stopped, we can safely gather information from the window layout information flow. Whenever the stream emits a value, we can check the displayFeature and then look for the FoldingFeature in a targeted manner.

override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        val windowInfoRepo = windowInfoRepository()

        // 在 STARTED 和 STOPPED 这两种生命周期状态之间安全地从 windowInfoRepo 中收集数据
        lifecycleScope.launch(Dispatchers.Main) {
            lifecycle.repeatOnLifecycle(Lifecycle.State.STARTED) {
                windowInfoRepo.windowLayoutInfo.collect { info ->
                    for (feature in info.displayFeatures) {
                        val fold = feature as? FoldingFeature ?: continue
                        // 使用 FoldingFeature
                    }
                }
            }
        }
    }

△ Recognize folding posture

After mastering the information about the folding posture, we can use some methods to check whether the device is in one of the above-mentioned postures. In book mode, the state of the device is HALF_OPENED , and its orientation is VERTICAL ; in desktop mode, the state is HALF_OPENED , and its orientation is HORIZONTAL .

// 书本模式是半打开的垂直折叠模式
fun FoldingFeature.isBookMode() =
    state == FoldingFeature.State.HALF_OPENED &&
        orientation == FoldingFeature.Orientation.VERTICAL

// 桌面模式是半打开的水平折叠模式
fun FoldingFeature.isTableTopMode() =
    state == FoldingFeature.State.HALF_OPENED &&
        orientation == FoldingFeature.Orientation.HORIZONTAL

△ Judgment condition between book mode and desktop mode

FoldingFeature also contains the folding position in the window. When the folding causes the content view to be split, we should update the layout parameters in time. You can make tweaks like placing the support panel on one side, or displaying the homepage banner in the top half of the fold. First, we need to know the position of the content view in the window, and the position information can be obtained through getLocationInWindow . We'll create a Rect object using these coordinates along with the width and height, so we get the view's bounds in the window's coordinate space.

FoldingFeature gives the collapse bounds in the window's coordinate space, so we can directly check if the two regions intersect, and if so, we can convert the featureRect's bounds to the view's coordinate space and return it. By the way, if you use SlidingPaneLayout for list/detail layout, you automatically get support for book mode. SlidingPaneLayout will place the panes on the opposite side of the collapsed position as long as both panes fit in.

fun getFoldBoundsInView(
       foldingFeature: FoldingFeature,
       view: View
): Rect? {
       // 获取视图在窗口坐标空间中的边界
       val viewLocation = IntArray(2)
       view.getLocationInWindow(viewLocation)

       val (viewX, viewY) = viewLocation
       val viewRect = Rect(
           left = viewX, top = viewY
           right = viewX + view.width, bottom = view + view.height
       )
    …

       //显示功能的边界已经在窗口的坐标空间中
       // 检查 view 的边界和显示功能的边界是否相交
       val featureRect = Rect(foldingFeature.bounds)
       val intersects = featureRect. intersect (viewRect)

       if (featureRect.isEmpty || ! intersects)
           return null
       }

       // 将功能的边界坐标转换为 view 的坐标空间
       featureRect.offset(-viewX, -viewY)
       return featureRect
}

△ Get the folded position information

test

If your app has special behavior related to the collapsed state, you will need to write unit tests for this. There is a test rule in Jetpack Window Manager that supports simulating FoldingFeature during instrumentation testing. Since the test needs a view, we add WindowLayoutInfoPublisherRule and ActivityScenarioRule , which together form a test rule chain. In this test method, we get Activity activityRule then create a window attribute to simulate desktop mode, build a WindowLayoutInfo object and publish it with publisherRule . After that, we can use the Espresso and JUnit assertions to check Activity in desktop mode.

private val publisherRule = WindowLayoutInfoPublisherRule()
private val activityRule = ActivityScenarioRule (MyActivity: :class.java)

@get :Rule
val testRule = RuleChain.outerRule (publisherRule) .around(activityRule)

@Test
fun testDeviceOpen_TableTop(): Unit = testScope.runBlockingTest {
    activityRule.scenario.onActivity { activity ->
        val feature = FoldingFeature (activity, HALF_OPENED, HORIZONTAL)
        val testWindowInfo = WindowLayoutInfo.Builder( )
                .setDisplayFeatures (listOf (feature))
                .build()

        publisherRule.overrideWindowLayoutInfo(testWindowInfo)
    }
        // 编写基于桌面模式的断言
}

△ Test the folded state

UI testing can be difficult because some tests must be performed on specific devices. To that end, Android Studio is adding support for Gradle-hosted virtual devices. You can use the Android Gradle plugin version 7.1 and above to experience this feature.

In the app-level build.gradle file, under the testOptions module, specify the virtual device configuration file as you would normally manage and run virtual devices in Android Studio. For example, here is a Pixel C tablet image, and Gradle will then create a target that can execute tests on the specified device, and even download the device image if needed.

android {
    testoptions {
        devices {
            pixelCapi30 (ManagedVirtualDevice) {
                device = "Pixel C" // 平板电脑设备
                apilevel = 30
                systemImageSource = "aosp" // 如需 GooglePlay 服务，使用 “google”
                abi = "x86”
            }
        }
    }
}

#Gradle target = {device name} + {build variant} + "AndroidTest"
./gradlew pixelCapi30debugAndroidTest

△ Virtual device configuration

To make it easier to distinguish which tests are for which devices, we will create a custom annotation LargeScreenTest and mark the test function with this annotation. When running the previous Gradle command, we add a parameter to AndroidTestRunner to ensure that only tests with this annotation are run. If you don't use annotations, you can also use TestRunner's other filtering options, such as running tests in a specific class. Combining these features, we can set up a consistent run configuration for our tests.

annotation class LargeScreenTest

@RunWith(AndroidJUnit4: :class)
class MyActivityTest {

    @Test @LargeScreenTest
    fun largeScreenDeviceTest() {
        // 在平板电脑设备上测试界面
    }
}

# 只运行带有指定注解的测试
. /gradlew pixelCapi30debugAndroidTest \-Pandroid.testInstrumentationRunnerArguments.annotation=com.mypkg.LargeScreenTest

△ Use custom annotations to write tests for specific devices

In addition to making the content on the screen look bigger, the big screen brings some other opportunities to help your app shine. In multi-window mode , your application can be used side by side with other applications. In addition to responsive adjustment, you can also consider how to make the application play a greater role in this mode, such as supporting drag and drop. This little feature increases user productivity and makes users more likely to use your app.

△ Multi-window mode effect

In addition to interaction through touch, large-screen devices also support other forms of interaction. The larger the screen size of the device, the more likely the user is to use a keyboard, stylus, mouse, gamepad, or other external device. If you want to improve the usability of your app in these situations, you can plan to support some of these input methods, for more details, see the article " Time for Perfect Input Support for Various Devices ".

In such a diverse hardware ecosystem, it can be difficult to have devices of all shapes and sizes, today the Android SDK provides emulator images for foldable devices that allow you to change the folded state to the angle of the hinge at any time . The upcoming Android Studio Chipmunk will also feature a resizable emulator, allowing you to freely change the size of the app window, and every developer can try their app on almost any type of device.

△ Resizable emulator in Android Studio Chipmunk

We've also been working on new tools in Android Studio and hope to support you in developing big screen apps. The new Layout Validation tool can preview layouts on reference devices covering various size categories, prompt for problem areas (such as text using long lines), and recommend different interface components for different breakpoints.

△ Layout Validation in Android Studio

Finally, we listed the app quality guideline for the large screen on the Android developer website. The first part of the guide describes basic compatibility expectations, such as whether the app supports both landscape and portrait mode, and the following parts Focus on supporting various screen types and states, and using specific screen types or states to create different experiences.

We hope you'll all be able to use what we've shared today, along with the new quality guidelines, to build apps that appeal to users across all screen sizes.

You are welcome click here to submit feedback to us, or share your favorite content and found problems. Your feedback is very important to us, thank you for your support!