Back-half - The foldable UX exploration
Back-half is a productivity and organization tool syncing the back and the front screen assisting users in creating notes and generating to-do lists with a focus on reducing the cognitive load.
This collaborative human-computer interaction design project was conducted between Huawei Technologies and Tsinghua University, exploring new possibilities of saving and extracting item information from different apps and integrating them on the back of our phones. This research project introduced a UI/UX design exploring the potential of interacting between the front screen and the Back-of-Device(BOD).
My Role
Product Designer
HCI Researcher
Project Type
HCI Design
UI/UX Design
Tangible Interaction Design
Invidual Project
Timeline
Summer 2020
(1 month)
Tools Used
Literature Review
Competitive Analysis
User Research
Usability Heuristics
Information Architecture
Wireframing
Prototyping
User Testing
Design Iteration
Overview
Background
Back-half was designed as a flexible, all-in-one solution for managing personal productivity on the back of our device. It offers a range of features, including the ability to create notes, extract item information from the apps, and store the captured info in the databases.
The abundance of information available online, including the proliferation of apps, has led to an "information explosion" that can be overwhelming for people. In response to this, I have come up with the concept of an app that allows users to easily access and organize information from various resources in one place. The goal of this app is to provide a simple and efficient way for people to manage and make sense of the vast amount of data available to them.
Users want portable devices with adaptable interfaces - add a whole new dimension to managing multiple apps with multi-active windows without increasing the cognitive load.
The Process
Project Goals
Challenges
Challenges faced
-
It's difficult to imagine syncing information from various apps on one platform
-
Users are yearning for an inclusive app to help collect and organize information across the platforms and therefore reduce their cognitive load
-
Exploration of the potential of Back-of-Device(BOD) has a huge impact on the functionality of the back of our mobile phone
Design Research
Research
Understanding the existing solutions
15+ UI/UX design audits
Key Objectives
-
Learn from the existing foldable devices' UX design guide
-
Understand the differences in the UI/UX implementation on different folded devices
-
Learn the design of the multi-page layout with a depth of exploring the types of relationships between the multiple pages
Benchmarking the existing apps
5 APP audits
Key Objectives
-
Unearth the unsolved user pain points
-
Identify the potential business goals and set appropriate performance targets
-
Provides valuable data for design improvement
Identify the design directions
20+ Paper Review
Key Objectives
-
Learn the technique dead-ends of the back of the device
-
Learn the research and experiment methods from a scientific perspective
-
Advance scientific knowledge and improve the overall rigor and reliability of research.
Redesign based on usability testing
2 design iterations
Key Objectives
-
Explore a wide range of potential solutions and ensure that the final design is well-rounded and optimized
-
Identify potential trade-offs and to make informed decisions about the direction of the design
Foldable Device Study
First, I researched competitors in the field of foldable phones, and there are three main types:
Versatile tablet
A versatile tablet with a vertical hinge that keeps each half of the display facing outward when the fold is closed and transforms into a massive display when opened
Inward folding phone
Inward folding phone with dual screens. Manufacturers make use of two distinct screens, which can be beneficial for users watching movies and playing games
Clamshell phone
This design folds down like a flip phone, creating a compact device that users can easily slip into a pocket
Usability auditing
Physical forms of the flexible/foldable devices:
-
Folded status: The screen is folded.
-
Unfolded status: The screen is not folded.
-
Semi-folded status: The screen is not completely folded, so that one half of the screen can function as a support, allowing the phone to be placed stably.
💡Takeaways:
These three characteristics pose new challenges to app design and development.
To fully utilize the advantages of the foldable screen and deliver satisfying user experiences, the expectation for the UX design of apps on foldable devices are as follows:
🧾
🧾
-
Expectation 1: Continuous experience
Provide consistent user experience by ensuring:
-
Screen compatibility: Due to the change of screen size, appropriate methods should be used to optimize the content layout on different screens.
-
App continuity: Applications run properly during the switchover between the folded and unfolded states.
-
2. Expectation 2: Enriched experience
-
Enrich services on the larger screen by providing:
-
More content: The screen can display more content, improving the display and operation efficiency.
-
Immersive experience: More details can be displayed for picture browsing, video watching, and gaming.
-
Multi-window interaction: With the increase in the screen size, content, and information types, the interaction relationships between information blocks becomes more complicated; two tasks may often be performed at the same time; users' operation area also expands. More interaction methods can be used, increasing the chances of multi-window interaction.
Conditions and constraints
Get to know hardware and UI considerations that are key to designing for adaptive experiences. Here I lay the conditions and constraints to consider when adapting apps for foldable screens out to explore the further design potentials.
Multi-page Layout and Information Architecture Design
The phone when unfolded allows the display of more content on a single page, as well as simultaneous display of two pages.
Relationship Types between Multiple Pages
When two pages in an app are directly associated with each other, combine them to provide better experience. The type of relationship between the multiple pages directly affects how the information should be rearranged and displayed. Apps need to select proper information architecture styles based on different relationship types.
There are two types of relationships between the multiple pages:
-
Hierarchical relationship
-
Principal and subordinate relationship
Hierarchical relationship
There are two types of hierarchical relationships: multi-level and single-level.
1. Multi-level
The multi-level relationship has a complete multi-level structure, for example, complex system settings menus, portals with massive content (such as products on comprehensive e-commerce platforms; video, image, or music media resources; massive news on news websites), and multi-level sub-pages.
The interaction logic of this type has the following features:
-
In addition to the basic categories at the first level, the sub-categories at all levels are also displayed in the form of lists.
-
Elements on the bottom-level list page are the minimum content elements, such as a single product, a single piece of media material, and a single piece of news.
2. Single-level
A single-level structure is generally presented in the form of "list+details". Each element in the list is an end element, and there is no level-2 category list in the element. For example, a list of content-type elements such as an email list, message list, and a memo list.
The interaction logic of this type has the following features:
-
The list contains only independent elements and does not contain sub-lists. The content attributes on the left and right sides are fixed, and users are not likely to get confused.
-
Users can touch an entry on the left to have the details displayed on the right.
-
Users can touch an entry on the left to quickly switch the content displayed on the right.
📂
📂
Principal and subordinate relationship
In the principal and subordinate relationship, the principal page on one side provides an immersive scene, while the subordinate page on the other side displays information such as comments, discussions, and references. The immersive content can be divided into the left-right or top-down structure.
The interaction logic of this type has the following features:
-
The two pages are independent of each other but there is an obvious principal and subordinate relationship.
-
The subordinate page depends on the principal page, and cannot exist independently when the principle page is closed.
-
The content presented on the principal page should provide and uninterrupted, immersive experience.
-
Users should be able to scroll and view the content presented on the subordinate page. This mode is applicable to information flow, in which users can participate in interaction.
Business potential / constraint reseach
Competitive Analysis
-
The total number of actions: While operating between different pages and going back with the return operation, it will typically require more effort for most tasks. Multi-tasking = context shifting. The key question is —— What should I do to reduce the cognitive load while designing the assets and UX logic of the back of the device?
-
More sub-tasks: The elements and categories on the back of the device should engage with sub-tasks to prevent users from getting when refocusing on the original task. Users should be able to scroll and view the content presented on the back page.
-
Immersive: The content presented on the back of the device should provide an uninterrupted, immersive experience.
-
Coherence: Users can touch an entry on the back page to have more details displayed on the right.
Paper Review
This Human-computer interaction design project concentrates on creating a horizontally flexible smartphone and investigating back-of-device interaction. To extend the front screen display and improve user experience, I designed a bendable phone solution with an extra back screen. Based on this idea, I've sketched out a preliminary wireframe for the front and back screens' interaction design. Users can swipe up to add the web pages to their favorites or re-read later when browsing information on their phones, and in turn, a module about the information will appear on the phone's rear screen. After that, the user can press the module to continue reading what they were on previously. This design can help users better manage the information on a daily basis in the era with the rapidly increasing amount of published information with an abundance of data. To better investigate the design method and understand the technical boundaries, I went to ACM and Google Scholar to read articles in related domains.
Paper 1: Luca A D, Zezschwitz E V, Nguyen N D H, Luca A D, Zezschwitz E V, Nguyen N D H, Luca A D, Zezschwitz E V, Nguyen N D H, Smartphones' back-of-device [C]/ SIGCHI Conference on Human Factors in Systems Proceedings. ACM Press, 2013.
This research introduces BoD(back-of-device) Shapes, a revolutionary smartphone input technique that uses the device's back to prevent shoulder surfing.
This design influenced me in the following ways: 1.BoD(Back- of- device) Shapes is a novel authentication method for smartphone that uses the back of the device for input. Since it requires relative movements, and the user doesn't have to hit specific targets on the back screen perfectly.
It encouraged me to create relative interactive gestures, regardless of where the interaction begins or ends. 1. The demo's hardware prototype also inspired me to use bumpers to connect separate phones to imitate the bendable phone design. To connect the system of the phones, I can install a custom application on each device and connect the two smartphones over Wi-Fi direct. 3. Conducting experiments in four different systems effectively verifies concept viability, including the accuracy test, average authentication speed test, and a successful shoulder surfing assault test.
Limitations: 1. When using rear screen motions to input the passwords, the error rates are substantially greater than when users use the front screen gestures since they can't see it from the front. 2. When people tap their texts quickly on the front screen, it seems others could not do shoulder surfing.
Paper 2:Tan, Maciej Kumorek, Andres A. Garcia, and others. A Foldable Mobile Device with Shape Interactive Applications is Projectagami. 2015.
Projectagami is a foldable mobile device design that could change its shape in various ways using hinges, which help develop mobile devices' potential. It bridges the gap between different-sized touch screen devices and allows for the interaction between different screens.
This paper motivates me to give a feasible presentation using 'Wizard of Oz' to demonstrate the concept. It can present various demos, including reading books, online shopping, playing board games, and street navigation using the Kinect and projector on the foldable mobile device.
Limitations:1. The 'Wizard of Oz' demonstration doesn't embed with the technical system solution and computer program design; thus, this hypothetical experiment cannot generate solid experimental data from the user.
Paper 3: Gomes A. PaperFold: A Shape-Changing Mobile Device with Multiple Reconfigurable Electrophoretic Magnetic Display Tiles[C]/CHI'14, ACM, 2014. PaperFold is a new shape-changing mobile gadget with several touch-sensitive thin-film electrophoretic magnetic display tiles that can be reconfigured. PaperFold investigates the advantages of combining numerous devices into a single mobile device with multiple displays.
This research has inspired me to reference the prototype's hardware design when developing the mobile gadget independently— (a) IMU Board. (b) Linear Hall Effect Sensor. (c) 1.0 X 3/8 X 1/16" Block magnets. (d) 1/4" X 1/4 "Cylindrical Magnet. (e) 1/8" X 1/8 "Block Magnets. (f) Copper Substrate (g) 20M Ohm Sensor Pins. (h) 40M Ohm Sensor Pins. (i) Common Send Pins. The specific implementation includes (1)using 3D printing to create the tiles, (2)designing the system which allows various tiles to be distinct from each other using Linear Hall Effect Sensor. (3) Calculate the absolute orientation of each tile using 9DOF IMUs.
Early solutions
High-fi prototype
Reflection
Takeaways
This is the first HCI design I have done! I have benefited the most from the paper review section which I gained an opportunity to see early areas of research directions and thus sort out the notion I wasn't aware of. After this project, I feel it's necessary for us to conduct a paper review before we decide on the design direction, which can help us avoid the dead end on a specific topic in academic research.
If I have more time to do this project, I may take the challenge of creating the physical model to test the feasibility and capability of the application I've designed. The physical model is a further step beyond the concept design of UI/UX to conduct the A/B test and iterate the design.
UI/UX Design, Service Design, Sustainable Design
Developed a service design system that extends the life cycle of shoes and reduces the carbon footprint through the recycling process
UI/UX Design, Hardware design, Design Thinking
Designed user experience of hearing aids and music app to improve the mental health of elderly people