The Final Product
The gestural interface is a type of user interaction, where the user interacts with the computing device through a set of well-defined gestures. In this project, the research and understanding of the meaning of different gestures were beneficial to me in designing this application.
Design Process
Research
The goal of this research was to observe the behavior of myself and a friend and to document the gestures we use in our daily lives. Through this study, I found that different people will use different gestures to express the same meaning. However, people will use almost the same gestures when expressing some common meanings. In this case, I thought it would be better to design and use some common gestures when designing gestures for this app. It would also be easier for people to use the app and remember the functions of the app.
Representing Gestures
Initially, I intended to design some special gestures for this application. However, as the design progressed, I realized that these gestures were difficult for the users to remember. In the end, I chose to let users use some common and simple gestures in this app to achieve their goals.
Echo and Semantic Feedback
Feedback is an important part of any interactive system - especially for gestural interfaces. Feedback gives users a more intuitive understanding of their interaction with the system. System feedback can tell users what they did, what they need to do, and how they can achieve that step. In the gestural interfaces, there is no mouse cursor to see moving, physical mouse buttons to click, or touch-screen to feel "tapped”. So good feedback is essential in the gestural interfaces.
Echo Feedback is unprocessed sensor data sent back to the user. It ‘echoes’ what the user is doing, without activating anything in the interface - such as activating a button. This kind of feedback tells the user the system is aware of the user’s interactions, and helps the user calibrate their physical movements. Semantic Feedback is processed sensor data turned into visual / auditory / tactile feedback to the user. It tells the user they have successfully performed and completed an action.
When the user makes a gesture in front of the screen, the navigation will give the user a quick feedback (turning green) to let him know that his hand is detected.
User Flow Diagramming
User Flow Diagrams of the app.
UI Patterns
UI patterns are reusable/recurring components which designers use to solve common problems in user interface design. The problem is that users are short on time and patience from having to complete basic tasks repeatedly. They want recognizable ways of accessing their information or getting their work accomplished. Using existing Patterns helps eliminate (or at least diminish) this pain point for them. They already know what to expect and how to interact.
A UI Pattern is different from a UX Pattern. UX Patterns refer to the experience - or user journey someone takes when using an app. UI Patterns focus on the layout and design of the visual interface.
However, if an inappropriate UI pattern is used, it may deliver misunderstanding info to users and may introduce inefficiency into the final product or solution.
Some examples of the UI patterns I found that can be used in this Apple Maps — Gesture-controlled Version.
Wireframes of the app
User Testing & User Testing Script
Main takeaways.
Maybe lead the user to try more gestures in the on-boarding part.
Maybe there is no need for the “Confirm” gesture.
Style Guides
Color palette design process
At first, I tried to use some warm colors on the wireframes. As I tried more, I found that the warm colors would confuse the user, and the contrast between the white font and the warm colors on the page was not strong enough for the user to see the content on the page clearly. Therefore, I finally chose to use black and white color.
Green Screen Filmmaking
Green Screen filming is a method used by the film industry to ‘cut out’ actors from their background, so they can be artificially composited in a fabricated scene. Using green screen in video replaces the background of a video with a digital or virtual background. It‘s also a useful tool for interaction and experience designers, for when they want to tell the story of how a product, service or experience will work - in relation to the body, or a specific space.
To create my green screen video, I first used Adobe Dimension to create a green screen phone model and combined it with the background. After recording the green screen of the hands, I used Adobe After Effects to adjust the angle of the hands to make them fit my background.
I think the biggest challenge is to make the hands look more real in the video. Because there will always be some shadows. Finally, I used Adobe Premiere to create and edit my green screen video and did my best to eliminate the shadows.
