Prototyping

vr/ar

prototyping tool

research

Last year, I joined in a really meaningful and inspiring project led by Michael Nebeling, who is a professor at the University of Michigan, School of Information, doing research around design authoring tools. And he led me to experience different methods and tools to prototype for VR/AR applications. Methods like Play-Doh Prototyping, paper prototyping, 360photo prototyping, Unity and Google Cardboard for VR prototyping and Unity and HoloKit for AR prototyping, used in these projects to communicate our design concepts and test our design flow on real users. If you want to know more about this project, click here.

Prototyping as an essential activity for most designers is pretty challenging for Mixed-Reality applications designers because of the following reasons:

2D to 3D. The canvas for designers is not flat anymore while our tools for prototypes are mostly flat. Not only do the components of the application like menus become stereoscopic, but also the application itself becomes three dimensional.
Multimodal Interaction. Immersive technology allows users to interact with the computational products in a more natural way, where they can unleash the power of their human body: eye movement, hand gestures, voice, and body language. However, it’s hard to prototype these interaction concepts even if you are a professional software engineer. You need to understand tracking technology, sensors, human movements and senses, and design to help you visualize the design concept and test it on real users.
Multiple input devices. There are a lot of different VR headsets and AR glasses around the market. Each of them has their own using logic: controllers and interaction techniques. We should be aware of what devices our target users would be more accessible too.
Limited Design Authoring Tool. Ok, I think you have to become a developer when you want to be a designer for VR/AR application.

Here are some prototyping examples of different VR/AR applications.

VR prototyping

VR Prototyping

VR mandarin Learning App

This application creates a great virtual environment that allows language learners to learn, practice and review the foreign languages in realistic foreign scenarios.

We selected two main tasks of this application to do the prototype. One is that users practice their speaking skills by chatting with NPCs in a restaurant and another one is that users chat with real users to apply what they learned in real life.

Physical Prototype

Goal: Tell the story of those two tasks and test how our audience perceives it.

Method: Physical Prototype. The process of building 3D prototypes with Play-Doh help us to capture some details of the interactions that we probably missed in the 2D sketch like how to place those characters and models before time-consumingly diving into a 3D development tool like Unity.

Digital Prototype

Goal: View the design in the real VR device

Method: Unity and Cardboard

Result:

The transition between the scenes is extremely important for users to understand the relationships between those scenarios.
Avoid strong lighting changing.

If you are interested in this project, HERE is the detailed description.

VR 3d sketch app

In this project, I designed and developed a light-weighted 3D Sketch VR application using HTC VIVE (VR headset).

Digital Prototype

Goal:

Test how users understand and feel about the virtual environment
Test whether the arrangement of interactive components - menu match their expectation of finding information?
Test users’ using posture
Test selection mechanism.

Method: Unity + Google Cardboard. Test the virtual environment and menu

Here is the testing scene, which is a Google Cardboard view.

Result: Change the position of the menu close to the user.

AR prototyping

If you are interested in this project, HERE is the detailed description.

AR Prototyping

Compared to VR prototyping, AR asks designers to pay more attention to real surroundings which are one crucial part of the user interface. As 3 dimensions are taken into account, similar to a “Material Design-X” (more height consideration), there are more possibilities in interactions but also a challenge as more biophysical factors needs to be considered as well.

AR Grocery shopping

In this project, this application supports the following features:

1) In-store Navigation:

Search items in the store by typing or voice input
Display a virtual map with navigation to the item
Show the directional arrows on the way and provide voice navigation option

2) Item Information Searching and Browsing

Display product detailed information in an intuitive way
Provide on-sale information and coupon available for a specific item
Highlight hot-sale products and enable cross-store product/price comparison

Digital Prototype

Goal: Test item searching process

Method: A-frame + 360 photo.

Result: We can use VR to test AR project by constructing a virtual environment that represents the AR using scenarios.

HERE to experience the prototype

AR Typing and Reading

In this project, I designed a wearable AR reading application.

Storyboard

Goal: Tell the concept of reading and taking notes in a wearable AR application.

Method: Keynote.

Result: Clearly communicate my design idea.

If you are interested in this project, HERE is the detailed description.

AR inspiration collection

Look-like prototype or work-like prototype, make your choice!

Ok, here is the look-like prototype.

We may have multiples tools for the prototype in VR and AR application, but sometimes, they can only work well for a specific application for some certain use case. For me, my approach is to keep the purpose of prototypes — what I need to test in mind and selecting the tool that I am most skilled at. For example, I’m good at Unity, so sometimes I would prefer to skip the step of the sketch, and build the environment in the Unity, and then test it in VR with Google Cardboard. For me, it will save a lot of time for me to sketch some 3D space on a flat paper, even though it is a 360paper.