Build AR "Core Building Blocks"
"Core Building Blocks" Unlike most of the 2D app design, to deliver a comprehensive AR application based on no existing platform, we need to define gestures and operations to illustrate how our user could explore this application. In that regard, inspired by Microsoft HoloLens, we came up with our own operation guideline according our research and observation.
Basic Gesture Gaze Gaze act like a cursor in a computer operating system where the user could determine the intent when they are looking at the onsite hologram. Facilitated by built in eye tracker, user could enjoy a more intuitive experience without using their head as the pointer. Given that the whole process is facilitated by eye tracker, there won't be actual cursor in the interface. Please refer the "Select" section under the "Basic Operation" for more about how gaze work to navigate between pages.
Voice Voice act as a complementary input in the rare occasion that other input is inefficient. It allows you to directly command a hologram without having to use gestures. You simply gaze at a hologram and speak your command. Voice input can be a natural way to communicate your intent.
Pinch- Unpinch, Short Pinch and Long Pinch The Pinch action, operates like the click and drag operation on a everyday computer. It provide our user with intuitive way of interactive with the onsite hologram. Short Pinch is where the user move his/her thumb and index finger closer. Like click/double click in a computer, it is used to select a certain hologram in the field.
Long Pinch is where the user use short pinch to select an object and didn't let go. Like drag or pull in a computer, it is used to relocate a certain hologram in the
Unpinch is the reverse of pinch where the user move their thumb and index finger to different direction. Like operating in a phone or tablet, unpinch is a more intuitive operation for scale an hologram The difference between Control Panel and the Onsite Hologram is defined in the Interaction Hierarchy
Air Tap Air tap operates like the click and drag operation on a everyday computer. It provide our user with intuitive way of interact with the Control Panel
Like click/double click in a computer, it is used to select a certain hologram in the field. It is helpful for HoloLens user operating in this hardware.
The difference between Control Panel and the Onsite Hologram is defined in the Interaction Hierarchy
Interface Orientation This chapter describe how the hologram is oriented in the field/scene.
Spatial Mapping, or in another word, indoor positioning is achieved by the RGB Camera and depth camera. It measure and simulate the physical condition of the working field. The hologram without image target reference would place themselves accordingly.
Image target is a borrowed term from the Vuforia Engine. It represents 2d images that our device could detect and track.
Spatial Mapping
Image Target
It enhances the calibration process, motion tracking, and triggers certain hologram.
Interaction Hierarchy Interaction Hierarchy describe the two types of interface(hologram) that would appears in the scene and the structure of these interface.
Onsite Hologram
Micro Control Panel
Onsite Hologram is a large hologram placed in the jobsite for relative personnel to reference as video/image/document/3D model. It could be scale/move/rotate/delete by gesture or buttons on control panel.
Control Panel is a complementary way of address operation towards an onsite hologram. It is triggered by the image target on the palm or the back of the user's hand. User could only interact with a hologram when it is selected.
Basic Operation Select - Visual Standard Visual Standard explains how object would looks like when they are unselected, targeted or selected. For the onsite hologram, user could only interact with a hologram when the hologram is selected.
Unselect Focal Point
Targeted
Selected
An example of Targeting and Selecting 3D and 2D Interface
Unselected When a hologram is unselected, it would show 50% of transparency. Targeted When an unselected hologram is being targeted, it is being gazed by the user. The hologram would show 75% of transparency and been highlighted in red color. Selected When a hologram is selected, user could interact with it using either gestures or control panel. The hologram would show 100% of transparency.
- Gesture To select an onsite hologram, Gaze and Short Pinch To select an button on the control panel, Tap the button with your index finger.
Move The user could use either gesture or control panel to move an onsite hologram. The move of an onsite hologram does not involve alteration of its relative height. To perform a move, the user has to select the intended onsite hologram, Long Pinch and Drag. When the hologram is placed at the designated location, the user could unpinch and release the hologram.
Resize The user could resize(scale) either an onsite hologram or the document of his/her palm control panel. It could be achieved by either gesture or control panel to resize an onsite hologram. Resize could be performed by using either one hand or two hands. For one hand operation, the user has to select the intended onsite hologram and unpinch. For two hands operation, the user has to select the intended onsite hologram, use their both hand to pinch it, and drag. One Hand Operation
Two Hand Operation
Follow By making an onsite hologram follow the user, he/she could be able to quickly refer to a model/document when he/she is constantly moving from one place to another. To make the onsite hologram follow the user, the user would long pinch the hologram and swipe. Obviously, the intended onsite hologram has to be selected before the operation.
Reset To reset the AR scene and clear the unnecessary onsite hologram, the user could gaze at the selected hologram he/she is going to keep on site, long pinch and shake. By doing this there would only be the intended hologram that still exists in the work field. To purge all the existing hologram, the user could gaze at none of the hologram, long pinch and shake.