Page 1

22 rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 06 10 http://www.t-immersion.com

2011-12-07

D’Fusion  Mobile User Guide

2011-12-07

Reference

Page number

DFusion Mobile - User guide.doc-rev20662.svn000.tmp.doc

65


D’Fus ion  Mobile User guide

2011-12-07

TABLE OF CONTENTS 1

INTRODUCTION 1.1 1.2 1.3 1.4

2

19

General recommendations....................................................................................................... 19 Video configuration file ........................................................................................................... 20 Camera calibration file ............................................................................................................ 23 Computer Vision requirements ................................................................................................ 25 Media requirements ................................................................................................................. 26 Lua recommendations ............................................................................................................. 28 Testing a mobile scenario on a non-mobile platform .............................................................. 29

30

Export with D’Fusion Studio .................................................................................................... 30 tiProtectorAR2 tool (Pro only) ................................................................................................. 33

39

Apple applications ................................................................................................................... 39 Android Application ................................................................................................................. 54

FINAL DEPLOYMENT 7.1 7.2

8

Components architecture ........................................................................................................ 12 Components interaction .......................................................................................................... 13 Interface for iOS ...................................................................................................................... 14 Interface for Android ............................................................................................................... 16 Component execution sequencing........................................................................................... 18

TEST AND DEPLOYMENT 6.1 6.2

7

12

EXPORT AND TEST KEY GENERATION 5.1 5.2

6

D’Fusion Mobile: SDK and tools ................................................................................................. 6 Deployment workflow with D’Fusion Mobile ............................................................................. 6 Technical requirements ............................................................................................................. 7 Hardware prerequisites for Android devices: dedicated GPU .................................................... 7 D’Fusion Mobile SDK for iOS product description ...................................................................... 8 D’Fusion Mobile SDK for Android product description ............................................................... 9 Mobile platform features availability....................................................................................... 10

PROJECT CREATION 4.1 4.2 4.3 4.4 4.5 4.6 4.7

5

6

D’FUSION MOBILE INTERFACE DESCRIPTION 3.1 3.2 3.3 3.4 3.5

4

Object ........................................................................................................................................ 4 External documentation ............................................................................................................ 4 Glossary & Acronyms ................................................................................................................. 5 Presentation rules ..................................................................................................................... 5

D’FUSION MOBILE PRODUCT DESCRIPTION 2.1 2.2 2.3 2.4 2.5 2.6 2.7

3

4

62

Steps for final deployment ...................................................................................................... 62 License coherency using multiple Software Keys .................................................................... 62

TROUBLESHOOTING & FAQ 8.1 8.2

63

License error messages ........................................................................................................... 63 FAQ .......................................................................................................................................... 64

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

2 / 65


D’Fus ion  Mobile User guide

2011-12-07

TABLE OF FIGURES Figure 1: HW incompatibility message displayed on Android devices without a dedicated GPU _____________7 Figure 2: D’Fusion Mobile SDK component architecture ___________________________________________12 Figure 3: D’Fusion Mobile SDK for Android components integration __________________________________13 Figure 4: D’Fusion Mobile SDK for iOS components integration _____________________________________13 Figure 5: Component execution sequencing ____________________________________________________18 Figure 6: XCode - new project _______________________________________________________________39 Figure 7: XCode - window-based application ____________________________________________________40 Figure 8: XCode - Get Info __________________________________________________________________41 Figure 9: XCode - Build active architecture _____________________________________________________41 Figure 10: D'Fusion Mobile library in XCode project_______________________________________________41 Figure 11: XCode project - resources and Frameworks ____________________________________________42 Figure 12: XCode - Add Existing Frameworks ___________________________________________________43 Figure 13: XCode - frameworks to select _______________________________________________________44 Figure 14: XCode - add libz.dylib to project _____________________________________________________45 Figure 15: XCode - MainWindow.xib ___________________________________________________________47 Figure 16: Interface Builder - UIView __________________________________________________________47 Figure 17: Interface builder - UIView connection ________________________________________________48 Figure 18: Interface builder - UIView connection ________________________________________________48 Figure 19: XCode - adding your scenario to XCode Resources ______________________________________51 Figure 20: XCode - Scenario hierarchy _________________________________________________________52 Figure 21: XCode Organizer window __________________________________________________________52 Figure 22: Organizer - Add Device to Provisioning Portal __________________________________________53 Figure 23: XCode Organizer - Team Provisioning Profile ___________________________________________53 Figure 24: Eclipse project creation ____________________________________________________________54 Figure 25: Android project creation ___________________________________________________________55 Figure 26: API 7 (OS 2.1) project selection _____________________________________________________56 Figure 27: Scenario and D'Fusion resources in the "assets" directory_________________________________57 Figure 28: libtiAndroidAR2.so and dfusionmobilesdk.jar folder organization ___________________________58 Figure 29: Add dfusionmobilesdk.jar to the build path ____________________________________________58 Figure 30: dfusionmobilesdk.jar is considered as a referenced library ________________________________59

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

3 / 65


D’Fus ion  Mobile User guide

2011-12-07

1 INTRODUCTION 1.1 Object This document describes the way an augmented reality application based on D’Fusion Mobile is to be created and used. This document is a technical reference and targets an audience with a technical background and directly involved in the usage of D’Fusion Mobile. This document is a counterpart for existing D’Fusion Studio Suite documentations and does not replace them. It provides complementary information, specific to mobile platforms.

1.2 External documentation 1.2.1 Reference documents [01]

D’Fusion Studio - User Guide, Total Immersion DFusion Studio - User Guide.pdf

[02]

D’Fusion Augmented Reality - Reference Manual, Total Immersion DFusion AR - Reference Manual.pdf

[03]

D’Fusion Augmented Reality - Lua API, Total Immersion DFusion AR - Lua API.pdf

[04]

D’Fusion Augmented Reality - Physics Plugin, Total Immersion DFusion AR - Physics Plugin.pdf

[05]

D’Fusion Computer Vision - Reference Manual, Total Immersion DFusion CV - Reference Manual.pdf

[06]

D’Fusion Exporter for Maya - User Manual, Total Immersion DFusion Exporter for Maya - User Guide.pdf

[07]

D’Fusion Exporter for Maya - Modeling Constraints, Total Immersion DFusion Exporter for Maya - Modeling Constraints.pdf

[08]

D’Fusion Exporter for 3dsMax - User manual, Total Immersion DFusion Exporter for 3dsMax - User Guide.pdf

[09]

D’Fusion Exporter for 3dsMax - Modeling Constraints, Total Immersion DFusion Exporter for 3dsMax - Modeling Constraints.pdf

1.2.2 Other documents [10]

LUA 5.1 Reference Manual www.lua.org/manual/5.1

[11]

Android SDK manual developer.android.com/sdk/index.html

[12]

Reference Eclipse www.eclipse.org

[13]

Reference dev iOS developer.apple.com/devcenter/ios/index.action

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

4 / 65


D’Fus ion  Mobile 2011-12-07

User guide

1.3 Glossary & Acronyms TI

Total Immersion

AR

Augmented Reality

MLT

Marker-less Tracking

CV

Computer Vision

1.4 Presentation rules Presentation XXXXXX

File name

XXXXXXX

Script: code samples

XXXXXXX

Key word

XXXXXXX

TOTAL IMMERSION

Content

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

Reminder: main information of the chapter

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

5 / 65


D’Fus ion  Mobile User guide

2011-12-07

2 D’FUSION MOBILE PRODUCT DESCRIPTION D’Fusion Mobile is a development toolkit available with D’Fusion® Studio Suite. D’Fusion Mobile will allow you to create augmented reality applications that can be deployed on mobile platforms. D’Fusion® Studio Suite also includes D’Fusion Studio, the software that will allow you to design and develop augmented reality scenarios and create the associated media.

2.1 D’Fusion Mobile: SDK and tools D’Fusion Mobile provides all the tools necessary to create AR applications for mobile platforms. It consists of two SDK: • D’Fusion Mobile SDK for iOS platforms (iPhone, iPod touch and iPad devices with camera) • D’Fusion Mobile SDK for Android platforms To develop your mobile AR application, you will also need a standard development environment specific to the targeted mobile platform operating system (for example, you can use Xcode for iOS and Eclipse for Android). In addition to the above platform-specific SDKs, D’Fusion Mobile also includes the tiPotectorAR2 tool (to generate Test Software Keys for specific data) and the PrepareFolderForUpdate tool (to create data packages). Both tools are platform-independent.

2.2 Deployment workflow with D’Fusion Mobile The following steps describe the workflow for creating a D’Fusion Mobile application:

1. Create your AR scenario Use D’Fusion Studio to create your AR scenario. You should consider reading the recommendations described in section 4 before you start your D’Fusion Studio project. Full documentation on how to use D’Fusion Studio to create AR scenarios is available in [01].

2. Export your project Export your D’Fusion Studio project using the “mobile” export target setting. When your project is exported, a Test Software Key (.dfk file) is automatically generated and saved next to your exported project (i.e. in your exported project folder). This key will allow you to test your AR scenario within your mobile application. Details are provided in section 5.

3. Deploy your application To deploy your D’Fusion Mobile application, the exported AR scenario and the D’Fusion Mobile SDK for the targeted platform are to be added to the development environment of the targeted platform (XCode/Eclipse for example). Your augmented reality mobile application will be built upon the D’Fusion Mobile SDK. The Test Software Key generated when exporting your D’Fusion Studio project will allow your mobile application to load the AR scenario. Details are provided in section 6.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

6 / 65


D’Fus ion  Mobile 2011-12-07

User guide

4. Final deployment Once your application is tested and validated, you can proceed to final deployment by requesting a Final Software Key from Total Immersion. If you are a D’Fusion Studio Suite Pro user, you can make this request by sending us your Test Software Key(s) (.dfk file(s)). Total Immersion will send the Final Software Key(s) (.dfk file(s)) back to you. The Final Software Key will allow you to play your augmented reality application in Final mode (without watermark). Details are provided in chapter 7.

2.3 Technical requirements Technical requirements on mobile devices and their operating system version are resumed in the following table: Platform

iPhone

OS (device)

Minimum: iOS 4.0 to 5.0 iPhone 3GS & 4 & 4S - iPod Touch 4G - iPad 2

CPU

Minimum: 600 MHz Recommended: 1000Mz

Memory

Minimum: 128 Mb Recommended: 256 Mb

Android Minimum: Android 2.1 Recommended: Android 2.3, 3.0

Graphic card Smartphone with 3D GPU Specifications depends on final usage

Camera

2.4 Hardware prerequisites for Android devices: dedicated GPU When targeting a large-scale end-user deployment, you should take into account that not all Android devices are compatible with D’Fusion Mobile executions. There exist Android devices (mobiles) that do not have a dedicated graphics processor (GPU), but a dedicated GPU is mandatory for a correct execution of an augmented reality application. This situation is correctly detected at runtime by D’Fusion Mobile and the following message is displayed:

Figure 1: HW incompatibility message displayed on Android devices without a dedicated GPU Were clearly identified as incompatible: -

TOTAL IMMERSION

low-end devices based on the chipset Qualcomm MSM7225

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

7 / 65


D’Fus ion  Mobile User guide

2011-12-07

2.5 D’Fusion Mobile SDK for iOS product description 2.5.1 SDK contents The contents of D’Fusion Mobile SDK for iOS are structured as follows: + Platform_iOS + SDK + dfusionres

- containing optional resources for the application runtime

- default.ttf - DFusion.fontdef - msgPoolAR2.xml - msgPoolCoeur.xml - msgPoolMLT.xml + Include - tiDFusionMobile.h + Lib

- header that defines the interface of D’Fusion Mobile SDK components - static libraries, use the correct one according to your project settings

- libDFusionMobile.a

- link to the most recent iOS version library

2.5.2 Minimum requirements Hardware and software minimum requirements to useg D’Fusion Mobile SDK for iOS are describe below. • Hardware requirements: • Apple device capable of recording video running on iOS 4.x • Apple Mac OS X computer used for iOS application development • PC running on Windows for the development of your AR scenario in D’Fusion Studio • Software requirements: • XCode installed • iOS SDK installed • D’Fusion Studio

We consider that you know how to create, build and execute a standard iOS project under the XCode environment.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

8 / 65


D’Fus ion  Mobile User guide

2011-12-07

2.6 D’Fusion Mobile SDK for Android product description 2.6.1 SDK contents The contents of D’Fusion Mobile SDK for Android are structured as follows: + Platform_Android + SDK + dfusionres

- containing optional resources for the application runtime

- default.ttf - DFusion.fontdef - msgPoolAR2.xml - msgPoolCoeur.xml - msgPoolMLT.xml - dfusionmobilesdk.jar

- java archive (library), the Android project must include it (link to it)

- libtiAndroidAR2.so

- native code dynamic library used by the .jar

2.6.2 Minimum requirements Hardware and software minimum requirements to use D’Fusion Mobile SDK for Android are described below: • Hardware requirements: • Android device with a dedicated GPU, a camera and Android OS version ≥ 2.1 installed • Android emulator is not compatible: you cannot develop your application using the Android’s emulator • PC running on Windows for the development of your AR scenario in D’Fusion Studio • For Android application development, you can use any of platforms that are supported by Android: Windows, Linux (i386), Mac OS X (Intel) • Software requirements: • Android SDK • Eclipse IDE Classic (3.4 or 3.5 or 3.6) • JDK (5 or 6) • Android Development Tools (ADT) plug-in • ADB (Android Debug Bridget) driver for your phone installed • D’Fusion Studio

We consider that you know how to create, build and execute a standard Android project under the Eclipse environment.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

9 / 65


D’Fus ion  Mobile 2011-12-07

User guide

2.7 Mobile platform features availability The following tables describe D’Fusion features’ availability through various platforms and products. All D’Fusion supported targets have been given for comparison purposes. D’Fusion Pro is set as a reference. As a result, if a feature is available for D’Fusion Pro, it will appear in the table even if it is not available or only partially available for another platform or product. Cross platforms features matrix Reference : D'Fusion Pro

@Home Windows

@Home Mac OS X

Mobile Android

Mobile iPhone

for Adobe Flash Player

+

+

+

+

Away3D Lite

Textures

+

+

Textures

+

+

Shaders

+

Meshes

3D Engine (Ogre 3D)

jpg, png, dds jpg, png, dds

+

+

Only size power of 2 -

Only size power of 2 -

(OpenGL-ES 1.X)

(OpenGL-ES 1.X)

+

+

Public only

Public only

+

Shadows

+

+

-

-

-

Envmaps

+

+

-

-

(shader)

2D Overlay

+

+

-

-

-

2D Text

+

+

-

-

-

Particles

+

+

+

+

-

Script Engine

Lua

Lua

Lua

Lua

Action Script

+

+

-

-

-

Video texturation from file

+

+

-

Without sound

+

Video recording (with sound)

+

+

-

-

Without sound

Screen capture

+

+

+

+

Image processing

+

+

-

-

-

+

+

+

+

Partial*

Mouse

+

+

-

-

+

Keyboard

+

+

Partially supported

-

+

GPS

-

-

+

+

-

Accelerometer

-

-

+

+

-

Multi-touch

-

-

+

+

-

Compass

-

-

+

+

-

Plugin SDK

-

-

-

-

-

Partial

Partial

+

+

-

+

+

+

+

-

Physic Engine (Bullet)

+ -

Video based features

Web manager Inputs

Component architecture Barcode plugin**

(*) partial Web Manager for Flash : many things available directly within Flash (**) details on supported barcode formats are to be found in D'Fusion AR2 documentation

Table 1: Cross-platform rendering features TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

10 / 65


D’Fus ion  Mobile 2011-12-07

User guide

Cross platforms features matrix Reference : D'Fusion Pro

@Home Windows

@Home Mac OS X

Mobile Android

Mobile iPhone

for Adobe Flash Player

+

+

-

-

-

Keyframe image format

*

*

-

-

-

Multi Planar Objects

+

+

-

-

-

Multi keyframes AutoInit (Reco)

+

+

-

-

-

3D Object tracking

+

+

-

-

-

Multi 3D Object tracking

+

+

-

-

-

Face detection

+

+

-

-

-

Face tracking

+

+

-

-

-

Exclusion zone

+

+

-

-

-

Pointing detection

+

+

-

-

-

+

+

+

+

+

Multi Planar Objects

+

+

+

+

+

Multi keyframes AutoInit (Reco)

+

+

+

+

+

Keyframe image format

*

*

bmp, jpg

bmp, jpg

jpg

3D Object tracking

+

+

+

+

+

Face detection

+

+

+

+

+

Face tracking

+

+

+

+

+

Motion detection

+

+

-

-

-

Background subtraction

+

+

-

-

-

Move detector

+

+

-

-

+

Driving Tool

+

+

-

-

+

MLT Classic

MLT Lite

CV Tools

Table 2: Cross-platform computer vision features

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

11 / 65


D’Fus ion  Mobile User guide

2011-12-07

3 D’FUSION MOBILE INTERFACE DESCRIPTION 3.1 Components architecture When developing augmented reality applications for mobile devices you must bear in mind that the application is built for a specific mobile platform. This means that you will be using the mobile platform’s development environment upon D’Fusion Mobile SDK components. These components include: • a dynamic library for the Android platform (.so) • a static library for the iOS platform (.a) Both D’Fusion Mobile SDKs provide most of D'Fusion Studio Suite engine features: • load D'Fusion Studio projects (.dpd files) • load Marker Less Tracking data to identify and track objects • execute Lua scripts associated with the project • draw 3D objects and play sounds In addition to these components, D’Fusion Mobile SDK also consists of other platform-specific include files and resources like .h, .xml and .jar files.

D’Fusion Mobile applications are built upon these components, together with the targeted mobile platform development environment.

Figure 2: D’Fusion Mobile SDK component architecture

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

12 / 65


D’Fus ion  Mobile User guide

2011-12-07

3.2 Components interaction Applications built for the iOS platform and applications built for the Android platform use different native code: • Android uses Java • iOS uses Objective-C, C or C++ For each platform, D'Fusion Mobile SDK provides an interface class called “tiComponent” which is a Java class for Android and an Objective-C class for iOS. These two interfaces are very similar. They provide methods to initialize the D'Fusion Mobile SDK components, load scenarios created with D'Fusion Studio, and communicate with the Lua scripts included in your scenario.

Figure 3: D’Fusion Mobile SDK for Android components integration

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

Figure 4: D’Fusion Mobile SDK for iOS components integration

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

13 / 65


D’Fus ion  Mobile User guide

2011-12-07

3.3 Interface for iOS 3.3.1 tiComponent declaration The declaration of the tiComponent interface for iOS is as follows:

@interface tiComponent : NSObject - (id)initialize:(UIView*)renderView; - (id)initialize:(UIView*)renderView withOrientation:(UIInterfaceOrientation) orientation; - (BOOL)terminate; - (BOOL)isInitialized; - (BOOL)loadScenario:(NSString*) dpdFilename; - (BOOL)playScenario; - (BOOL)pauseScenario; - (BOOL)isScenarioPaused; - (BOOL)registerCommunicationCallback:(NSString*)callbackStringIdentifier callbackObject: (id)callbackObject callbackMethod:(NSString*)callbackMethod; - (BOOL)enqueueCommand:(NSString*)commandName args:(NSArray*) args; - (BOOL)registerNotificationLicense:(id)callbackObject callbackMethod:(SEL)callbackMethod; @end

3.3.2 tiComponent description - (id)initialize:(UIView*)renderView; - (id)initialize:(UIView*)renderView withOrientation:(UIInterfaceOrientation) orientation; The “initialize()” method will initialize the D’Fusion Mobile SDK components and instantiate all the necessary objects for their execution. The ‘renderView’ argument is an UIView object that must be allocated and initialized before calling this function. The ‘orientation’ argument is the required D’Fusion rendering orientation; only Landscape Right and Portrait orientation are managed. ATTENTION! ATTENTION, only "Portrait" orientation is supported by D'Fusion through the User Interface configuration (UIView). D'Fusion landscape rendering orientation can be set with the method "initialize:(UIView*)renderView withOrientation:(UIInterfaceOrientation) orientation" but it is not recommended

- (BOOL)terminate; The ‘terminate’ method will close the D’Fusion Mobile SDK components and deallocate their internal data.

- (BOOL)loadScenario:(NSString*)dpdFilename; The ‘loadScenario’ method will load the D'Fusion Studio project file (.dpd extension) specified in the ‘dpdFilename’ argument. TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

14 / 65


D’Fus ion  Mobile User guide

2011-12-07

- (BOOL)playScenario; The ‘playScenario’ method will enable project rendering.

- (BOOL)pauseScenario; The ‘pauseScenario’ method suspends the project rendering.

- (BOOL)isScenarioPaused; The ‘isScenarioPaused’ method reflects the current state of the project playback.

- (BOOL)registerCommunicationCallback:(NSString*)callbackStringIdentifier callbackObject:(id)callbackObject callbackMethod:(NSString*)callbackMethod; The ‘registerCommunicationCallback’ method will register an iOS notification name - the string ‘callbackStringIdentifier’ - for later communication between the Lua script and the main application. An NSNotification with this identifier will be sent when a Lua script will post a communication command with the callbackMethod’ string. The ‘callbackObject’ is the object that will be assigned to the object property of the sent notification.

- (BOOL)enqueueCommand:(NSString*)commandName args:(NSArray*)args; The ‘enqueueCommand’ method sends to the Lua script a command represented by a string ‘commandName’ with possible arguments ‘args’. Note that the commandName’ is not a name of a Lua script method, but a string that will be returned when the ‘componentInterface:pullCommand’ will be called in Lua. It’s up to the you to define the required commands, the way they are parsed within the Lua script and the actual communication sequencing that involves ‘enqueueCommand’ on the main application-side and ‘componentInterface:pullCommand’ on the Lua script-side.

- (BOOL)registerNotificationLicense:(id)callbackObject callbackMethod:(SEL)callbackMethod; The ‘registerNotificationLicense’ method registers a user-defined method for license-related notifications reception. Calling this method overrides the default license behavior (default UI message).

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

15 / 65


D’Fus ion  Mobile User guide

2011-12-07

3.4 Interface for Android 3.4.1 tiComponent description In your application, in order to communicate with the D'Fusion Studio Suite engine, you manipulate a ti.dfusionmobile.tiComponent object:

tiComponent(Activity currentActivity) That's the constructor, it loads the library.

boolean tiComponent::initialize(FrameLayout frameLayout) boolean tiComponent::initialize(FrameLayout frameLayout, int cameraWith, int cameraHeight, int cameraFps) It creates the 3D view and the Android videocapture. ‘frameLayout’ must be a valid (= initialized) object. ‘cameraWith’, ‘cameraHeight’ and ‘cameraFps’ are optional parameters for Android’s camera mode adjustment. If they are not filled, as in majority of cases, the Android’s camera object is opened with default parameters: 320x240 at 15fps.

void tiComponent::onPause() void tiComponent::onResume() These methods must be called if the activity is paused or resumed. These methods don't "play/pause" the application but 3D resources are unloaded or reloaded because when the activity is paused, the GL context can be destroyed.

void tiComponent::terminate() Call this method before you quit the application. Videocapture and 3D rendering are stopped. Once you have called this method, you must call tiComponent::initialize() to reinitialize the component.

boolean tiComponent::isInitialized() Returns true if the component has been initialized.

boolean tiComponent::loadScenario(String filename) Loads the filename project. This method returns true if the scenario has been correctly loaded. If your scenario is in the "assets" directory, you must execute the following code in your java file:

ApplicationInfo appInfo = null; PackageManager packMgmr = getApplicationContext().getPackageManager(); try { appInfo = packMgmr.getApplicationInfo(getPackageName(), 0); } catch (NameNotFoundException e) { e.printStackTrace(); throw new RuntimeException("Unable to locate assets, aborting..."); } String dpdfile = appInfo.sourceDir + "assets/myproject.dpd"; _tiComponent.loadScenario(dpdfile); TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

16 / 65


D’Fus ion  Mobile User guide

2011-12-07

boolean tiComponent::pauseScenario() boolean tiComponent::playScenario() boolean tiComponent::isScenarioPaused() These functions play/pause the scenario.

boolean tiComponent::enqueueCommand(String commandName, String [] args) EnqueueCommand method allows you to send a command with arguments to the Lua script for execution of the required script instructions. Notice that methodName is not the name of a Lua method but a string that will be provided by the componentInterface:pullCommand Lua method and that can be tested:

-- receive a command from java code : isCommand, command = componentInterface:pullCommand() if isCommand then if command["CommandName"]=="ScaleRobot" then doSmthgWithArgs(command["arg0"],command["arg1"], command["arg2"]) end end boolean tiComponent::registerCommunicationCallback(String functionId, Object object, String methodName) This function registers the "methodName" method on the Object "object" with the id "functionId". When the Lua script executes something like :

componentInterface:executeAppFunc("functionId", string.format("%d", trackingCount), string.format("%f", trackingTime)) object.methodName(String [] args) is called. The "methodName" method must use only a String tab as parameter and must return "void". The method returns true if the "methodName" method has been correctly registered.

void tiComponent::activateAutoFocusOnDownEvent(boolean activate) Activate or deactivate the auto-focus when the user touches the screen. Auto-focus can be useful to improve tracking.

void tiComponent::doCameraFocusNow() Force the Android OS camera to execute the focus sequence immediately.

String tiComponent::getVersion() This method returns the version of the current D'Fusion Mobile SDK component.

boolean tiComponent::registerNotificationLicense(Object callbackObject, String callbackMethod) This method registers a user-defined Java method for a license-related message callback. Calling this method overrides the default license behavior (default UI message).

void tiComponent::setCustomUserAppDataDirectory(String filepath) This method allows you to define a custom UserAppDirectory. This directory can be retrieved in Lua with the getUserAppDataDirectory() function. By default getUserAppDataDirectory is /sdcard/ti/<app.id>/.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

17 / 65


D’Fus ion  Mobile User guide

2011-12-07

3.5 Component execution sequencing The following figure presents a typical execution sequence of D’Fusion Mobile SDK components, involving an allocation of a component instance, initialization within the application context, loading of a D’Fusion Studio project, and an example communication between the project’s Lua scripts and the main application.

Figure 5: Component execution sequencing

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

18 / 65


D’Fus ion  Mobile User guide

2011-12-07

4 PROJECT CREATION The following sections describe D’Fusion Studio project creation recommendations specific to mobile platforms. For more information on how to create AR scenarios with D’Fusion Studio, see the relevant documentation [01].

4.1 General recommendations • Never modify manually the project.dpd file that is generated by D’Fusion Studio. Otherwise, you could encounter problems and potentially (and irreversibly) damage your project. • Use the getOSType() Lua function to retrieve the current OS type. This permits you to configure your Lua script execution as conditional, depending on the actual runtime platform. For example, not all platforms have complete functionalities: GPS, accelerometer or touchscreen can be lacking on a PC or mobile platforms do not have mouse input devices. • D’Fusion Studio project settings are not taken into account for mobile targets. It’s the dedicated mobile project (developed with Xcode for iOS or Eclipse for Android) that will set the application’s window rendering sizes. • Use the “Lite” mode when working with the D’Fusion Studio “Computer Vision” tool and follow the recommendations described in chapter 4.4. It is particularly important to use only “Planar Autoinit” objects when targeting mobile platforms.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

19 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.2 Video configuration file Apple and Android mobile devices use specific video configurations files. Several video configuration file presets for mobile platforms are available from the D’Fusion Studio interface > “Video” panel > “File” menu > “New” command. As an example, two presets are listed below in sections 4.2.4 and 4.2.5. You can do all your development and tests with the video configuration of your webcam (i.e. the camera installed on the Windows PC used for project-development), then use the option “Alternative platform” in the video capture’s “Object Editor” window (in D’Fusion Studio) to set a different video configuration file for the Android and for the iOS platforms. During development, consider configuring your (Windows) webcam video capture with the same values set for your final (mobile) video capture. For example, if you use a resolution of 320x240 in your final video configuration file, you should also use 320x240 as the resolution of your webcam (so as to simulate final video conditions during development).

4.2.1 General recommendations Please follow these recommendations for best all-over performances: • Use 320x240 image dimensions, although better resolutions can be used on high-end devices. • Frame rate (FPS): use 15 FPS for low-end devices. For most recent, high-end devices, you can go up to 20, 25, or even 30 FPS. Higher FPS means more fluent video, but the all-over performance also on your D’Fusion Studio project content and on the actual tracking configuration. • Delay: •

Delay is not taken into account on single-core mobile platforms (execution is synchronous), so there is no need to specify it.

On devices having 2 (or more) cores, the tracking can be executed in a separate thread; in this case, the delay value is effectively taken into account and should be adjusted according to your target device performance. In general, we recommend 30 FPS and a delay of 3 frames.

4.2.2 Front versus back camera Front- and back-camera selection is done via the <VIDEO_CAPTURE_FRIENDLY_NAME> tag in the video configuration file (see also templates below). Two values are allowed: “front” and “back”. For a front-camera selection use: <VIDEO_CAPTURE_FRIENDLY_NAME>front</VIDEO_CAPTURE_FRIENDLY_NAME> and for back-camera selection use: <VIDEO_CAPTURE_FRIENDLY_NAME>back</VIDEO_CAPTURE_FRIENDLY_NAME>

4.2.3 No Video Input If the mobile device does not have the requested device (e.g. does not have a front camera, but the xml video configuration file requires it) the video capture will not be opened. In this case, the image “No Video Input” will be displayed instead of the video on the background texture:

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

20 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.2.4 Typical iOS video configuration file (back camera) We recommend using BGR32_1 pixel format for performance reasons (possible BGR32_1 or NV12). The following code describes a typical video configuration file for an Apple mobile device, taken from the “videoConfig_iOS_backcam_320x240_15fps.xml” preset:

<?xml version="1.0" standalone="yes" ?> <TI> <VIDEOCAP> <COMMENTS>iOS_backcam_320x240_15fps</COMMENTS> <VIDEO_CAPTURE_FILE></VIDEO_CAPTURE_FILE> <VIDEO_CAPTURE_NUM_DRIVER>0</VIDEO_CAPTURE_NUM_DRIVER> <VIDEO_CAPTURE_PIXEL_FORMAT>BGR32_1</VIDEO_CAPTURE_PIXEL_FORMAT> <VIDEO_CAPTURE_WIDTH>320</VIDEO_CAPTURE_WIDTH> <VIDEO_CAPTURE_HEIGHT>240</VIDEO_CAPTURE_HEIGHT> <VIDEO_CAPTURE_DELAY>1</VIDEO_CAPTURE_DELAY> <VIDEO_CAPTURE_RATE>15.000000</VIDEO_CAPTURE_RATE> <VIDEO_CAPTURE_NB_FRAME_BUFFERS>4</VIDEO_CAPTURE_NB_FRAME_BUFFERS> <VIDEO_CAPTURE_INTERLACED>0</VIDEO_CAPTURE_INTERLACED> <VIDEO_CAPTURE_INVERTED>1</VIDEO_CAPTURE_INVERTED> <VIDEO_CAPTURE_PRIORITY>3</VIDEO_CAPTURE_PRIORITY> <VIDEO_CAPTURE_TYPE>0</VIDEO_CAPTURE_TYPE> <VIDEO_CAPTURE_NAME></VIDEO_CAPTURE_NAME> <VIDEO_CAPTURE_CPU>0</VIDEO_CAPTURE_CPU> <VIDEO_CAPTURE_PAUSE>1</VIDEO_CAPTURE_PAUSE> <VIDEO_CAPTURE_DEINTERLACE>1</VIDEO_CAPTURE_DEINTERLACE> <VIDEO_CAPTURE_SHADER>0</VIDEO_CAPTURE_SHADER> <VIDEO_CAPTURE_FRIENDLY_NAME>back</VIDEO_CAPTURE_FRIENDLY_NAME> <VIDEO_CAPTURE_DEBUG>0</VIDEO_CAPTURE_DEBUG> <VIDEO_CAPTURE_RATE_IS_DYNAMIC>true</VIDEO_CAPTURE_RATE_IS_DYNAMIC> <VIDEO_CAPTURE_RESYNCHRO_PERIOD>0</VIDEO_CAPTURE_RESYNCHRO_PERIOD> <VIDEO_CAPTURE_STEADY_STATE_MARGIN>0.330000</VIDEO_CAPTURE_STEADY_STATE_MARGIN> <VIDEO_CAPTURE_TIMECODE_REINIT_PERIOD>0.500000</VIDEO_CAPTURE_TIMECODE_REINIT_PERIOD> </VIDEOCAP> </TI>

Available resolutions are 192x144, 320x240, 480x360 and 640x480 (if the device allows it). As the 320x240 is a cropped version of the 480x360, there is a “zoom” effect in the video capture. It is strongly recommended to use the 320x240 resolution. Resolutions higher than 320x240 consume too much processing power and will impact the overall performance. Only the iPad2 device is able to handle a 640x480 resolution.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

21 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.2.5 Typical Android video configuration file (back camera) We recommend using the Android’s native video capture NV12 pixel format. The following code describes a typical video configuration file for the Android device, taken from the “videoConfig_android_backcam_320x240_15fps.xml” preset:

<?xml version="1.0" standalone="yes" ?> <TI> <VIDEOCAP> <COMMENTS>android_backcam_320x240_15fps</COMMENTS> <VIDEO_CAPTURE_FILE></VIDEO_CAPTURE_FILE> <VIDEO_CAPTURE_NUM_DRIVER>0</VIDEO_CAPTURE_NUM_DRIVER> <VIDEO_CAPTURE_PIXEL_FORMAT>NV12</VIDEO_CAPTURE_PIXEL_FORMAT> <VIDEO_CAPTURE_WIDTH>320</VIDEO_CAPTURE_WIDTH> <VIDEO_CAPTURE_HEIGHT>240</VIDEO_CAPTURE_HEIGHT> <VIDEO_CAPTURE_DELAY>1</VIDEO_CAPTURE_DELAY> <VIDEO_CAPTURE_RATE>15.000000</VIDEO_CAPTURE_RATE> <VIDEO_CAPTURE_NB_FRAME_BUFFERS>4</VIDEO_CAPTURE_NB_FRAME_BUFFERS> <VIDEO_CAPTURE_INTERLACED>0</VIDEO_CAPTURE_INTERLACED> <VIDEO_CAPTURE_INVERTED>1</VIDEO_CAPTURE_INVERTED> <VIDEO_CAPTURE_PRIORITY>3</VIDEO_CAPTURE_PRIORITY> <VIDEO_CAPTURE_TYPE>0</VIDEO_CAPTURE_TYPE> <VIDEO_CAPTURE_NAME></VIDEO_CAPTURE_NAME> <VIDEO_CAPTURE_CPU>0</VIDEO_CAPTURE_CPU> <VIDEO_CAPTURE_PAUSE>1</VIDEO_CAPTURE_PAUSE> <VIDEO_CAPTURE_DEINTERLACE>1</VIDEO_CAPTURE_DEINTERLACE> <VIDEO_CAPTURE_SHADER>0</VIDEO_CAPTURE_SHADER> <VIDEO_CAPTURE_FRIENDLY_NAME>back</VIDEO_CAPTURE_FRIENDLY_NAME> <VIDEO_CAPTURE_DEBUG>0</VIDEO_CAPTURE_DEBUG> <VIDEO_CAPTURE_RATE_IS_DYNAMIC>true</VIDEO_CAPTURE_RATE_IS_DYNAMIC> <VIDEO_CAPTURE_RESYNCHRO_PERIOD>0</VIDEO_CAPTURE_RESYNCHRO_PERIOD> <VIDEO_CAPTURE_STEADY_STATE_MARGIN>0.330000</VIDEO_CAPTURE_STEADY_STATE_MARGIN> <VIDEO_CAPTURE_TIMECODE_REINIT_PERIOD>0.500000</VIDEO_CAPTURE_TIMECODE_REINIT_PERIOD> </VIDEOCAP> </TI>

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

22 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.3 Camera calibration file Important : A correctly configured camera calibration file is crucial for your application ! A correct camera configuration for a given platform guarantees correct tracking results and the coherency between the real world and the 3D virtual objects. Apple and Android mobile devices use specific camera calibration files. Several camera calibration file presets for mobile platforms are available from the D’Fusion Studio interface > “Camera Calibration” panel > “Camera” menu > “New” command. As an example, two presets are listed below in sections 4.3.1 and 4.3.2. You can do all your development and tests with the camera calibration of your webcam (i.e. the camera installed on the Windows PC used for project-development), then use the option “Alternative platform” in the camera’s “Object Editor” window (in D’Fusion Studio) to set a different camera calibration file for the Android and for the iOS platforms. During development, consider calibrating your (Windows) webcam camera with the same values used for your final (mobile) camera calibration. For example, if you use a sensor size of 320x240 in your final camera calibration file, you should also use 320x240 as the sensor size of your webcam (so as to simulate final camera conditions during development).

4.3.1 Typical iOS camera calibration file (for 320x240 videoConfig) The following code corresponds to the “cameraCalib_iOS_320x240.xml” preset:

<?xml version="1.0" standalone="yes" ?> <CAMERABASE> <CAMERABASE> <COMMENTS>iOS_320x240</COMMENTS> <MODELE>OPENCV_0</MODELE> <MAXX>320</MAXX> <MAXY>240</MAXY> <PX>160</PX> <PY>120</PY> <FX>545</FX> <FY>545</FY> <A1>0</A1> <A2>0</A2> <P1>0</P1> <P2>0</P2> <DEDISTORTION>false</DEDISTORTION> </CAMERABASE> </CAMERABASE>

If the resolution in the video configuration file is different from 320x240, replace the “FX” and “FY” values with 363.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

23 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.3.2 Typical Android camera calibration file (for 320x240 videoConfig) The following code corresponds to the “cameraCalib_android_320x240.xml” preset: <?xml version="1.0" standalone="yes" ?> <CAMERABASE> <CAMERABASE> <COMMENTS>android_320x240</COMMENTS> <MODELE>OPENCV_0</MODELE> <MAXX>320</MAXX> <MAXY>240</MAXY> <PX>160</PX> <PY>120</PY> <FX>330</FX> <FY>330</FY> <A1>0</A1> <A2>0</A2> <P1>0</P1> <P2>0</P2> <DEDISTORTION>false</DEDISTORTION> </CAMERABASE> </CAMERABASE>

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

24 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.4 Computer Vision requirements • In D’Fusion Studio “Computer Vision” panel, use the “Lite” tracking mode:

It is mandatory to use directly your flyer image as input. This means that you have to use the “Planar Autoinit” object. Do not use a simple “Planar” object, which has to be created using your web-cam, since it surely has different optics/calibration parameters than the targeted mobile device camera. • It is possible to define and use more than one keyframe image for a “Planar Autoinit” object. • It is possible to use more than one “Planar Autoinit” object. • Keyframe image requirements: Format: .jpg is supported Ratio: keep the same ratio as the ratio of the real-world image to be tracked Resolution: in order to obtain a correct recognition, the image has to be imperatively smaller than the resolution of the video capture (e.g. smaller than 320x240) and its dimensions in pixels should correspond to the dimensions of the zone where the final user will naturally try to recognize (e.g. about 80 % of the video capture dimensions). When doing the resize, do not forget to preserve the image ratio. • If you modify your flyer, you must regenerate D’Fusion Studio computer vision files; in other words, you have to suppress the keyframe and re-import the image flyer. • We recommend: -

1 object tracked at the same time

-

4 keyframes maximum for one project being executed, mainly because of the performance incidence where more used keyframes results in a less final FPS rate of the application. Note that these 4 keyframes are supposed to be a total count for whole your MLT configuration in execution, but the actual configuration is up to your convenience. You can have:

TOTAL IMMERSION

-

either 4 objects, each having 1 keyframe,

-

or 2 objects, each having 2 keyframes,

-

or 1 object with 4 keyframes,

-

etc.

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

25 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.5 Media requirements Mobile devices have a computation capacity substantially smaller than standard PCs, and it is therefore required to use media that is adequate to these devices. Furthermore, the 3D engine used by D’Fusion Mobile is Ogre3D, OpenGL-ES 1.X, which is different from the one used for PCs and Macs and requires some additional considerations.

4.5.1 3D requirements • Export in “public” mode. Use the D'Fusion Exporter and export your .scene in "Public" mode. (See documentation [06 & 07] and [08 & 09] for more information on Public mode) • Use only textures having a size power-of-two • Avoid big textures – try to adapt the texture size and make it coherent with what will be finally rendered E.g. you don’t need big textures for a static panel finally rendered into a zone having dimensions of dozens of pixels • Don't use shaders • Don’t use shadows • 8000 triangles max if your scene is not animated. 4000 triangles max if your scene is animated • Skeletal animations are limited as most of the time they are not hardware accelerated (depending on the device).

4.5.2 Particle Systems • Particles: when using particles, be sure that your particle system description begins with the "particle_system" keyword. This is required for D'Fusion Mobile only. particle_system Particle/Firework { quota 250 material Material/Firework particle_width 2 particle_height 2.5 ... etc … }

4.5.3 Videos • Release 3.20 now supports video texture (texturation using a video file) on iPhone with the following limitations: o

No sound

o

Native iPhoneOS video codecs

o

No streaming

o

The video file must not be contained in a zip/archive file

Note: you cannot videotexture an object in real-time with an HD video file. Instead, use a video resolution of 320x40 for iPhone and 640x480 for iPad2, while take account of performance constraints. Video textures are not supported for Android.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

26 / 65


D’Fus ion  Mobile User guide

2011-12-07

• But, you can always play a video file in your main application by using a platform-specific player (under ObjectiveC / Java). In this case, it’s not the D’Fusion Mobile SDK that handles the video; it is up to the main application to ensure video playback.

4.5.4 Sounds • .ogg files are supported • In order to obtain smaller project size, we recommend encoding sound files with only one channel (mono).

4.5.5 Further media considerations Framerate can heavily depend on the number of triangles, materials, lights, textures, skinned animations. You should consider the following points to optimize framerate: • The biggest constraint is the animation done by skinning: the skeleton used for animation must minimize the number of nodes and joints per node. • The batches number in a mesh must be minimized. If some non animated objects are using the same material, they must be regrouped in a single mesh. •

The light number must be minimized. Light type preferred order is: directional, then point, then spot.

The material number must be minimized.

The texture number must be minimized.

The use of multi-textured material must be well considered as it can really slow the framerate.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

27 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.6 Lua recommendations • Avoid unused script files in your project directory, it increases the weight of your final package and slows down the download of your application from the market/appstore. • Note that when developing your scenario with D’Fusion Studio you are using a Windows PC. Your scenario is therefore executed on a different hardware/software (e.g. Windows with x86-like processor) than the final targeted one (iOS/Android with ARM processor). Furthermore, you will be unable to use GPS, compass and accelerometer. To avoid issues during development, you can use the following functions to identify if the given platform supports a hardware-specific feature:

bool

isCompassDeviceAvailable ()

Returns true if a compass is available on the current device.

bool

isAccelerometerDeviceAvailable ()

Returns true if an accelerometer is available on the current device.

bool

isGPSDeviceAvailable ()

Returns true if a GPS (or any other location manager system) is available on the current device.

bool

isTouchScreenDeviceAvailable ()

Returns true if a touchscreen is available on the current device. • In your Lua scripts, you are able to use mobile-specific getComponentInterface() function in order to communicate with your high-level application. • In your Lua scripts, you can use the function getOSType() to include some platform-specific lines.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

28 / 65


D’Fus ion  Mobile User guide

2011-12-07

4.7 Testing a mobile scenario on a non-mobile platform During your scenario development, you will most likely want to test your scenario before final deployment. Test can be carried out under D’Fusion Studio, and therefore on a non-mobile operating system (Windows) using a different camera (web-cam) and a different, much faster processor (x86), that is probably multi-core with hyperthreading capabilities (while the mobile devices are in majority of cases mono-core processors that were not designed to execute many threads at the same time). To be as close as possible to a mobile-like platform, please consider the following remarks when testing under D’Fusion Studio: •

Use an external webcam instead of the integrated webcam of a laptop if your mobile is to have a back-camera; by doing this, you will be close to the final conditions where the mobile device webcam will be behind the screen.

Subsequently, do not use the mirror mode.

If possible, test your scenario on the lowest configuration you can find, in order to emulate the hardware capabilities of a mobile device.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

29 / 65


D’Fus ion  Mobile User guide

2011-12-07

5 EXPORT AND TEST KEY GENERATION 5.1 Export with D’Fusion Studio A Test Software Key is automatically generated when your D’Fusion Studio project is exported. This Test Software Key will allow you to test your AR scenario when deploying your mobile application. This section only explains how to export your D’Fusion Studio project. Deployment is explained in section 6.

5.1.1 Choose the correct Target/Platform First of all, make sure that you’re working with the “mobile” target selected:

If this is not the case and there is no “mobile” target registered in your D’Fusion Studio project yet, just add it by choosing “<Add Target>” in the left-hand combo-box:

In the window that pops up, choose “mobile” (default selection). You can copy the settings from another existing target configuration, but it’s not mandatory:

Then, in the second combo-box, select the correct platform you’re working on – either “android” or “iPhoneOS”; or possibly choose the “All Platforms” wildcard token:

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

30 / 65


D’Fus ion  Mobile User guide

2011-12-07

If the platform you actually need to use is not mentioned there (yet), just add it by choosing “<Add Platform>” command in the right-hand combo-box:

Then, in the dialog that pops up, choose the platform you want to add. You can copy settings form any existing mobile configuration, but it’s not mandatory:

5.1.2 Include files for export Include files for export (if it’s not already the case). To do this, select your project’s files that you want to export, make a right-click – a context menu will appear – then, select the “Include in export list” command, as:

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

31 / 65


D’Fus ion  Mobile User guide

2011-12-07

Note that the “Include in export list” is an important step: only files selected for export for the actual Target/Platform will be processed and covered by the Test Software Key.

5.1.3 Export your project To export your project, select “Export” under the “Project” menu:

Make sure you specify the correct “Application ID”: •

For an Apple application, the application ID appears in the "target info" window of your XCode project. Please refer to section 6.1.4.1 “Getting the Application ID (AppID)” for more information.

For an Android application, the application ID is the package name mentioned in your application's “AndroidManifest.xml” file. See section 6.2.4 “Configure the Manifest file”.

You also need to specify a destination (empty) folder for the export. This will be your “exported project folder”. When the project is exported, a Test Software Key is automatically generate. You can check the presence of this Test Software Key (“.dpd.dfk” file) in your “exported project folder”, among all your exported resources:

Now, you can embed this exported scenario into your mobile project (Android only in this example). TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

32 / 65


D’Fus ion  Mobile User guide

2011-12-07

5.2 tiProtectorAR2 tool (Pro only) In some cases, you will need to generate Test Keys for independent data. For example, when your AR scenario has to dynamically download data from a web server, you will need to generate a Test Key for each package to download (so that it can be loaded with the scenario afterwards). The tiProtectorAR2 will allow you to generate Test Keys for independent data: •

Use the tiProtectorAR2 tool on the data to download (and dynamically use). The tiProtectorTool will export this data, i.e. it will generate a Test Software Key for this data and encrypt it if necessary.

Prepare your folder as a package to download using the PrepareFolderForUpdate tool.

Both tools are installed with D’Fusion Studio Pro, in the folder “Deployment > Tools”. This section describes how to create a Test Software Key(s) for specific data using the tiProtectorAR2 (command line) tool.

5.2.1 Using tiProtectorAR2 To use tiProtectorAR2 for D’Fusion Mobile, you must execute the following command line:

tiProtectorAR2.exe parameters.xml [outfolder] –t mobile [-p Platform] [-e Entrypoint ] -slave

The following table reviews the arguments used above: Arguments of the tiProtectorAR2.exe Mandatory

Argument

Comment

YES

parameters.xml

This is the name of an XML file containing the description of the data to download. A sample type is provided below.

YES

outfolder

Folder where all the data to download is to be exported. The software key generated by tiProtectorAR2 will also be placed here.

YES

-t mobile

Option specifying that you’re creating a SWKey for a Mobile target.

Other variables that can be used with the tiProtectorAR2 tool, as well as a sample type for “parameters.xml”, are described in the following section. Mobile-specific use cases can be found in section “5.2.4 Use case examples”.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

33 / 65


D’Fus ion  Mobile 2011-12-07

User guide

5.2.2 tiProtectorAR2 arguments •

parameters.xml: this is the complete path (absolute or relative) to the xml file that references all the data to export, and for which a Test Key is to be generated. You must create this file from scratch with a key-exporting compatible configuration, as shown in the example below: <dfusion version="3.20.xxxxx"> <configurations> <configuration name="@Home|All Platforms" encryption="none" /> <configuration name="mobile|All Platforms" encryption="none" /> <configuration name="mobile|android" encryption="none" /> <configuration name="mobile|iPhoneOS" encryption="none" /> </configurations> <settings> <application applicationid="mobile|android:appId; mobile|iPhoneOS:appId " title="" icon="" /> </settings> <resources> <files> <file relativePath=".\file1.mesh" /> <file relativePath=".\file2.scene" /> ….. </files> </resources> </dfusion>

Each tag specified in the parameters.xml file is described below: <configurations> <configuration> Attribute

Description

name

String description of a “Target|Platform” configuration

encryption

String that defines the configuration is exported

encryption

process

to

apply

when

the

<settings> <application> Attribute

Description

applicationid

String that describes the application IDs for each configuration defined in <configurations> Target|Platform:applicationID;Target2|Platform2:applicationID2

<resources> <files><file> Attribute

Description

relativePath

File path to the data to generate a test key for, relative to the file “parameters.xml”

outfolder: Optional (but recommended). If specified, the files that are listed in your “xml” file are exported into this folder, and encrypted according to the encryption specified for the selected Target and Platform (see next optional arguments). If not specified, the Test Software Key is generated in the folder where your “xml” file is contained, and there is no encryption.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

34 / 65


D’Fus ion  Mobile 2011-12-07

User guide

-e Entrypoint: Optional. If specified, for example “-e car.scene”, the generated Software Key will be named “car.scene.dfk”, and the FIRST media to load from the exported folder must be the “car.scene” media.

-t Target: Optional. If not specified, the first <configuration> specified in the <configurations> list is used. If specified, it corresponds to the Target for which the export has to be done. This Target needs to be listed in the <configurations> tag. Possible values are: @Home

mobile

Pro

-p Platform: Optional. It corresponds to the Platform for which the export has to be done.

-p Platform specified

-p Platform NOT specified

-t Target specified

The first configuration in parameters.xml that matches the specified Target and Platform is used

The first configuration in parameters.xml that matches the specified Target is used

-t Target NOT specified

The first configuration in parameters.xml is used

The first configuration in parameters.xml is used

Possible values for the Platform argument are listed below:

Target

@Home

mobile

Pro

Platform

All Platforms windows macOSX

All Platforms android iPhoneOS

All Platforms windows macOSX

-slave: Mandatory for data to download dynamically. If not specified, the Software Key is marked “master”. If “-slave” is specified, the Software Key is marked “slave”. “Slave” indicates that the set of files for which the test key is to be generated are part of a bigger project, typically the files (media) to download dynamically from a webserver.

The “-slave” option must be set for data to download dynamically. A “master” scenario can ONLY use external data signed as “slave”. A “master” scenario can NOT use external data signed as “master”. A D’Fusion runtime can not load a project signed as “slave”.

5.2.3 Generated keys naming conventions The following points describe the naming conventions used for keys generated with the tiProtectorAR2 tool: If the input xml file is a file named “myinput.xml”, the name of the output Test Software Key is “myinput.xml.dfk” If the input xml file is an xml file named for example “myinput.xml”, and the outfolder option is set to “MyFolderToDownload”, o

If no “entry point” is defined in the “–e” argument, then the key name is “MyFolderToDOwnload.dfk”,

o

If the –e “Entrypoint” argument is specified, then the key name is “Entrypoint.dfk”

Other tiProtectorAR2 arguments combinations are not recommended.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

35 / 65


D’Fus ion  Mobile User guide

2011-12-07

5.2.4 Use case examples Imagine a folder to download organized as follows: + MyFolderToDownload - parameters.xml - file1.scene - file2.mesh - file3.jpg + subFolder1 - file4.mesh - file5.jpg + subFolder1.2 - file6.mesh - file7.jpg + subFolder2 - …

------> ------> ------> ------> ------> ------> ------> ------> ------> ------> ------> ------>

NOT to be exported to be exported to be exported to be exported Empty folder cannot be exported to be exported to be exported Empty folder cannot be exported to be exported to be exported Empty folder cannot be exported NOT to be exported

According to the instructions above, your “parameters.xml” should contain the following code: <dfusion version="3.10.16149"> <configurations> <configuration name="@Home|All Platforms" encryption="none" /> <configuration name="@Home|windows" encryption="none" /> <configuration name="@Home|macOSX" encryption="none" /> <configuration name="mobile|All Platforms" encryption="none" /> <configuration name="mobile|android" encryption="none" /> <configuration name="mobile|iPhoneOS" encryption="none" /> </configurations> <settings> <application applicationid="mobile|android:com.applicationid.testapp; mobile|iPhoneOS:com.applicationid.testapp " title="" icon="" /> </settings> <resources> <files> <file relativePath=".\file1.scene" /> <file relativePath=".\file2.mesh" /> <file relativePath=".\file3.jpg" /> <file relativePath=".\subFolder1\file4.mesh" /> <file relativePath=".\subFolder1\file5.jpg" /> <file relativePath=".\subFolder1\subFolder1.2\file6.mesh" /> <file relativePath=".\subFolder1\subFolder1.2\file7.jpg" /> </files> </resources> </dfusion>

Be carreful: The player rendering engine, Ogre3D, can use downloaded resources only from newly created folders. CONSEQUENTLY, A SCENARIO MUST DOWNLOAD ADDITIONAL DATA INTO A NEWLY CREATED FOLDER ON THE USER COMPUTER.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

36 / 65


D’Fus ion  Mobile User guide

2011-12-07

5.2.4.1 Typical use 1 (RECOMMENDED) When you use the tiProtectorAR2 tool to sign a folder to download, you can use the following command line: tiProtectorAR2.exe MyFolderToDownload\parameters.xml FinalOutName -t mobile –p iPhoneOS -slave The result of the previous command line execution is: + FinalOutName NO CHANGE ALLOWED on the signed files. - FinalOutName.dfk - file1.scene Key name = Parent folder name + “.dfk” - file2.mesh First media to load = any media - file3.jpg + subFolder1 In our example, the parent folder name of the key must always be - file4.mesh - file5.jpg “FinalOutName” we will have to download + subFolder1.2 - file6.mesh - from http://www.mysite.com/.../FinalOutName - file7.jpg - to the user computer into <User App Directory>/…/FinalOutName

5.2.4.2 Typical use 2 When you use the tiProtectorAR2 tool to sign the folder to download, you can use the following command line: tiProtectorAR2.exe MyFolderToDownload\parameters.xml FinalOutName -t mobile –p iPhoneOS –e file1.scene -slave

The result of the previous command line execution is: + FinalOutName - file1.scene.dfk NO CHANGE ALLOWED on the signed files. - file1.scene Key name = Media name + “.dfk” - file2.mesh First media to load = THE Media - file3.jpg + subFolder1 - file4.mesh No constraint on folder names, on the webserver or on the user - file5.jpg computer once downloaded. + subFolder1.2 - file6.mesh In our example, the first media that the scenario must load from the - file7.jpg downloaded folder is “file1.scene”. Be careful: The rendering engine part of the player, Ogre3D, can use downloaded resources only from newly created folders. CONSEQUENTLY, A SCENARIO MUST DOWNLOAD ADDITIONAL DATA INTO A NEWLY CREATED FOLDER ON THE USER COMPUTER.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

37 / 65


D’Fus ion  Mobile User guide

2011-12-07

5.2.5 Usage remarks Other important remarks: •

The .dfk extension is mandatory

There shouldn't be any hidden files (.svn directories, .thumbs.db, etc.) in the folder hierarchy, otherwise, they will be assumed to be part of your data to download.

You should ensure that the output file does not exist within the folder hierarchy. If it does, you should erase it before you launch the executable.

You have to ensure that your configuration file (.xml) has the appID set correctly. The application ID is the Bundle Identifier for iOS (identical to your XCode project settings of the application you are using to test - see section 6.1.4.1 “Getting the Application ID (AppID)”) or the Java package name for Android (which you specify in the AndroidManifest.xml file - see section 6.2.4 “Configure the Manifest file”). To prevent problems between platforms, the identifier should be lowercase and must not contain whitespaces.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

38 / 65


D’Fus ion  Mobile User guide

2011-12-07

6 TEST AND DEPLOYMENT The following sections describe how to deploy a D’Fusion Mobile application.

6.1 Apple applications To deploy an Apple mobile application, you need to follow these steps: 1. Create a new iOS mobile project with XCode 2. Add D’Fusion Mobile SDK components and resources to your XCode project 3. Integrate D’Fusion Mobile SDK in Objective-C code 4. Add your exported scenario to your XCode project 5. Register an Apple device to the AppStore to test your application 6. Build and test your application before submitting it to the AppStore

We recommend the following compiling configuration to submit an app with D'Fusion 3.2: •

D'Fusion: 3.2

Xcode: 4.2

iOS: 5

MacOSX: 10.7

Target iOS: 4.0

You may also check the community website for more up-to-date information.

6.1.1 Create a new iOS project 1. You can create a new iOS project by selecting the “Create a new XCode project” button from the XCode Launch window, or from the XCode menu “File > New Project”

Figure 6: XCode - new project

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

39 / 65


D’Fus ion  Mobile User guide

2011-12-07

2. In the “iOS > Application” panel, select the “Window-based Application” icon and press the button “Choose…”

Figure 7: XCode - window-based application 3. You can create your new project near your D’Fusion Mobile SDK folder. It is not mandatory but convenient.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

40 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.2 Configure your XCode project for D’Fusion Mobile 6.1.2.1 Select Active Architecture Only 1. To open the target configuration window, select the target, open the context menu and click on “getInfo”:

Figure 8: XCode - Get Info 2. In the target configuration menu, check the “build Active Architecture Only” option.

Figure 9: XCode - Build active architecture For new applications developed with D’Fusion 3.2, choose armv7. Add the armv6 architecture ONLY IF you need to update an application already published on the AppStore with armv6.

6.1.2.2 Add D’Fusion Mobile SDK components In order to use the D'Fusion Mobile SDK components, the XCode project must link to the library libDfusionMobile.a. Like other static libraries, it is to be added to the project:

Figure 10: D'Fusion Mobile library in XCode project

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

41 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.2.3 Rename Objective-C extensions in Objective-C++ extensions The D’Fusion Mobile SDK library is partly made of C++ code. It is necessary to link it with the C++ standard libraries. For that, rename at least one of your file sources with the “.mm” Objective-C++ extension.

6.1.2.4 Add D’Fusion Mobile SDK additional resources D’Fusion Mobile SDK comes with additional resources that can be included into your project or left aside (e.g. if you want to have a smaller size application). These files are grouped in the “dfusionres” folder of the D’Fusion Mobile SDK. They include xml files for logging and debugging, and a default font definition file. They are not required for deployment on the AppStore. However, they can be useful during development. If some text is to be displayed in your D’Fusion Mobile application render window, you must embed a font definition in your project. It can be the default one, but you can change it easily for any other TrueType font.

Figure 11: XCode project - resources and Frameworks TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

42 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.2.5 Add frameworks Several frameworks are required by D'Fusion Mobile. These frameworks are shown in the screenshot above. If they are missing, errors will appear when linking. To add these frameworks, select the “Frameworks” folder, open the context menu, and select “Existing Frameworks”:

Figure 12: XCode - Add Existing Frameworks

The frameworks to add are: •

AudioToolbox

AVFoundation

CoreLocation

CoreMedia

CoreVideo

Foundation

OpenAL

OpenGLES

QuartzCore

UIKit

Please note that the AudioToolbox framework is required by D’Fusion Mobile 3.2 and later even if you developed your project with a D’Fusion 3.2-earlier version.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

43 / 65


D’Fus ion  Mobile User guide

2011-12-07

In the framework selection window, select the frameworks to add and click on the “Add” button.

Figure 13: XCode - frameworks to select

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

44 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.2.6 libz dynamic library In addition to frameworks, you will need the libz dynamic library. If missing, errors will appear when linking. To add this library, select the “Frameworks” folder, open the context menu, and select “Existing Frameworks”. Select the libz.dylib library and press the “Add” Buton:

Figure 14: XCode - add libz.dylib to project

6.1.3 Integrate D’Fusion Mobile SDK in Objective-C code This section describes how to integrate D’Fusion Mobile SDK components in the Objective-C code.

6.1.3.1 Application delegate header

The window member is the default member variable. Added member variables are: •

UIView*

tiComponent* mPlayer - the instance of the D’Fusion Mobile SDK component object

mRender - the view in which the D’Fusion rendering will be done

The two methods “start” and “stop” are detailed below. TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

45 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.3.2 didFinishLaunchingWithOptions method The didFinishLaunchingWithOptions method is part of the NSApplicationDelegate protocol. It will be called when the application has finished its loading phase.

The most important line is:

mPlayer = [[tiComponent alloc] initialize:mRender]; This method allocates and initializes the D’Fusion Mobile SDK main component. The mRender argument is the UIView in which the rendering will be done. It must be allocated and initialized: a nil value will make the initialization fail.

6.1.3.3 Creating the UIView programmatically The mRender UIView can be allocated and initialized programmatically:

mRender = [[UIView alloc] initWithFrame:CGRectMake(0, 0, 320, 480)]; [window addSubView:mRender]

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

46 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.3.4 Creating the UIView with Interface builder The UIVIew can also be created and initialized by using the Interface Builder application. In this case, the code to initialize it is not needed. 1. To edit the Graphic interface, double-click on the MainWindow.xib file in the resource group:

Figure 15: XCode - MainWindow.xib

2. The Interface Builder application will open. The next step is to select the UIView in the library window and to drop it on the MainWindow window. This will automatically add the UIView has a child of the main window.

Figure 16: Interface Builder - UIView

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

47 / 65


D’Fus ion  Mobile User guide

2011-12-07

3. In the MainWindow.xib window: 1. Select the view 2. Click on the “Inspector” icon, the “Inspector” window will open 3. In the “Inspector” window, select the “Connections” tab 4. Select the “New Referencing Outlet” 5. Drag and drop it on the My First App Delegate

Figure 17: Interface builder - UIView connection 6. A “mRender” context menu will appear. Select mRender. 7. In the inspector window, the binding will appear:

Figure 18: Interface builder - UIView connection

4. You can now quit “Interface Builder” and save the user interface modification.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

48 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.3.5 Start, stop and dealloc methods. In the “start” method, your AR scenario is loaded by calling the “loadScenario” method of the D’Fusion Mobile SDK component interface. The AR scenario is then started by calling the method “playScenario”. Here is the “start” method:

In the “stop” method, your AR scenario is paused by calling the “pauseScenario” method, after testing that the D’Fusion Mobile SDK component is not already paused (with the “isScenarioPaused” method). Here is the stop method:

In the “dealloc” method, the D’Fusion Mobile SDK component (“mPlayer”) is stopped. Before releasing, the method “terminate” is called on the “mPlayer”. Here is the “dealloc” method:

After that the “dealloc” method, the UIView member is released (this is not necessary if it was created with Interface Builder). Finally, tne UIWindow member is released and the “dealloc” method is called.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

49 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.4 Integrate your AR scenario Once your AR scenario is ready in D’Fusion Studio, you can export it into a dedicated folder (i.e. a folder which will contain all the exported files and project resources). To export your project, you’ll need to get the application ID of your Apple mobile application. Once exported, you’ll be able to integrate your exported scenario into your XCode project. More information on how to export your project is available in the D’Fusion Studio User Guide documentation.

6.1.4.1 Getting the Application ID (AppID) Your Apple mobile application has a unique ID which is required to properly export your D’Fusion Studio project. This ID can be found in the “target info” window. To show target properties, open the target context menu and select GetInfo.

In the Properties tab of the Info window, you will see the identifier field. It contains the Application identifier. For example, for a project name “MyFirstApp”, if you set your application ID to be com.t-immersion.${PRODUCT_NAME:rfc1034identifier} you’ll get com.t-immersion.MyFirstApp as the AppID when your project is built.

The application ID can also be found in the info.plist project file, in the section Bundle Identifier.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

50 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.1.4.2 Adding your AR scenario to the XCode project Select your exported project folder in the finder. Drag and drop the folder in the “Resources” group of your XCode project. This will automatically embed the Scenario folder inside the application bundle, during the building phase.

In the window that appears, select the “Create Folder References for any added folders” option. This option will preserve the folder hierarchy inside the Scenario folder.

Figure 19: XCode - adding your scenario to XCode Resources TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

51 / 65


D’Fus ion  Mobile User guide

2011-12-07

In the Resources group, the Scenario hierarchy will appear. Note that the folders are in blue. This is the color identifier for folders added as groups:

Figure 20: XCode - Scenario hierarchy

6.1.5 Register an Apple device for development You will need to register a mobile device for development to the AppStore (iPhone, iPad or iPod touch) if you want to test your mobile application on a real (physical) Apple mobile device. One possible way to do so is as follows: 1. When the device is connected to your mac, go to the XCode Organizer window (“Window” menu > “Organizer”) and select the connected device:

Figure 21: XCode Organizer window TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

52 / 65


D’Fus ion  Mobile User guide

2011-12-07

2. Then open the context menu and select “Add Device to Provisioning Portal…”

Figure 22: Organizer - Add Device to Provisioning Portal 3. This will register the device to the Apple iPhone/iPad/iPod touch Developer Provisioning Portal, and will download the Team Provisioning Profile necessary to test applications on the Apple device. 4. In the Provisioning panel of the window, you should see the new downloaded profile:

Figure 23: XCode Organizer - Team Provisioning Profile You have successfully registered your device for development to the AppStore. You can now build and test your Apple D’Fusion Mobile application on the registered device before submitting your application to the AppStore. Note: a tattoo will be displayed when testing your application. See chapter “7 Final deployment”.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

53 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.2 Android Application This section explains how to deploy an Android mobile application: 1. Create a new Android project with Eclipse 2. Add your exported AR scenario to your Eclipse project 3. Add D’Fusion Mobile SDK components and resources to your Eclipse project 4. Configure the Manifest and Activity files 5. Build and test your application before submitting it to the Android market

6.2.1 Create a new Android project Create a new Eclipse project:

Figure 24: Eclipse project creation

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

54 / 65


D’Fus ion  Mobile User guide

2011-12-07

Select “Android Project”:

Figure 25: Android project creation

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

55 / 65


D’Fus ion  Mobile User guide

2011-12-07

Select API 7 (OS 2.1):

Figure 26: API 7 (OS 2.1) project selection

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

56 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.2.2 Add your scenario to your project You can start by creating a new Android project with Eclipse, and setting the activity to MainActivity. Then add the contents of your exported D’Fusion Studio project to the “assets” directory of your Android project:

Figure 27: Scenario and D'Fusion resources in the "assets" directory

Note that the application ID of your exported project must match the application ID you specify in the “AndroidManifest.xml” file (see section 6.2.4 “Configure the Manifest file” below). You can also add the D’Fusion Mobile SDK additional resources (“dfusionres” directory) to the “assets” directory. The "dfusionres" directory contains useful .xml files for debugging and a default font. It is advised to not include these files in the final application so as to reduce the size of the package. However, if your AR scenario uses the default font for Text2D, the files default.ttf and DFusion.fontdef have to be copied into your scenario directory.

6.2.3 Add D’Fusion Mobile SDK libraries (.so/.jar files) In the file hierarchy of your Android project, create a "libs" directory and add the D’Fusion Mobile SDK files libtiAndroidAR2.so and dfusionmobilesdk.jar as shown below:

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

57 / 65


D’Fus ion  Mobile User guide

2011-12-07

Figure 28: libtiAndroidAR2.so and dfusionmobilesdk.jar folder organization

In Eclipse, go into the “Package Explorer” and right-click on dfusionmobilesdk.jar: select "Build Path” > “Add to Build Path".

Figure 29: Add dfusionmobilesdk.jar to the build path

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

58 / 65


D’Fus ion  Mobile User guide

2011-12-07

The dfusionmobilesdk.jar is now considered as a "referenced library".

Figure 30: dfusionmobilesdk.jar is considered as a referenced library

6.2.4 Configure the Manifest file You will have to edit the “manifest” file (“AndroidManifest.xml”) with the code below in order for the activity to handle the orientation, keyboard and keyboardHidden changes. Otherwise, the activity will shut down and restart for each of these events. 1 - You can define a screenOrientation to avoid some latency in the application when the orientation changes:

<application android:icon="@drawable/icon" android:label="@string/app_name" android:theme="@android:style/Theme.NoTitleBar.Fullscreen" android:debuggable="true"> <activity android:name=".MainActivity" android:label="@string/app_name" android:screenOrientation="landscape" android:configChanges="orientation|keyboard|keyboardHidden"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER"/> </intent-filter> </activity>

2 - Because of the video capture, you must add the following line in order to guarantee the access to the camera by our activity:

<uses-permission android:name="android.permission.CAMERA" /> <uses-feature android:name="android.hardware.camera" /> 3 - If you enable the auto-focus option (tiComponent::activateAutoFocusOnDownEvent(true)), you have to notify the use of this functionality:

<uses-feature android:name="android.hardware.camera.autofocus" /> TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

59 / 65


D’Fus ion  Mobile User guide

2011-12-07

6.2.5 Configure the Activity file The Activity class is an important part of an application's overall lifecycle. Lifecycle events have an important effect on the D’Fusion Mobile SDK component. See documentation [11] for more information about activity lifecycle.

Use the following “import” declaration to start using the D’Fusion Mobile SDK tiComponent class:

import ti.dfusionmobile.tiComponent; You can now create an instance of tiComponent within the "onCreate" method: public class MainActivity extends Activity { private FrameLayout private tiComponent

_frameLayout; _tiComponent;

/** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { Log.e("MainActivity", "onCreate"); super.onCreate(savedInstanceState); _tiComponent = new tiComponent(this); }

Create and initialize the D'Fusion Mobile SDK component in the "onStart" method: @Override public void onStart() { Log.i("MainActivity", "onStart"); // init the D'Fusion mobile component : _frameLayout = new FrameLayout(this); _tiComponent.initialize(_frameLayout); _tiComponent.activateAutoFocusOnDownEvent(true); // attach to the window : setContentView(_frameLayout); super.onStart(); // load a scenario (_frameLayout and _tiComponent // must be created and initialized) ApplicationInfo appInfo = null; PackageManager packMgmr = getApplicationContext().getPackageManager(); try { appInfo = packMgmr.getApplicationInfo(getPackageName(), 0); } catch (NameNotFoundException e) { e.printStackTrace(); throw new RuntimeException("Unable to locate assets, aborting..."); } String dpdfile = appInfo.sourceDir + "/assets/Scenario1/project.dpd"; _tiComponent.loadScenario(dpdfile); // don't forget to play your scenario: _tiComponent.playScenario(); }

D'Fusion Mobile uses a FrameLayout object to add the camera preview required by some devices and the 3D view. This FrameLayout must be empty (but initialized) when you pass it to the tiComponent::intialize() method. TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

60 / 65


D’Fus ion  Mobile User guide

2011-12-07

After the initialization you can include this FrameLayout in your own Layout. See the SampleInputs for more information. You can define videocapture size and framerate as arguments of the tiComponent::initialize() method. You must use the values defined in your video configuration file. Don't forget to implement onResume/onPause methods. When the activity is paused the GL context is destroyed, so all media is unloaded. GL context is recreated and media is reloaded when the "resume" event is called. @Override public void onResume(){ Log.i("MainActivity", "onResume"); super.onResume(); // we re-create the GL Context and reload the scenario _tiComponent.onResume(); // play the scenario _tiComponent.playScenario(); } @Override public void onPause() { Log.i("MainActivity", "onPause"); super.onPause(); // we pause the scenario _tiComponent.pauseScenario(); // we notify the onPause event to the component : // medias are released but the library is not unloaded. _tiComponent.onPause(); }

Close the component when the activity receives the "stop" event. At this moment, the 3D view and the video capture are stopped and destroyed. They will be recreated in the "onStart" method if needed. @Override public void onStop() { Log.i("MainActivity", "onStop"); super.onStop(); // the component is closed, media are unloaded // 3d view and videocapture are destroyed. _tiComponent.terminate(); }

When the "onDestroy" method is called, we have to clean all data. Next time the activity is passed in, the "onCreate" method will be re-called. @Override public void onDestroy() { Log.i("MainActivity", "onDestroy"); super.onDestroy(); // delete everything. _frameLayout = null; _tiComponent = null; }

You can now build your application and test it on your Android device. Don't forget to look at the “logcat” window. It can contain important information about the application status and errors.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

61 / 65


D’Fus ion  Mobile 2011-12-07

User guide

7 FINAL DEPLOYMENT 7.1 Steps for final deployment Test Software Keys do not allow the deployment of AR applications for commercial use. To deploy a final version of your application, you will need to acquire a “Final” Software Key. “Final” Software Keys are delivered to D’Fusion Studio Suite Pro users only. To request a “Final” Software Key, please contact Total Immersion. Your “Test” Software Key is then to be replaced by the “Final” Software Key, in the AR scenario folder of your mobile application XCode/Eclipse project. To make sure you project reflects the new data, make a complete clean: -

Erase the application from the test device

-

Erase build directories (“build” on iOS, “bin” on Android)

-

Refresh your project

-

Recompile, install and run

7.2 License coherency using multiple Software Keys A Software Key can be in three different modes, depending on the tool used to generate it: “Test” “Test Free” “Final” D’Fusion Studio generates a “Test Free” Software Key when exporting a project. D’Fusion Studio Pro and the tiProtectorAR tool generate “Test” Software Keys. As explained in the previous chapter of the current document, a Master key is a key that can be loaded firstly. A Slave key corresponds to data that can be added dynamically to the master project, and loaded in second time (after a download for example).

License coherency between Software Key modes Slave “Test”

Slave “Test Free”

Slave “Final

Master “Test Free” with..

Slave “Test”

Slave “Test Free”

Slave “Final”

Master “Final” with..

Slave “Test”

Slave “Test Free”

Slave “Final”

Master “Test” with..

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

62 / 65


D’Fus ion  Mobile 2011-12-07

User guide

8 TROUBLESHOOTING & FAQ 8.1 License error messages The D’Fusion Mobile SDK provides a default visual feedback via UI dialogs in the case when license evaluation detects some incoherency / abnormality. These UI messages are shown using pre-defined UI methods, incorporated within the SDK. Note that the main application is able to override this default behavior by specifying a new license notification callback, thus, you are able to change the visual aspect and handle the correct application terminate on the main application level.

License incoherency UI messages License abnormality cause

iOS

Android

Warning : Mixing Software Keys (Final and Test keys are used at the same time) This should not arrive in Final deployment if software keys were correctly generated and validated. It prevents the developer that the TI tattoo will be shown.

Error:

Preview not available

File corrupted. Could not be loaded by the engine. Stops the engine

Error: General license problem. (leaking license, license expired, license not correct) Stops the engine.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

63 / 65


D’Fus ion  Mobile User guide

2011-12-07

8.2 FAQ Q/A: Can I use a .mov/.mp4 file in order to have video-texturation on a mesh? Yes, you can in D’Fusion Mobile for iOS. Video-texturation is not supported in D'Fusion Mobile for Android. Q/A: The video texturation is not possible in D'Fusion Mobile for Android, but if we consider the video playback as an animation (2D), do we can « simulate » a video texturation? You can create a texture-based D'Fusion Mobile for Android scenario, where a dynamic texture change of a 3D textured mesh can create a video-like playback effect. But note that this solution has some disadvantages: • "video" size is limited by the GPU memory, • thus, the possible duration of such "video" is depending on the actual image resolution and the playback framerate • by consequence, an animation is limited to few (<10) seconds Q/A: How does the deployment of a D’Fusion Mobile based application work? Do I need separate applications for iOS and Android platforms? If yes, how do I deploy? A D’Fusion Mobile based application is delivered through a standard process for all mobile applications: AppStore for iOS and Android Market for Android. The integration is different for Android/iOS as a package has to be built and then sent through the normal process to the AppStore. In other words, there are 2 differences: • D'Fusion Mobile is a software component to be integrated in a native application (Java for Android and Objective C for iOS). The developer has to generate the final application using the standard development environment (for instance XCode for iOS). • Before publication, you need to obtain fom Total Immersion a “Final” Software Key, based on your development “Test” Software Key. Q/A: iPhoneOS is surveying the duration of the application launch and kills applications that do not respond under 20 seconds, what can I do to overcome this? You can be confronted to this type of behavior if you compile your application for “Distribution” and if your “applicationDidFinishLaunching” callback takes longer than 20 seconds to execute. This arrives typically when the SDK initialization and the load of the scenario are done within this callback; as it’s the case for the “SamplePlayer” example. To overcome this, 2 possible solutions, both on the application-side and both making the callback “applicationDidFinishLaunching” not-blocking: • Either invoke the SDK initialization / scenario load later during the application execution - once the application graphics user interface is completely loaded and responding: You can do this on some user-triggered action, e.g. a button press. • Or, if really desired to perform the SDK initialization / scenario load just after the application launch, encapsulate them into separate loading thread functionality. In such case, the thread itself can be created and started within “applicationDidFinishLaunching” callback; but, the scenario will be loaded in parallel – neither the callback nor the graphics user interface interaction will be blocked.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

64 / 65


D’Fus ion  Mobile User guide

2011-12-07

Q/A: How to inform Total Immersion of a bug in D’Fusion Mobile? • Synthesize the problem description/symptoms. To signal a problem in the D’Fusion Mobile component so that we are able to act efficiently and rapidly, several elements be provided: -

Make a description of “what happened?” The application is locked, has crashed, or is not working as expected.

-

Describe the context: “How did it happen?” How did the user get the application in that state? What is the event sequence?

-

Make a screen-shot, if possible.

-

Describe the hardware context: what kind of mobile device is it (manufacturer, model, etc.)?

-

What is the operating system (OS) concerned (Android, iOS) and what is its version?

-

Get the corresponding log files. On the Android platform, you can find useful logs in the “logcat” window in Eclipse.

• Signal the problem -

Go to the Total Immersion support web site: http://support.t-immersion.com

-

Copy the log files and join them to the problem description.

-

Do not forget to precise the project you are working on.

• Note that for us, every problematic issue reporting is important and is treated as such. It permits us to understand your problem in details and by consequence, to act rapidly and to find a correct solution for your project needs. We thank you in advance.

TOTAL IMMERSION

22, rue Edouard Nieuport 92150 SURESNES +33 (0) 1 46 25 97 42 www.t-immersion.com

DFusion Mobile - User guide.docrev20662.svn000.tmp.doc

65 / 65

User guide  

DFusion Mobile - User guide

Read more
Read more
Similar to
Popular now
Just for you