Interservice/Industry Training, Simulation, andEducation Conference (I/ITSEC) 2010
The Future Immersive Training Environment (FITE) JCTD: Improving Readiness Through Innovation Pete MuHer
Potomac Training Corp Lansdownc, VA niuller(q).notomaetrainingcorp.com
INTRODUCTION
"Across all warftghtingcommunities, training advances
have been significant, yet the use of advanced simulation technology has not yet achievedfor infantry training what we take as routinefor aviation, armoror maritime simulation training... technology must now he harnessed to bring state-of-the-art simulation to small infantry units. Though the rudimentary simulation designedfor close combat currently affords units some level of challenge, it does notyet approach the level of
professional
role-players
also
portray
scenario
characters and special effects pyrotechnics are used to simulate Improvised Explosive Devices (IEDs) and Rocket Propelled Grenades (RPGs). Special Effects Small Arms Marking System (SESAMS), a militarized
paintball-like system, is fired from modified tactical weapons as well as a custom-built laser system that is attached to the end of each weapon barrel to engage the virtual characters. See Figure 1.
sophistication deemed essential in other disciplines. " —General James N. Mattis, USMC Commander, US
Joint Forces Command (2010)
The demand Irregular Warfare places on small tactical units increases the importance of providing the infantry with sophisticated training and learning. One way to achieve more advanced training and education is through immersive simulation. However, although infantry immersive simulation has been researched for well over a decade, only recently has the technology become mature enough to support effective training.
Figure 1. The IIT at Camp Pendleton
HISTORY
in Camp Pendleton, California, requested that the
Support for an environmentally controlled IIT has received mixed support, primarily due to facility costs
Office of Naval Research (ONR) field a prototype
and a reluctance to look at non-traditional methods. The
In January 2007, I MEF (Marine Expeditionary Force)
training system to prepare infantry Marines deploying to Iraq. ONR had been sponsoring research in infantry
IIT represented only one type of immersion and there are other technologies that need to be examined,
immersion under the VIRtual Technologies and Environments (VIRTE) program and had several
demonstrated, and validated.
prototypes. In examining the requirements of I MEF, it became clear that the full immersion infantry VIRTE
prototype would not meet their needs. This led to the design and development of a new immersive infantry simulation: the Infantry Immersion Trainer (IIT).
The IIT used mixed reality to place virtual characters in selected rooms of a realistic Iraqi village (recently
updated to Afghanistan). It was built in an abandoned 32,000 square foot tomato-packing plant in the San Mateo area of Camp Pendleton. To add to the realism.
2010 Paper No. 10148 Page 2 of 7
JOINT CAPABILITY TECHNOLOGY DEMONSTRATION (JCTD)
In April 2008, a select group of Service experts in simulation and training, led by the United States Joint Forces Command (USJFCOM), proposed a Joint Capability Technology Demonstration (JCTD) program to address the gaps in immersive infantry training. This effort, called the Future Immersive Training Environment (FITE), was awarded $36 million and
Interservice/Industiy Training, Simulation, and Education Conference (1/1TSEC) 2010
given two-years to demonstrate the value of advanced small-unit immersive infantry training. FITE JCTD Overview
As a JCTD, FITE is required to integrate and demonstrate mature technologies. The team proposed three different classes of infantry immersion training technologies: immersive virtual reality, mixed reality, and augmented reality. FITE demonstrated individual immersive virtual reality, first, during its Spiral 1.
move through a virtual environment and perform both individual and squad level tasks in a doctrinally correct manner. Although the Army was the only Service with a formal requirement, Spiral 1 reflected a Joint desire for the technology, including input from the Marine Corps, SOCOM, Air Force, and Navy. The Marine Corps and Air Force security forces, in particular, now have serious interest in dismounted training. Approach
The FITE JCTD team consists of diverse professionals
and subject matter experts drawn from a number of different disciplines including cognitive behavior, decision-making, scenario design, team training, human performance, and infantry training. The team practiced an iterative approach, stressing collaboration, integration, and "give and take" between requirements
SPIRAL 1
and technical capabilities. Baseline Hardware Selection
The requirements of the Army's Dismounted Soldier, particularly those regarding individual field of view, led
Figure 2. Trainees Wear the FITE JCTD individual worn virtual reality systems during a Spiral I Demonstration at Camp Lejeune, NC Spiral 1 focused on individually worn immersive virtual reality systems (see Figure 2), and the training emphasized decision-making in squad-sized elements (9-13 man). Each person was equipped with an individual worn virtual reality system. The environment
was completely computer-generated and presented to the trainee through a Head Worn Display (HWD) and headphones. The trainee moves through the virtual environment experiencing sensory cues and indicators,
allowing him to make sense of the situation and develop shared situational awareness with his teammates. The individual systems are wireless networked, so that the entire squad is in the same virtual environment and can train together. The following section documents the process of integrating, demonstrating, and testing this technology. Requirements
Before the JCTD was officially approved, the FITE Operational Manager hosted a series of workshops to bring stakeholders together and formally document requirements. The Army had already articulated the need
for
Dismounted
Soldier,
an
un-tethered
dismounted infantry system allowing the Soldier to
2010 Paper No. 10148 Page 3 of 7
to the selection of an immersive virtual reality technical solution. ONR's extensive experience in infantry immersive virtual reality, from VIRTE Demo 2, helped
provided a solid foundation for this effort (see Schaffer. 2006).
We selected the Quantum3D ExpeditionDl as the baseline hardware platform. The system includes a body-worn computer, an eMagin 800 x 600 pixel Organic Light Emitting Diode (OLED) microdisplaybased Helmet Mounted Display, three InterSense InertiaCube 3-Degree of Freedom (DOF) sensors, a realistic replicated weapon, and a built-in wireless network. Both the Army and Marine Corps have performed experiments and Small Business Innovative Research (SBIR) programs with versions of the ExpeditionDl system. Our focus was on demonstrating a capability and identifying requirements that best support infantry immersive training and in no way indicates support to any one system or approach. Baseline Software Selection
Both the Marine Corps and the Army have purchased enterprise licenses for Virtual Battlespace 2 (VBS2), a commercially available system from Bohemia Interactive. Although earlier versions of VBS2 had been run on the ExpeditionDl hardware, it had never been demonstrated at the squad level.
Interservice/lndustryTraining, Simulation, and Education Conference (1/ITSEC) 2010
Scenario Development: Cognitive Task Analysis
Haptics Integration
In order to demonstrate the technology, operationally relevant training material needed to be crafted. With the short amount of time to deliver a training capability, the team had to develop an innovative method to rapidly determine important sensory cues in the training environment, cues that recalled previous experiences and triggered critical decision-making. The FITE JCTD team was fortunate enough to have access to infantry Marines and soldiers recently returned from Iraq and Afghanistan. We held workshops and conducted extensive interviews at Camp Pendleton, Camp Lejeune, and at Fort Campbell in order to develop a deep understanding of the operating environment for small units. The results of these workshops provided combat experiences for developing training scenarios.
Our experience from the IIT has shown the benefit of using SESAMS to provide visual and physical feedback
Scenario Development: Technical Needs
The technical team insisted on very early definitions of the training scenarios. The actual scenario was not important, but it was crucial to determine requirements for critical visual and auditory cues to see how well they could be represented in the virtual environment. If, for example, the scenario called for a character to light a cigarette, this would begin a dialog between the operational and technical teams. First, we would
determine if lighting a cigarette was currently supported in VBS2. Then we would examine what the critical
cues were in this event. Is it the actual lighting of the cigarette? Is it the smoke? Is it the heat from the tip of the cigarette when viewed with a thermal vision device? Is it the movement from the hand to the mouth? Is it the
way the cigarette is held in the hand? Are there other cues that have a similar effect that might be easier to implement in the software? Given the limited development time and budget, this approach allowed
the implementation of the higher priority training cues. It was not an elegant process, but it brought simulation issues to support training to the top of the priority list. A similar process was followed for other products that had the potential to improve the training. In addition to
to the trainee that he has been hit with a bullet. In
addition, the IIT has an Advanced Interactive Systems (AIS) instructor remote controlled ShootBack速 Cannon that fires plastic balls down an exposed corridor (AIS, 2009). This system has also demonstrated success. In order to provide physical feedback to the trainee that he has been hit with a bullet in a fully virtual world, we experimented with several different technologies. PhaseSpace, a small technology company, worked with the Worcester Polytechnic Institute to develop a prototype modular vest that uses computer controlled vibro-tactile motors. While this approach has been used in a number of research programs, it is primarily used as an alternative information channel (Lindeman, 2006). With early prototype testing at Camp Pendleton, our young Marines told us they wanted a haptic signal that was more like an actual bullet. A vest designed for gamers from TN Games was then tested that uses compressed air to drive actuators to simulate a bullet hit. Although promising, trainees still wanted a bigger effect.
We examined devices that use high voltage electrical discharges to provide feedback. We obtained an Electronic Prisoner Transport Belt from NOVA-USA that provides 50,000 volts for 6 seconds when activated. One of the authors tried it and can attest that it is
painful and incapacitating to the extent that it is not suitable for use in a training environment. In addition, the possession or sale of devices in this class is restricted or outlawed in some U.S. cities and states. Next
we
evaluated
the
Threat-Fire
from
VirTra
Systems, which, unlike the Prisoner Transport Belt, was specifically designed for use in a training environment. It features computer-controllable shock durations, in the range from 30 milliseconds to 2.5 seconds, a wireless interface, and an API allowing integration with the FITE virtual environment.
looking at systems from traditional Department of Defense vendors, we also looked at research in
universities and emerging products from the entertainment industry. Early exposure of prototypes and concepts to young Marines and soldiers was also helpful in concentrating on high payoff items. To illustrate how this process worked, we will discuss the integration of Haptics into the system.
As this haptics example illustrates, the FITE team utilized a test-integrate-test approach to exploring the technology; this allowed us to examine many different types of technologies and clearly assess their potential. SPIRAL 1 SOFTWARE DEVELOPMENT
After examining the requirements, it became clear that neither VBS2 nor any single current gaming system could meet all the requests. Therefore, VBS2 needed to be extended. As part of their enterprise licenses, both
2010 Paper No. 10148 Page 4 of 7
Interservice/lndustry Training, Simulation, andEducation Conference (I/ITSEC) 2010
the Army and Marine Corps have active, independent development programs with Bohemia Interactive to improve VBS2. We worked closely with both Services to ensure that we coordinated the development efforts.
Although VBS2 has a wealth of weapons, ammunition, optics, and computer generated forces (CGF) available, the US DoD has never formally validated them. A
SPIRAL 1 DEMONSTRATIONS
The FITE team held a number of integration events
around the country, although many were focused in Camp Pendleton because of the ready access to Marines. The first technical demonstration was held in
late September 2009 at Camp Pendleton, CA (see Figure 3). Then in spring 2010. official Operational
validation of VBS2 entities and behaviors is well
Demonstrations were held at
beyond the scope of the FITE JCTD. We did make specific improvements to VBS2 to support FITE's
Simulation Center at Camp Lejeune, NC and a Close Combat Tactical Training (CCTT) Facility in Fort
the II MEF Battle
unique requirements.
Benning, GA.
Terrain
We examined a number of small preexisting terrain databases that are suitable for infantry squad operations
and chose a geotypical Afghan village. This required no software extension.
Optics Improvements We did, however, decide to address improvements to
optics since they play a critical role in observation skills. We took a portion of the technical team to the Electro-Optical Test Facility at Marine Corps Systems Command Product Group (PG) 13 in Quantico, VA to examine both the current weapons and optics while simultaneously examining their representation in VBS2. This led to improvements to the optics modeling in
VBS2 including corrections of the magnification of optics and the addition of glint from the sun. Character Improvements
We made several improvements to the VBS2 virtual characters. One of these was for characters to have
different heights and body types. Another was the addition of cultural gestures, such as smoking and lighting cigarettes. We also added insurgent and suicide bombers.
Figure 3. FITE Technical Demonstration (2009)
After Action Review Improvements
For the demonstrations, all of the gear was shipped in
In addition to run-time improvements, we also spent
transport cases, with the goal of going from cases to training in less than an hour. Each squad of trainees was
significant effort improving the After Action Review capability. Two specific AAR capabilities that were recommended by the Army Research Institute included displaying the characters field of view and adding "inkspots" which surround a character and grow larger based on how long the character stays in one place.
2010 Paper No. 10148 Page 5 of 7
placed in a large room, and each infantryman had a dedicated space (6'x6'), marked by tape on the floor. This constraint is to prevent physical contact between trainees in case they make violent or sudden movements as part of their immersion. The scenarios were relatively short, lasting approximately 50 minutes. Each scenario had a number of "decision points," each of which leads to multiple potential outcomes depending on the decisions made. Then, following completion of a scenario, an integrated AAR was
Interservice/lndustry Training, Simulation, and Education Conference (I/ITSEC) 2010
immediately available
to
replay
any
actions
or
2. Effective Training Environment
communication.
SPIRAL 1 ASSESSMENT
An important part of any JCTD is independent assessment. The Joint Technology Assessment Activity (JTAA) performed an independent assessment of the
All forty-four trainees agreed: FITE could help provide better combat preparation. They also reported that the playback effectively supported AARs. 3. Suitability for the Operational Environment
FITE JCTD to determine if it provides operational utility to the trainees. Four investigative issues provided the framework for determining the operational utility
The majority of the trainees agreed that equipment was comfortable and easy to use. There were very few
for Spiral 1:
several technical issues documented during the demonstration, primarily involving VBS2, HeadMounted Displays, body trackers, and weapons. Most of the issues could be addressed locally with on-site spares and system reboots, and none of the issues substantially effected the execution of the
1. Does FITE provide a realistic training environment? This item involves immersiveness,
the physical and cultural environment, the common operational environment, responsiveness, and combined arms integration. 2. Does
FITE
provide an
effective training
environment? This item involves the real-time
exercise monitoring, post-scenario review, cognitive skills, decision-making, self confidence, and training time. 3. Is FITE suitable for the intended operational environment? This item involves the equipment, training requirements, usability, compatibility and interoperability, maintenance and failures, safety,
symptoms of simulator sickness. However, there were
demonstration.
4. Impacts of Transitioning FITE
Platoon leadership and FITE could be used to standards and augment, training. Although ten
Service SMEs indicated that help meet established training but not replace, existing live people were used in support
roles for the demonstration, this number could be
greatly reduced by combining roles and automating behaviors.
and security. 4. What are the impacts of transitioning FITE? Finally, this item involves questions related to the capability to meet established training standards, CONOPS/TTP inputs, DOTMLPF considerations While a detailed report on the FITE JCTD Assessment is beyond the scope of this paper, we highlight some of the results, as space allows.
1. Realistic Training Environment
Thirty-five of forty-four trainees reported feeling like they were actually "there." Visual and Auditory realism scores were above 90%, while olfactory and tactile scores were 30%. The tactile response was expected, since we only replicated the feel of the weapon (without recoil) and a shock when hit. The shock device proved critical to the realism, because the threat of pain kept the trainees cautious; however, there was no other ability for the trainees to feel other interactions with the environment. The olfactory response was surprising, however, since the observers could clearly smell the olfactory cues, but the trainees did not. The importance of olfactory cues clearly needs more research.
2010 Paper No. 10148 Page 6 of 7
SPIRAL 2
The second series of demonstrations, called Spiral 2, will take place in September 2010 at the IIT in Camp Pendleton. The FITE team is currently working to refit
and upgrade the IIT to include new special effects, animatronics, and virtual displays, as well as mixed and augmented reality. In Spiral 2MR (Mixed Reality), FITE will demonstrate technologies to support mixed reality environments that are more immersive and interactive than the existing
IIT. Similarly, in Spiral 2AR (Augmented Reality), the FITE team will integrate augmented reality into the infantry training environment. In this demonstration, the trainee wears a see-through HWD with elements of the training environment, primarily computer-generated characters, presented realistically in their field of view. CONCLUSIONS
There has been a decade worth of work by a large body of people to make the FITE JCTD an overnight success. Although none of the individual technologies or methods are new or unique, the attention and focus the FITE JCTD has brought to the field of infantry simulation has been extraordinary. The Department of
Interservice/lndustry Training, Simulation, and Education Conference(I/1TSEC) 2010
Defense has allocated $285 million from FY 11 to 15
specifically for Close Combat/Infantry Immersive Training. With the Joint focus on the infantryman, we hope the FITE JCTD will chart the course of research for years to come. ACKNOWLEDGEMENTS
REFERENCES
Advanced Interactive Systems, PRISim ShootBack® Cannon, retrieved 17 June 2009 at http://www.aissim.com/facts/facts_shootback.htm JFCOM FITE JCTD Proposal Paper 091808, retrieved 28 July 2009 at
https://www.us.army.mil/suite/portal/index.jspysessi The author would like to thank USJFCOM, DUSD
Rapid Fielding Directorate, ONR, and JIEDDO for their support of the FITE JCTD Management team.
onid=lAA5CDC472E8E665CFC5BAA4EAF58680.
appd02_l Lindeman, R.W., Yusuyuki, Y., Haruo, N., & Hosaka, K. (2006). Wearable virbrotactile systems for virtual contact and information display. Virtual Reality, 9:203-215, DOI 10.1007/sl0055-005-0010-6
Schaffer, R., Cohn, J., Hoover, A., Fouad, H„ Martin, G. A., & Milham, L. (2006). "A System for Evaluating
Training Transfer between Virtual and Live Urban Environments", in Proceedings of the Huntsville Simulation Conference, October 2006, The Society for Modeling and Simulation International.
2010 Paper No. 10148 Page 7 of 7