Communicating through the digital lens

Research Essay: COMMUNICATING THROUGH THE DIGITAL LENS Amber Barton - 699287 Contemporary Digital Practice - ABPL90149

Communicating through the Digital Lens Insights into channels of communication within BIM enabled workflows of digital practice using VR and AR KEYWORDS BIM, Virtual Reality, Augmented Reality, Mixed Reality, Communication, Language of Data, Measuring Experience, Experiencing Measurements, Augmented Construction, Virtual Facility Management INFORMATION Author: Amber Barton Submitted: 25 May 2018 Subject: Contemporary Digital Practice University: The University of Melbourne

1. INTRODUCTION The age of digital practice has transformed the way in which the architecture, engineering and construction (AEC) industries work across all aspects of the design lifecycle. With Building Information Modelling (BIM) moving towards acceptance as the industry norm, increasingly AEC are underpinned by a language of data.

Consequently, now more than ever before, it is essential for designers to engage in effective channels of communication which transform the collections of embedded information into tangible entities.

Virtual Reality (VR) and Augmented Reality (AR) are suggested to supply the means of moving from embedded data to displayed design. While neither VR or AR are new concepts, recent technological innovations have seen a dramatic growth in industry interest, such that both are no longer novel elements of science-fiction but instead legitimate tools for visual communication.

The linkages between BIM, VR and AR will firstly be explored in relation to the communication channels between architect and client. Focus will be on the possibilities of VR to facilitate greater bi-directional understanding of experience.

The second channel of communication to be explored is current and potential BIM-AR workflows, focusing on architect and

construction contractor interactions. Focus will be on the ability for traditional techniques of construction to be made more efficient in terms of time, cost and material wastage.

Some final suggestions will also be made on the potential for AR to be used to link BIM data to the operations and management phase of the building lifecycle.

This paper proposes that in the present age of BIM enabled digital practice, VR and AR provide the essential connection between embedded data and displayed design. However, they are in every sense of the word, restricted to use as the empowering lens. BIM enabled VR and AR are not replacements for good design, but they may supply the digital communication tool from within which good design can be created.

2. VIRTUAL REALITY Virtual Reality (VR) is not a new concept, in fact some consider it to date back as far as the first cave drawings over 30,000 years ago, or to the first paintings from the European Middle Ages, as both are ‘representations of realities’ 1.

While this may be true in some respects, this paper will apply the more contemporary understanding of VR as a technological system related to the generation of interactive virtual environments from computer generated simulations 2.

Page 1 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

VR has drawn a large amount of interest within academic and industry discussions over the past 6 years. This is due to the parallel improvements in computer efficiency, software for three-dimensional (3D) modelling, corresponding plug-ins for visualisation, and tools for virtual simulation, in particular head mounted devices (HMD) 3.

ABPL90149

gained some traction within AEC digital practice off the back of similar software and hardware innovations. However, it has not reached maturity as a digital communication tool within the larger AEC industry. Figure 2 shows AR being used during the design process to show 3D models overlaid on 2D printed drawings.

Combined these innovations of software and hardware have seen VR begin to find used within the AEC industry as a tool for communicating experience through complete visual (and sometimes auditory) immersion within a simulated environment 4. VR is typically taken from the first-person perspective such that the user, when wearing an HMD, may physically turn their head and experience a corresponding shift in the virtual rendered environment 5.

Figure 1 shows VR being used for the first time as an education tool for undergraduate architecture studio presentations at The University of Melbourne. Students were required to create Full VR renders using Unreal Engine.

Fig. 2. Codessi Architects, AR usage in the office, 2013, photograph.

AR requires an extension of the hardware and software components of VR, as it incorporates tracking registration to situate the pairings of digital and real sources of information7. The display aspect of AR is also difficult as many of its potential applications are either outside, and not suited to simple tablet screens due to glare, or require other task to be completed while viewing the AR 8.

Unlike VR in which the computer generates the entire user environment, AR creates an enhanced version of reality 9. This involves a layering of virtual information over sitespecific views of the physical world 10. This combining of digital and real information allows AR to become a tool for communication of the usually hidden or abstract features of a project 11.

Fig. 1. Amber Barton, VR usage as an educational tool for modelling experience, 2018, photograph.

3. AUGMENTED REALITY Like VR, Augmented Reality (AR) has been around for a long time, with its history closely related to the gaming industry 6. AR has also

The ability of AR to add value to the real world has seen interest grow within the construction process to more efficiently access imbedded information on-site 12. This ability to recall virtualise data in-situ has seen recent discussion of AR turn towards operations and maintenance 13.

Page 2 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

4. BUILDING INFORMATION MODELING

ABPL90149

5. CURRENT TRENDS OF VR

Building Information Modelling (BIM) forms the enabling driver for the usage of both VR and AR within digital practice. BIM is best thought of as a methodology that provides an accurate virtual 3D model of a building and its infrastructures 14. The power of BIM for collaboration lies within its structuring and exchange of information from a centralized source 15. This creates an ease of data acquisition; however, this information intensive work environment is not necessarily facilitating of increased communication, and in fact may be more confusing, especially for stakeholders 16.

Consequently, VR and AR become important channels of communication within the design process as information may be directly extracted from the BIM environment and transformed into a more easily understood visual entity.

An important aspect of the streamlining of the model to visualisation process occurred in 2014 with the release of Revit 2014. This release of Revit allows for custom export API such that material and texture as well as texture coordinates may be extracted 17.

This interchange of information between 3D model and visualisation software has been continuously increasing to the point where onthe-fly changes may be made while within the VR simulation. The author’s recent interactions with members of architectural and engineering practices in Melbourne, Australia has suggested that BIM-VR workflows have reached a stage of relative fluidity with bi-directional communication being implemented in the design development stages 18.

The bi-directional exchange of information from model to display and back again is perhaps one of the most interesting channels of communication enabled by BIM. It posses very interesting potentials for the back-end of the design lifecycle using Mobile AR (MAR).

These technologies are prevalent in research and investigative prototypes, yet there is a lack of industry usage beyond bespoke instances 19.

Current trends of economic investment and global interest (through Google searches) suggest that BIM, VR and AR are here to stay and have the potentially to greatly impact the AEC industries. BIM interest has seen a steady growth over the past 6 years and is expected to continue to grow in popularity as it becomes the norm within AEC practices 20.

The increasing popularity of BIM has spurred an even more dramatic interest in VR over the past 6 years with VR having now surpassed BIM in terms of Google search history 21. This is in large due to the interest surrounding major investments by big companies such as Google, Samsung, Sony, Facebook and Apple, from about 2012 22. The value of the VR industry is estimated to be in excess of $12.1 million USD in 2018 and is forecast to continue to double each year until 2020 23.

The release of ‘Google Glass’ in 2012 marks the tentative begins of the current 360 stereo panoramas VR market 24. Stereo panorama VR is like traditional 2D renders, except you can look around the entire 360 degrees of the scene.

Most Revit rendering plug-ins now offer this option and are almost as easy to set up and run as a traditional render 25. This type of VR does not allow model interaction and is consequently limited to usage as a display device typically for interior architectural spaces 26.

Devices: - Samsung Gear VR: $99 USD - Google Day Dream: $79 USD - Google Cardboard: $15 USD + smart phone Renderers: - Autodesk Cloud Renderings - Lumion - Enscape - Vray The Samsung and Facebook, owned Oculus VR released the Oculus Rift in 2016 and commenced the mainstream competitive market for full VR27. Full VR means you can move around inside the model and interact with it, for example the time of day may be changed during the VR viewing to test different lighting

Page 3 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

ABPL90149

conditions 28. Full VR requires more expensive devices and graphic cards, however the possibility for immersive interaction has seen AEC firms in Australia and internationally invest in Full VR as part of the communication and collaboration workflow 29.

Devices: - Oculus Rift: $399 - HTC Vive: $599 Renderers - Iris Prospect - Enscape - Yulio While interest had been dramatically increasing it is suggested that VR may have reached its maturity and interest will decline in the following few years 30. This is not to suggest usage is in decline in fact it is at an all-time high, with VR having hit mass market and found place within the everyday workflow of AEC professionals.

6. RESEARCH AND CURRENT USAGE OF VR Hall and Tewdwer-Jones (2010) suggest that one of the most common problems encountered during the design and planning process is communication difficulties between 31 stakeholders . This is further supported by Johansson, Roupe, and Tallgren (2014) who suggest these communication difficulties are usually resulting from information not being presented in such a way that people can understand it 32. Past research has shown that VR may offer a communication platform capable of overcoming this experience divide.

Barker (2015) investigated VR as a tool to enable greater stakeholder understanding of Building Physics analysis, with a particular focus on reflected glare 33. The research used Oculus Rift HMD and Unity 3D gaming engine to create Full VR simulations of glare in various scenarios and times of the day, as seen in Figure 3. Barker (2010) concluded that VR facilitated an improved stakeholder appreciation of risks and successfully bridged often technical elements of building design with experience of end users through visualisation 34.

Fig. 3. David Barker, Comparison of technical data representation to VR experience, 2015.

Kuliga et al. (2015) also promoted VR as an empirical research tool for human-environment interaction 35. They found VR representation to be almost equivalent to the real world in terms of users ability to understand spatiality and wayfinding 36. However, they warn that VR can be easily misleading in terms of atmospheres often appearing sterile and empty 37.

This research has important implications for the potential usage of VR in industry as a tool for communicating experience as it reveals a need to build the model up to a high level of detail 38.

In this sense VR usage as a tool for communicating experience is limited to design stages when detailing within the digital model is at a significant level.

Kuliga et al. (2015) suggest that industry has already had wide acceptance of VR as a form of digital representation, but there is an arising field of Virtual Experience assessment which is not yet common practice. VE allows real time feedback of user experience, such as what they looked at and for how long, while within VR environments 39.

Yulio is one such VE enabled VR platform which allows for renders to be created through direct plug-in to Revit, Sketchup and Rhinoceros 3D. These may then be shared via a

Page 4 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

link with the viewers actions while within the VR recorded and statistics sent back to the designer 40. Figure 4. shows current usage of VR fly throughs within Billard Leece Partnership, an Australian architecture firm. BLP currently employs VR during design development phases using Yulio to create, share and analyse client and stakeholders engagement 41.

ABPL90149

undergo substantial developments over the next 2-5 years 45. AR has seen a study growth in interest since 2012 however within the AEC industry it is yet to find mass market appeal and remains largely within the realms of research and development 46. The instances of AR technology within consumer markets is growing, but these products are mainly for entertainment purposes and lack the accuracy required by the construction industry 47.

Microsoft Hololens seems to be a promising mixed reality combined headset and computer which allows the user to engage with holograms in the real world. Combined with Microsoft Layout, this HMD has the opportunity for 3D models to be experienced at 1:1 scale so architects and stakeholders may share and edit spaces in real time 48. Microsoft’s partnership with thyssenkrupp Aerospace engineers reveals portential applications for the Hololens to be used by construction workers and facility managers for more efficient data recall onsite 49.

Fig. 4. BLP, VR flythrough for Barwon Health North design development phase, 2017.

7. FUTURE USAGE OF VR VR has established itself as an important tool of communication within the workflow of AEC professions due to its ability to relatively quickly and easily convert models into immersive design experiences for clients and stakeholders. However, there remains some key limitations to usage which may provide future directs of growth for VR. Standalone VR provides a greater portability and accessibility for VR as it does not need a computer to view the model 42. Multiple user VR environments are currently used in many video games but has not yet reached possibility within AEC digital practice 43. This could further enhance the collaborative environment as meetings may take place totally within the virtual environment. Greater interactivity with objects and simulation of atmospheres within the VR may allow a greater ‘realness’ such that user subjectivity and experience is brought even closer to the built outcome 44.

9. PREVIOUS RESEARCH AND CURRENT USAGE OF AR While VR is now a fairly widely understood and streamlined process, this is not the same for AR. Many instances of AR usages remain bespoke applications, lacking the universality of process required for large scale industry adoption.

Research by Chu, Matthews and Love (2018) revealed that efficiency in terms of time and wastage during the construction stage can be significantly improved through the implementation of AR integrated with paper documentation 50.

Cloud-based mobile BIM AR system was created which linked QR codes within the documentation set to relevant information from the BIM model, as shown in Figure 5.

8. CURRENT TRENDS OF AR AR has followed similar trends to that of the computer, leading many to believe that it is to

Page 5 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

ABPL90149

10. FUTURE USAGE OF AR AR has the potential to impact upon the whole design lifecycle from schematic design of massing studies in-situ; through to the construction stage with workers able to access digital scheduling data overlaid over the real world construction site; and even into facility management as maintenance information is accessible on site through a tablet or HMD. The potential applications are immense but so too are the software and hardware developments as well as the legal and cultural changes that need to occur to facilitate these new digital practices.

Fig. 5. Chu, Matthews and Love, AR visualizations of windows overlaid on documentation, 2018.

Information Surveyed Point for Observation and Tracking (InfoSPOT) is a combined tracking system and mobile application aimed at enhancing the decision-making process of facility managers (FM) 51. The research suggested that ‘magic lens’ AR technology may be useful for FMs, however the difficulty lies in creating a BIM model to a level of accuracy that the mobile platform is able to filter and display relevant information based on situational awareness 52. While intended as an easily replicated process, it relies heavily on the presence of a BIM management plan that focuses on inclusion of all operations stage information in a single model, which often not the focus for many design teams.

The areas that are beginning to see some industry usage are during the design phase of public projects and infrastructure whose often fast-tracked timelines require quick and efficient communication channels with clients.

In this sense AR is beginning to take over VR usage as a more power communication tool.

Shown in Figure 6, are trials of AR technology undertaken by BLP as a replacement for physical prototyping during early design phases.

Key to usage growth within construction will be developments in hardware which allow for extreme accuracy (to the cm if not to the mm) in terms of Indoor Location Tracking 53. This must be incorporated with software developments which allow for smoother integration into pre-existing information structures within BIM methodologies.

The other big change needed to accommodate AR for facility managers is a stronger connection between the operations and management deliverables and the output at the backend of BIM 54. Currently, BIM tools and the final federated model have little hands on benefit for facility managers 55.

11. CONCLUSION The AEC industry is on the cusp of change.

With BIM taking its place as the industry norm for design process, VR and AR are also growing in popularity as the logical tools to maximise on this new language of data. BIMVR has reached a point of widespread acceptance within workflows of digital practice, enabling quick and effect display of space. Research into VR has shown its ability to greatly improve understanding of design from the side of clients and stakeholders as they can completely immerse themselves within the space using HMD. Architects benefit from direct analysis through VE analysis which improve the efficiency of the feedback loop during design development.

While VR is without a doubt an essential tool for communication between AEC professionals

Fig. 6. BLP, Augmented reality model displays over real plans for Picton High School, 2018.

Page 6 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

and stakeholders, many believe it is only the beginning of change for the communication channels of digital practice. AR is the anticipated future. While only in very early stages of usage within the AEC industry, already there is talk that it may be as revolutionary as the computer 56. AR promises to greatly improve efficiency at all stages of the design process, but most importantly during the construction and into facility management, area which are currently highly manual and often wasteful in terms of time and material. But only time will tell if the innovations in terms of hardware, software, legal frameworks, and cultural practices will correlate into an integrated BIM-AR workflow.

With BIM as the enabling methodology, VR and AR supply the logical digital to transform embedded data into displayed experience and information, ultimately enabling streamlined communication across the whole design lifecycle.

ABPL90149

LIST OF FIGURES Fig. 1. Barton, Amber. VR usage as an educational tool for modelling experience, 2018, photograph.

Fig. 2. Codessi Architects, AR usage in the office, 2013, photograph. http://www.codessi.net/architecture-ageaugmented-reality Fig. 3. David Barker, Comparison of technical data representation to VR experience, 2015. David Barker, “Immersive experiences of building physics analysis to improve human connection to technical data,” Energy Procedia 78 (2015): 507-512.

Fig. 4. BLP, VR flythrough for Barwon Health North design development phase, 2017. BLP records Melbourne.

Fig. 5. Chu, Matthews and Love, AR visualizations of windows overlaid on documentation, 2018. In Michael Chu, Jane Mathews and Peter E.D. Love, “Integrating mobile Building Information Modelling and Augmented Reality systems: an experimental study,” Automation in Construction 85 (2018): 305-316, doi: 10.1016/j.autcon.2017.10.032 Fig. 6. BLP, Augmented reality model displays over real plans for Picton High School, 2018. BLP records Melbourne.

ENDNOTES 1

Steven M. LaValle, Virtual Reality (Illinois, US: Cambridge University Press, 2017), 23-24.

Rick Burchett, “Virtual reality for collaboration and communication, not just presentation,” Autodesk User Group International (November 8, 2017), https://www.augi.com/articles/detail/virtual-reality-for-collaboration-andcommunication-not-just-presentation 3 Maffei Luigi et al., “On the validity of immersive virtual reality as tools for multisensory evaluation of urban spaces,” Energy Procedia 78, (2015): 471-476.

4 Bruce Thomas, Wayne Piekarski, and Bernard Gunther, “Using augmented reality to visualise architecture design in an outdoor environment,” Design Computing on the Net (November 1999): 1 – 8, https://www.cs.unisa.edu.au/cisbht/DCNET99/dcnet99.html 5 David Barker, “Immersive experiences of building physics analysis to improve human connection to technical data,” Energy Procedia 78 (2015): 507-512.

6 LaValle, Virtual Reality, 15-26.

7 Thomas, Piekarski, and Gunther, “Augmented reality in an outdoor environment,” 2.

8 Rana Abboud, “Architecture in an Age of Augmented Reality: opportunities and obstacles for Mobile AR in Design, Construction, and Post-Construction,” NAWIC International Women’s Day Scholarship 2013, (March 2013): 28-29, http://www.codessi.net/architecture-age-augmented-reality 9 Michael Chu, Jane Mathews and Peter E.D. Love, “Integrating mobile Building Information Modelling and Augmented Reality systems: an experimental study,” Automation in Construction 85 (2018): 305-316, doi: 10.1016/j.autcon.2017.10.032 10 ThomsonAdsett and Auggd, “The augmented and virtual reality of architecture: ThomsonAdsett and Auggd,” Architecture and Design (June 2015), https://www.architectureanddesign.com.au/features/comment/the-augmented-and-virtual-reality-ofarchitecture 11 Thomas, Piekarski, and Gunther, “Augmented reality in an outdoor environment,” 2.

12 Chu, Mathews, and Love, “Integrating mobile Building Information Modeling and Augmented Reality systems,” 305.

13 Dominik Holzer, “The practical application of field BIM,” FM Magazine (March 2013), https://www.fmmagazine.com.au/sectors/the-practical-application-of-field-bim/ 14 Patti, “Information Modelling for Virtual and Augmented Reality,” 15 Chu, Mathews, and Love, “Integrating mobile Building Information Modeling and Augmented Reality systems,” 305.

16 Chu, Mathews, and Love, “Integrating mobile Building Information Modeling and Augmented Reality systems,” 305.

2

Page 7 of 8

Amber Barton

COMMUNICATING THROUGH THE DIGITAL LENS

17

ABPL90149

Mikael Johansson, Mattias Roupe, and Mikael Viklund Tallgren, “From BIM to VR: integrating immersive visualizations in the current design process,” eCAADe 32, (2014): 1-7.

18 Mark Healey, in conversation with Amber Barton (Bates Smart Office, Melbourne, April 30th 2018); Mark Mitchell, in conversation with Amber Barton (Billard Leece Partners Office, Melbourne, April 18th 2018).

Alex Rodger, in conversation with Amber Barton (Norman Disney Young Office, West Melbourne, May 1st 2018).

Soren Luckins, in conversation with Amber Barton (Buro North Office, Melbourne, April 4, 2018).

19 Jiao et al. “Towards cloud Augmented Reality for construction application by BIM and SNS integration,” Automation in Construction 33 (2013): 37-47.

20 Ferguson, Alex. “VR, AR and BIM – Where’s all the hype?,” BECA, November 7, 2017, https://www.beca.com/igniteyour-thinking/ignite-your-thinking/november-2017/vr-ar-and-bim-where-s-all-the-hype 21 Ferguson, “VR, AR and BIM – Where’s all the hype?”

22 Yulio, “VR is here to stay,” Yulio: Industry News (blog), May 11, 2018, https://blog.yulio.com/blog-digital-reality-hereto-stay 23 Yulio, “VR is here to stay,” (blog).

24 Yulio, “VR is here to stay,” (blog).

25 Revit Pure, “Pamphlets Issue 6: Virtual Reality,” Revit Pure Presents Pamphlets (Blog), September 12, 2017: 1-22, https://revitpure.com/blog/vr 26 Revit Pure, “Pamphlets Issue 6: Virtual Reality,”(Blog).

27 Barker, “Immersive experiences of building physics analysis,” 508.

28 Revit Pure, “Pamphlets Issue 6: Virtual Reality,”(Blog).

29 Healey, conversation; Luckins, conversation; Mitchell, conversation; Rodger, conversation.

30 Ferguson, “VR, AR and BIM – Where’s all the hype?”

31 Peter Hall and Mark Tewdwr-Jones, Urban and Regional Planning (London: Routledge, 2010): 4 – 50.

32 Mikael Johansson, Mattias Roupe, and Mikael Viklund Tallgren, “From BIM to VR.” eCAADe 32 (2014): 1-9.

33 Barker, “Immersive experiences of building physics analysis,” 510-512.

34 Barker, “Immersive experiences of building physics analysis,” 510-512.

35 S.F. Kuliga et al., “Virtual reality as an empirical research tool – exploring user experience in a real building and a corresponding virtual model,” Computers, Envrinment and Urban Systems 54 (2015): 363-375.

36 Kuliga et al., “Virtual reality as an empirical research tool,” 372-374.

37 Kuliga et al., “Virtual reality as an empirical research tool,” 372-374.

38 Kuliga et al., “Virtual reality as an empirical research tool,” 372-374.

39 Kuliga et al., “Virtual reality as an empirical research tool,” 372-374.

40 Yulio, “VR is here to stay,” (blog).

41 Mitchell, conversation.

42 Revit Pure, “Pamphlets Issue 6: Virutal Reality,”(Blog).

43 Revit Pure, “Pamphlets Issue 6: Virutal Reality,”(Blog).

44 Kuliga et al., “Virtual reality as an empirical research tool,” 372-374.

45 Ferguson, “VR, AR and BIM – Where’s all the hype?”

46 Ferguson, “VR, AR and BIM – Where’s all the hype?”

47 Javier Irizarry et al. “InfoSPOT: a mobile augmented reality method for accessing building information through a situation awareness approach,” Automation in Construction 33 (2013): 11.

48 Lorraine Bardeen, “Introducing two new mixed reality business applications: Microsoft Remote Assist and Microsoft Layout,” Windows Experience Blog (Blog) May 7, 2018, https://blogs.windows.com/windowsexperience/2018/05/07/introducing-two-new-mixed-reality-business-applicationsmicrosoft-remote-assist-and-microsoft-layout/ 49 Bardenn, “Introducing two new mixed reality business applications” (blog).

50 Chu, Mathews, and Love, “Integrating mobile Building Information Modelling and Augmented Reality systems,” 305.

51 Irizarry et al. “InfoSPOT,” 22.

52 Irizarry et al. “InfoSPOT,” 22.

53 Irizarry et al. “InfoSPOT,” 22.

54 Holzer, “The practical application of field BIM”.

55 Holzer, “The practical application of field BIM”.

56 Ferguson, “VR, AR and BIM – Where’s all the hype?”

Page 8 of 8