XR DATA PRIVACY
CHING-WEN CHANG
Keyword: XR Data
Introduction
As more companies invest in the XR (extended reality) market, major players like Meta and the startup Distance Technologies are working on developing the next generation of immersive products. With the increasing adoption of XR technology in fields such as education, manufacturing, and healthcare, it is essential to elevate discussions about XR security and data privacy. Effective data management is imperative in XR, as spatial computing gathers information about users’ behaviours, environments, and bystanders, which raises ethical and privacy concerns.
Consequently, finding ways to protect users’ XR data amidst the rapid adoption of this technology is vital. This discussion highlights the significance of privacy and security in XR data management. The XR data lifecycle comprises data collection, storage, sharing and use, with each stage presenting different privacy risks, starting from the embedded sensors in XR devices. Recommendations to improve privacy awareness in XR include implementing de-identification techniques in line with privacy regulations and enhancing the user interface, user experience and the design of XR devices to facilitate better protection of XR data privacy.
From the Origins of Data in History to XR Data in the Context of Privacy
The Oxford English Dictionary indicates that the earliest recorded use of the term “data” in English dates back to a theological tract from 1645, where it is referred to as “a heap of data” (Furner, 2016; Rosenberg, 2013). In a medical context, “data”—the Latin plural of “datum”—is used to describe multiple facts or observations (McAlister, V. C., 2016).
A shift in data interpretation began approximately 1850 to 1900, as sociology and statistics resulted in organized tables representing numerical values. These arrangements of facts and numbers present a scientific approach to quantities, obtaining them from inspection and calculation. Data revolutionized the computing world, beginning with the launch of the IBM 701 Electronic Data Processing Machine in
P. 2 | XR Data Privacy for Product Enhancement (Research Paper)
1953. The concept of data started representing attribute values, such as readings, measurements, and scientific results, primarily expressed numerically (Furner, 2016).
As the digital landscape evolves, data is often understood as “bits,” the basic binary digits (0s and 1s) that computers manipulate at their core. This evolution underscores the vital role of data in our increasingly interconnected world, especially in technology.
According to Furner, data can be categorized into two main types: numerical (quantitative) and non-numerical (qualitative or categorical). These data types can be collected through digital footprints and targeted group research. However, significant privacy concerns surround data management, particularly regarding digital numerical data.
Extended Reality (XR) offers an immersive experience combining physical and virtual worlds. It includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR data refers to the information collected, processed, inferred, and shared by XR devices concerning users, their surroundings, and bystanders. This data is essential for creating and enhancing XR experiences. However, privacy concerns arise when users utilize XR devices, particularly regarding data collection, storage, sharing, and use throughout the XR data lifecycle. Understanding the privacy risks associated with these processes is crucial for users, bystanders, developers, and industries. Identifying these risks can improve XR products and environments while prioritizing user privacy.
XR Data Acquisition
Data acquisition is a significant privacy concern in extended reality (XR). It is essential to collect information with users’ consent. Various sensors in Internet of Things (IoT) devices and XR products gather data about users’ activities, interactions, and biometric information. Users’ data is collected through embedded sensors in XR devices, which utilize multiple tracking and recording technologies, such as eye tracking, motion tracking, cameras, and microphones (Acheampong et al., 2023; Juárez & Rudick, 2024).
P. 3 | XR Data Privacy for Product Enhancement (Research Paper)
XR devices come equipped with sensors that can lead to privacy invasions due to extensive data collection. This process includes tracking users’ movements, interactions, and eye-tracking data. Eye tracking monitors where and how long users look at specific objects or areas within the XR environment, providing insights into their focus and attention. Motion tracking sensors record gestures and body and head movements, capturing how users interact within the virtual space. Data monitoring in XR presents a dual-edged concern: while it offers valuable insights for product development, it also poses significant privacy risks related to surveillance. Users may engage in virtual activities unaware they are continuously monitored, as the various embedded sensors collect and record sensitive information.
A lack of awareness about what data is being collected through their XR devices can unintentionally disclose sensitive information in virtual spaces and expose the surrounding environments where users are physically located, as cameras and microphones often record them. Bystanders in the same physical environment as XR users may encounter privacy violations (Abraham et al., 2022). This concern escalates when sensitive information about military bases is uncovered, posing a threat to national security and confidentiality. Moreover, data acquisition raises concerns about children’s privacy, as sensors collect information about classroom environments and classmates’ physical, behavioural, and biometric data during XR’s use for educational purposes. Additionally, biometric data related to physiological responses can provide insights into users’ emotional and cognitive states during XR experiences through heart rate and skin conductance. The privacy concerns extend beyond users’ physical information to their mental privacy. XR data collection can facilitate a sophisticated understanding of digital identity, leveraging low-level brain activity data to infer behaviour and intent (McGill, 2021). This privacy risk means that sensitive information can be revealed regarding mental, cognitive, and phenomenological experiences, along with psychological aspects like stress, arousal, emotion, and affective state. Overall, the data collected in XR encompasses information from both the virtual and physical worlds, raising significant privacy and ethical concerns that users and bystanders may not be fully aware of while navigating the XR experience.
XR Data Storage
The volume of XR data is significant, as millions of data points are collected during a twenty-minute XR
experience (Pahi & Schroeder, 2022). This data can be stored locally on devices or in the cloud. However, accessing XR content requires an internet connection, which increases the risk of data breaches. The storage of XR data may expose sensitive information about users, bystanders, and environmental factors, leading to serious privacy violations due to potential cyberattacks by hackers.
Existing cloud-based methods for preserving privacy in big data have advantages and disadvantages when protecting sensitive information. Some common technical examples of de-identification techniques include K-anonymity, L-diversity, and T-closeness (Jain et al., 2016). However, these methods have significant limitations. They are vulnerable to homogeneity or attribute disclosure attacks and struggle to balance privacy with utility. While more substantial privacy criteria can reduce the risk of re-identification, they often result in a more significant loss of helpful information. These weaknesses in statistical approaches can result in privacy breaches that endanger sensitive data, as hackers may identify similar patterns for malicious purposes.
Big data streams refer to large volumes of time-stamped data, such as sensor data, call center records, clickstreams, and healthcare information (Jain et al., 2016). XR data is a subset of these big data streams requiring real-time processing. However, this need poses challenges for existing anonymization algorithms like K-anonymity, which are more suited for processing static data. Homomorphic encryption is a more secure computational technique for processing sensitive data, particularly in XR applications (Alkaeed et al., 2024). Homomorphic encryption’s primary aim is to enable computations on encrypted data without decrypting it while safeguarding data privacy during communication and storage. This approach addresses the increasing demand for preserving data privacy in public clouds or on untrusted computers (Tourky et al., 2016). Users can encrypt their XR data directly on their XR devices before processing it on the server. This encryption ensures that sensitive data remains secure and private while stored in the cloud and prevents the server from accessing the raw XR data. Aggregation operations, such as summing gradients, can be performed by a central server using the encrypted XR data without decryption (Alkaeed et al., 2024; Fang & Qian, 2021).
XR Data Sharing and Use
Users’ XR data plays a crucial role in shaping business strategies. This information helps improve products through iterative development and enables targeted advertising as part of marketing efforts. However, the awareness and protection of data privacy must be carefully considered, as it often involves sensitive information (Heller & Bar-Zeev, 2021; Mhaidli & Schaub, 2021).
In XR (Extended Reality), data may be shared with application developers and companies, leading to targeted advertising and marketing initiatives based on users’ emotional or physiological responses (Heller & Bar-Zeev, 2021). Many XR users are unaware of how sensitive and valuable their data is when shared or utilized by third parties for profit. The misuse of XR data reveals users’ behavioural and biometric patterns, which can identify sensitive personal information such as health data, ethnic and racial backgrounds, sexual orientation, religious beliefs, or political affiliations. Furthermore, embedded XR sensors can capture extra environmental information, bystander data, and nonverbal behaviours, including facial expressions, interpersonal distance, gestures, posture, and eye gaze (Bailenson, 2018).
According to the studies by Mhaidli and Schaub (2021), a negative implication of data use is that advertisers could exploit XR data to target users when they are vulnerable or emotionally susceptible, leading to manipulative advertising practices.
As a result, implementing additional privacy protection measures is essential. Privacy-enhancing technologies (PETs) can be valuable tools for privacy risk management. For instance, advancements in encryption and differential privacy enable data analysis and sharing under privacy protection. Using synthetic datasets can address XR privacy concerns about sharing and further use. Polonetsky and Renieris (2020) state that homomorphic encryption is a technical method that allows encrypted data inference without decryption. This approach is particularly suited for analyzing and sharing XR data while preserving the confidentiality of sensitive information and ensuring user anonymity. Furthermore, the entire XR data lifecycle can benefit from homomorphic encryption, from the encryption of XR data on users’ XR devices to subsequent processing stages, effectively preventing privacy breaches.