XR Data Privacy_Ching-Wen Chang

Page 1


XR DATA PRIVACY

CHING-WEN CHANG

Keyword: XR Data

Introduction

As more companies invest in the XR (extended reality) market, major players like Meta and the startup Distance Technologies are working on developing the next generation of immersive products. With the increasing adoption of XR technology in fields such as education, manufacturing, and healthcare, it is essential to elevate discussions about XR security and data privacy. Effective data management is imperative in XR, as spatial computing gathers information about users’ behaviours, environments, and bystanders, which raises ethical and privacy concerns.

Consequently, finding ways to protect users’ XR data amidst the rapid adoption of this technology is vital. This discussion highlights the significance of privacy and security in XR data management. The XR data lifecycle comprises data collection, storage, sharing and use, with each stage presenting different privacy risks, starting from the embedded sensors in XR devices. Recommendations to improve privacy awareness in XR include implementing de-identification techniques in line with privacy regulations and enhancing the user interface, user experience and the design of XR devices to facilitate better protection of XR data privacy.

From the Origins of Data in History to XR Data in the Context of Privacy

The Oxford English Dictionary indicates that the earliest recorded use of the term “data” in English dates back to a theological tract from 1645, where it is referred to as “a heap of data” (Furner, 2016; Rosenberg, 2013). In a medical context, “data”—the Latin plural of “datum”—is used to describe multiple facts or observations (McAlister, V. C., 2016).

A shift in data interpretation began approximately 1850 to 1900, as sociology and statistics resulted in organized tables representing numerical values. These arrangements of facts and numbers present a scientific approach to quantities, obtaining them from inspection and calculation. Data revolutionized the computing world, beginning with the launch of the IBM 701 Electronic Data Processing Machine in

P. 2 | XR Data Privacy for Product Enhancement (Research Paper)

1953. The concept of data started representing attribute values, such as readings, measurements, and scientific results, primarily expressed numerically (Furner, 2016).

As the digital landscape evolves, data is often understood as “bits,” the basic binary digits (0s and 1s) that computers manipulate at their core. This evolution underscores the vital role of data in our increasingly interconnected world, especially in technology.

According to Furner, data can be categorized into two main types: numerical (quantitative) and non-numerical (qualitative or categorical). These data types can be collected through digital footprints and targeted group research. However, significant privacy concerns surround data management, particularly regarding digital numerical data.

Extended Reality (XR) offers an immersive experience combining physical and virtual worlds. It includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR data refers to the information collected, processed, inferred, and shared by XR devices concerning users, their surroundings, and bystanders. This data is essential for creating and enhancing XR experiences. However, privacy concerns arise when users utilize XR devices, particularly regarding data collection, storage, sharing, and use throughout the XR data lifecycle. Understanding the privacy risks associated with these processes is crucial for users, bystanders, developers, and industries. Identifying these risks can improve XR products and environments while prioritizing user privacy.

XR Data Acquisition

Data acquisition is a significant privacy concern in extended reality (XR). It is essential to collect information with users’ consent. Various sensors in Internet of Things (IoT) devices and XR products gather data about users’ activities, interactions, and biometric information. Users’ data is collected through embedded sensors in XR devices, which utilize multiple tracking and recording technologies, such as eye tracking, motion tracking, cameras, and microphones (Acheampong et al., 2023; Juárez & Rudick, 2024).

P. 3 | XR Data Privacy for Product Enhancement (Research Paper)

XR devices come equipped with sensors that can lead to privacy invasions due to extensive data collection. This process includes tracking users’ movements, interactions, and eye-tracking data. Eye tracking monitors where and how long users look at specific objects or areas within the XR environment, providing insights into their focus and attention. Motion tracking sensors record gestures and body and head movements, capturing how users interact within the virtual space. Data monitoring in XR presents a dual-edged concern: while it offers valuable insights for product development, it also poses significant privacy risks related to surveillance. Users may engage in virtual activities unaware they are continuously monitored, as the various embedded sensors collect and record sensitive information.

A lack of awareness about what data is being collected through their XR devices can unintentionally disclose sensitive information in virtual spaces and expose the surrounding environments where users are physically located, as cameras and microphones often record them. Bystanders in the same physical environment as XR users may encounter privacy violations (Abraham et al., 2022). This concern escalates when sensitive information about military bases is uncovered, posing a threat to national security and confidentiality. Moreover, data acquisition raises concerns about children’s privacy, as sensors collect information about classroom environments and classmates’ physical, behavioural, and biometric data during XR’s use for educational purposes. Additionally, biometric data related to physiological responses can provide insights into users’ emotional and cognitive states during XR experiences through heart rate and skin conductance. The privacy concerns extend beyond users’ physical information to their mental privacy. XR data collection can facilitate a sophisticated understanding of digital identity, leveraging low-level brain activity data to infer behaviour and intent (McGill, 2021). This privacy risk means that sensitive information can be revealed regarding mental, cognitive, and phenomenological experiences, along with psychological aspects like stress, arousal, emotion, and affective state. Overall, the data collected in XR encompasses information from both the virtual and physical worlds, raising significant privacy and ethical concerns that users and bystanders may not be fully aware of while navigating the XR experience.

XR Data Storage

The volume of XR data is significant, as millions of data points are collected during a twenty-minute XR

experience (Pahi & Schroeder, 2022). This data can be stored locally on devices or in the cloud. However, accessing XR content requires an internet connection, which increases the risk of data breaches. The storage of XR data may expose sensitive information about users, bystanders, and environmental factors, leading to serious privacy violations due to potential cyberattacks by hackers.

Existing cloud-based methods for preserving privacy in big data have advantages and disadvantages when protecting sensitive information. Some common technical examples of de-identification techniques include K-anonymity, L-diversity, and T-closeness (Jain et al., 2016). However, these methods have significant limitations. They are vulnerable to homogeneity or attribute disclosure attacks and struggle to balance privacy with utility. While more substantial privacy criteria can reduce the risk of re-identification, they often result in a more significant loss of helpful information. These weaknesses in statistical approaches can result in privacy breaches that endanger sensitive data, as hackers may identify similar patterns for malicious purposes.

Big data streams refer to large volumes of time-stamped data, such as sensor data, call center records, clickstreams, and healthcare information (Jain et al., 2016). XR data is a subset of these big data streams requiring real-time processing. However, this need poses challenges for existing anonymization algorithms like K-anonymity, which are more suited for processing static data. Homomorphic encryption is a more secure computational technique for processing sensitive data, particularly in XR applications (Alkaeed et al., 2024). Homomorphic encryption’s primary aim is to enable computations on encrypted data without decrypting it while safeguarding data privacy during communication and storage. This approach addresses the increasing demand for preserving data privacy in public clouds or on untrusted computers (Tourky et al., 2016). Users can encrypt their XR data directly on their XR devices before processing it on the server. This encryption ensures that sensitive data remains secure and private while stored in the cloud and prevents the server from accessing the raw XR data. Aggregation operations, such as summing gradients, can be performed by a central server using the encrypted XR data without decryption (Alkaeed et al., 2024; Fang & Qian, 2021).

XR Data Sharing and Use

Users’ XR data plays a crucial role in shaping business strategies. This information helps improve products through iterative development and enables targeted advertising as part of marketing efforts. However, the awareness and protection of data privacy must be carefully considered, as it often involves sensitive information (Heller & Bar-Zeev, 2021; Mhaidli & Schaub, 2021).

In XR (Extended Reality), data may be shared with application developers and companies, leading to targeted advertising and marketing initiatives based on users’ emotional or physiological responses (Heller & Bar-Zeev, 2021). Many XR users are unaware of how sensitive and valuable their data is when shared or utilized by third parties for profit. The misuse of XR data reveals users’ behavioural and biometric patterns, which can identify sensitive personal information such as health data, ethnic and racial backgrounds, sexual orientation, religious beliefs, or political affiliations. Furthermore, embedded XR sensors can capture extra environmental information, bystander data, and nonverbal behaviours, including facial expressions, interpersonal distance, gestures, posture, and eye gaze (Bailenson, 2018).

According to the studies by Mhaidli and Schaub (2021), a negative implication of data use is that advertisers could exploit XR data to target users when they are vulnerable or emotionally susceptible, leading to manipulative advertising practices.

As a result, implementing additional privacy protection measures is essential. Privacy-enhancing technologies (PETs) can be valuable tools for privacy risk management. For instance, advancements in encryption and differential privacy enable data analysis and sharing under privacy protection. Using synthetic datasets can address XR privacy concerns about sharing and further use. Polonetsky and Renieris (2020) state that homomorphic encryption is a technical method that allows encrypted data inference without decryption. This approach is particularly suited for analyzing and sharing XR data while preserving the confidentiality of sensitive information and ensuring user anonymity. Furthermore, the entire XR data lifecycle can benefit from homomorphic encryption, from the encryption of XR data on users’ XR devices to subsequent processing stages, effectively preventing privacy breaches.

Recommendations for Enhancing Privacy Awareness in XR Data

XR devices should provide real-time on-screen notifications to inform users when their data is being collected (Abraham et al., 2022). This approach should prioritize user-centered design and thorough user research to avoid obstructing users’ views. Additionally, it should indicate where the data is processed, stored, and used internally and externally. This transparency will help users understand the lifecycle of their data in XR and empower them to manage and protect their privacy effectively.

Users should have control over their customized privacy settings to access, manage, and adjust the extent of XR data collection. The system should acquire the minimum amount of XR data for basic functionality, especially if users desire high privacy and security in their XR experience. Additionally, users can opt in or out of further data collection, including sensitive personal information such as race or ethnicity, religious beliefs, political affiliations, biometrics, health data, and financial information. All XR data must be encrypted during processing to anonymize identifiable information and prevent breaches. Users can decide what data is shared, how it is shared, and whether third parties can use it for targeted advertising or product improvement.

Research on XR privacy issues (Lake et al., 2024) indicates that information disclosure is the most significant privacy threat in the XR experience. Protective measures in XR data anonymization are pivotal. One of the tactics is the application of differentially private algorithms within the XR ecosystem. A mathematical technique, differential privacy (DP), deployed in machine learning, can safeguard the confidentiality of sensitive information in datasets. (Alkaeed et al., 2024; Blanco-Justicia et al., 2021; Fang & Qian, 2021; Jain et al., 2016; Lake et al., 2024). It works by introducing noise to the data, which makes it difficult for attackers to determine which individual data points contribute to the overall results, thereby protecting users’ XR privacy while still allowing analysis of large datasets. This privacy-protective method can be implemented on users’ XR devices through local differential privacy, enabling data encryption and safeguarding against privacy leakage. Preprocessing encrypted XR data on local devices can enhance data processing efficiency and prevent raw data sharing in subsequent steps.

When implementing data protection techniques such as homomorphic encryption or de-identification, it is vital to carefully evaluate the trade-offs between privacy, performance, and practicality. Compliance with up-to-date privacy regulations, such as the General Data Protection Regulation (GDPR), is also essential (Hine et al., 2024). This consideration applies to the use of differentially private algorithms as well. By focusing on these privacy issues, we can develop improved methods for processing encrypted extended reality (XR) data that protect the rights of users and businesses within the legal framework. Ultimately, this will enhance the security of the XR experience.

Lastly, the design of XR devices should incorporate camera privacy covers, allowing users to physically control the collection of environmental data to ensure their privacy and that of bystanders. Additionally, green indicator lights should be a mandatory privacy feature placed next to cameras and microphones to alert bystanders when an XR device is in use and when data collection sensors, such as cameras and microphones, are active and continuously recording environmental data. Users must have control over their privacy to prevent unintended XR data collection. The alert notification feature should be integrated into the user interface of the virtual space and represented through physical indicators on the XR devices. This privacy consideration can ensure that both users and bystanders are clearly aware of the device’s operational status through visual and auditory alerts during the XR experience.

Conclusion

XR data is a critical and sensitive asset that significantly impacts users and their experiences. Collecting data from embedded sensors on XR devices generates a vast amount of detailed user, environmental, and bystander information, posing significant privacy risks. The disclosure of this information is a major threat to XR’s privacy. Addressing XR privacy concerns is essential by encrypting XR data on users’ devices and throughout the broader XR ecosystem to mitigate the risks of potential cyberattacks. Privacy-protective measures should be considered at every stage of the XR data lifecycle, including data acquisition, storage, sharing, and use. Users should be able to customize their privacy controls in virtual environments and on physical XR devices. This customization ensures protection for themselves, their surroundings, and bystanders. Implementing indicators for data collection within XR

spaces and on XR devices can enhance privacy awareness for users and bystanders through visual and auditory alerts during the XR experience. XR users should be informed about data collection, and alerts should be given to protect their physical and mental privacy. They should also retain ownership of their XR data, allowing them to decide whether reusing this information benefits XR platforms, applications, advertisers, and businesses involved in product development or marketing. These proposals emphasize the need for improvement. While XR data can enhance the immersive experience, creating a robust privacy environment in XR requires collaboration among users, bystanders, developers, updated privacy regulations, and relevant business and governing entities.

References

Abraham, M., Saeghe, P., Mcgill, M., & Khamis, M. (2022). Implications of XR on Privacy, Security and Behaviour: Insights from Experts. Nordic Human-Computer Interaction Conference https://doi. org/10.1145/3546155.3546691

Acheampong, R., Balan, T. C., Popovici, D.-M., Rekeraho, A., Sacco, M., Arpaia, P., & De Paolis, L. T. (2023). Embracing XR System Without Compromising on Security and Privacy. In Extended Reality, 14218, 104–120. Springer. https://doi.org/10.1007/978-3-031-43401-3_7

Alkaeed, M., Qayyum, A., & Qadir, J. (2024). Privacy preservation in Artificial Intelligence and Extended Reality (AI-XR) metaverses: A survey. Journal of Network and Computer Applications, 231, 103989. https://doi.org/10.1016/j.jnca.2024.103989

Bailenson, J. (2018). Protecting Nonverbal Data Tracked in Virtual Reality. JAMA Pediatrics, 172(10), 905–906. https://doi.org/10.1001/jamapediatrics.2018.1909

Blanco-Justicia, A., Domingo-Ferrer, J., Martínez, S., Sánchez, D., Flanagan, A., & Tan, K. E. (2021). Achieving security and privacy in federated learning systems: Survey, research challenges and future directions. Engineering Applications of Artificial Intelligence, 106, 104468.

Fang, H., & Qian, Q. (2021). Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet, 13(4), 94. https://doi.org/10.3390/fi13040094

Furner, J. (2016). “Data”: The data. In Information Cultures in the Digital Age. Springer. https://doi. org/10.1007/978-3-658-14681-8_17

Heller, B. & Bar-Zeev, A. (2021). The Problems with Immersive Advertising: In AR/VR, Nobody Knows You Are an Ad. Journal of Online Trust and Safety https://doi.org/10.54501/jots.v1i1.21

Hine, E., Rezende, I. N., Roberts, H., Wong, D., Taddeo, M., & Floridi, L. (2024). Safety and Privacy in Immersive Extended Reality: An Analysis and Policy Recommendations. Digital Society : Ethics, Socio-Legal and Governance of Digital Technology, 3(2). https://doi.org/10.1007/s44206-02400114-1

Jain, P., Gyanchandani, M., & Khare, N. (2016). Big data privacy: a technological perspective and review. Journal of Big Data, 3(1), 1–25. https://doi.org/10.1186/s40537-016-0059-y

Juárez, S. F., & Rudick, C. K. (2024). Imagining futures for Communication Education: XR/VR and the promise of educational technology. Communication Education, 73(2), 244–246. https://doi.org/ 10.1080/03634523.2024.2319739

P. 10 | XR Data Privacy for Product Enhancement (Research Paper)

Lake, K., Mc Kittrick, A., Desselle, M., Padilha Lanari Bo, A., Abayasiri, R. A. M., Fleming, J., Baghaei, N., & Kim, D. D. (2024). Cybersecurity and Privacy Issues in Extended Reality Health Care Applications: Scoping Review. JMIR XR and Spatial Computing, 1(1), e59409. https://doi. org/10.2196/59409

McAlister, V. C. (2016). Datum isn’t; data are. Canadian Journal of Surgery, 59(4), 220–221. https://doi. org/10.1503/cjs.009316

McGill, M. (2021). White Paper-The IEEE Global Initiative on Ethics of Extended Reality (XR) Report--Extended Reality (XR) and the Erosion of Anonymity and Privacy. Extended Reality (XR) and the Erosion of Anonymity and Privacy-White Paper, 1-24.

Mhaidli, A. & Schaub, F. (2021). Identifying Manipulative Advertising Techniques in XR Through Scenario Construction. In 2021 CHI Conference on Human Factors in Computing Systems https://doi. org/10.1145/3411764.3445253

Pahi, S., & Schroeder, C. (2022). Extended Privacy for Extended Reality: XR Technology Has 99 Problems and Privacy is Several of Them. 4 Notre Dame J. Emerging Tech http://dx.doi.org/10.2139/ ssrn.4202913

Polonetsky, J., & Renieris, E. (2020). Privacy 2020: 10 Privacy Risks and 10 Privacy Enhancing Technologies to Watch in the Next Decade. Future of Privacy Forum. Rosenberg, D. (2013). Data before the Fact. In “Raw Data” Is an Oxymoron. MIT Press. https://doi. org/10.7551/mitpress/9302.003.0003

Tourky, D., ElKawkagy, M., & Keshk, A. (2016). Homomorphic encryption the “Holy Grail” of cryptography. 2016 2nd IEEE International Conference on Computer and Communications (ICCC), 196–201. https://doi.org/10.1109/CompComm.2016.7924692

P. 11 | XR Data Privacy for Product Enhancement (Research Paper)

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.