The emergence of commercial MAR and MMR applications, as well as the mobile platform itself, can produce new challenges concerning information security and the privacy of the users. This is especially true with possible smartglass MAR/MMR applications combined with other mobile devices, since such systems are still quite novel. Roesner et al. [2014] divide the challenges into three categories: challenges with 1) single applications, 2) multiple applications and 3) multiple systems, and list the following characteristics of AR/MR technologies and applications which may produce security and privacy risks:
• A complex set of input and output devices which may be always on (camera, GPS, microphone, display, earpiece, etc.);
• A platform that can run multiple applications simultaneously, with these applications sharing the aforementioned different input and output devices;
• The ability to communicate wirelessly with other systems, including other mobile AR/MR systems, smart devices and remote computers.
Concerning single applications, Roesner et al. [2014] note that malicious applications could provide the user with false information overlaid on the augmented view of the real world (such as incorrect translations of foreign text, incorrect speed limits on navigation applications, etc.), or cause sensory overload to the user (flashing bright lights on the view, playing loud sounds, etc.).
While some malicious applications acting this way might only be a minor nuisance, in some cases they could prove to be extremely hazardous (such as in traffic environments). Providing false augmented information could also lead to failure in any project or event the user is performing in the MAR/MMR environment. Other concerns include malicious applications gaining control over the device's data access (i.e. access to sensors, video or audio) and leaking this information to unwanted parties. With multiple applications, security risks can include a malicious application obscuring, hijacking or altering the content presented by an other application, the risks would be very similar as those mentioned above.
Roesner et al. [2014] also point out that allowing only one application to control the output (display) at a time is not a sufficient solution, since it is more or less required that different applications must have access to the display when necessary. Otherwise the user would have very limited choices of what to do and when, which in turn would result in lowering the overall user experience of MAR/MMR systems.
Users would probably also like to have the possibility to share virtual objects between applications, and as with other systems, mobile augmented and mixed reality applications would probably share APIs with each other. Both cases would require appropriate measures to be taken with access control for cross application sharing. Concerning multiple systems (i.e. MAR/MMR systems belonging to different users or other parties), security and privacy risks may arise with applications that communicate with each other and allow sharing of virtual objects or spaces, or other information between users, and how access to personal and sensitive information is controlled.
Figure 2: Table outlining possible AR (and MR) security and privacy challenges categorized by system scope and functionality [Roesner et al., 2014]
Security and privacy approaches used on the mobile phone platform alone are not sufficient enough when considering other platforms, independent or linked to mobile phones or other devices, such as smartglasses (of which Google Glass is a good example). Mobile phones have their sensors (including cameras and GPS) turned off occasionally, and even if the device is hijacked by malicious software and the sensors are turned on without the user knowing, a mobile phone is not equipped most of the time (spending most of its time in the users pocket instead). However, on a head-mounted system such as Google Glass, the camera (or any other input device) is typically always on when the device is in use, and if hijacked by malicious software, the camera of the device can be accessed and used by hidden malicious applications without the user knowing anything about it. This can pose a serious security and privacy risk at all times while the user has the device equipped [Roesner et al., 2014].
Mobile AR and MR systems may be seen as intrusive regarding privacy and security even if there is no malicious intentions with their use. For example, a person might feel his or her privacy being violated if viewed by another person via a MAR/MMR display, possibly including video recording or facial recognition software. Similarly sensitive information can be accidentally compromised by a MAR/MMR device's video recording capabilities. Even voluntary sharing of virtual information or virtual objects might compromise users' privacy if applications are allowed to share such data with other applications, users, or even remote systems by default. Roesner et al.
[2014] mention that in addition to technical solutions to minimize privacy and security risks, social, policy or legal approaches concerning augmented reality systems may be called for. Enforcing such regulations might prove to be difficult, however, unless the legal and social approaches are similar in all environments (around the world), and also if users act under false identities. Additionally, users might be concerned about sharing private information, such as name, age and location in a MMR environment [Dhir et al., 2012].
Despite the challenges with security and privacy issues regarding MAR/MMR applications, Roesner et al. also note that some augmented reality platforms, such as smartglasses, can also provide increased information security to a user. For example, personal MAR/MMR displays, like Google Glass, efficiently prevent shoulder surfing, since the interaction is visible to the users own view only. If a system or application is proven to be secure enough, personal MMR applications can be used to display information to the user which might otherwise be risky to display in a public mobile environment (for example, presenting passwords or similar information overlaid on the MMR display). Nonetheless, the concerns with privacy and information security are a topic that needs addressing as MAR and MMR systems continue to develop and become a part of the everyday devices people will use, especially since the core functionality and information security of mobile systems is perhaps even more obscured to the everyday user than similar issues with the desktop platform. Many mobile systems synchronize, share and save personal information as a
default, and if a similar approach is used with MAR and MMR applications, the development process needs to address the information security and privacy issues discussed in this chapter.