7
2 Applications
In this chapter, I introduce the applications that were used for the evaluations and studies presented in this dissertation. Each of the applications have similar characteristics and were developed back to back as an iterative process. One common denominator for these applications is that their tasks involve wayfinding in VEs. Two users also employed them collaboratively: one user acted as a tourist, trying to locate a local landmark (usually a tourist attraction), and the second one as a guide, helping the other user to find the goal. This concept was first introduced in Publication II.
The first application, Berlin Kompass, was evaluated in publications II, III, and IV. The second one, CityCompass, was rated in Publication V, and the third one, CityCompass VR (or Amaze360, as it was called in Publication VI) was assessed in publications VI and VII.
2.1 BERLIN KOMPASS
Berlin Kompass is a collaborative application that supports two simultaneous users. The first user takes the role of a tourist who has just arrived at a new city and needs to locate a local tourist attraction. Another user, acting as a guide, helps the tourist with the task. The application’s content consists of 360-degree panorama photographs from various cities.
This content is then ordered into sequential routes that the users can follow.
Both users are located in different spaces and have their individual view of the application, which they can pan around freely. To go from one panorama (i.e., location) to another, the tourist must move along the route as per the guide’s instructions. The tourist’s view of the application can be seen in Figure 3.
Figure 3. The tourist’s view of the Berlin Kompass, as used by the researcher. The panorama image is projected with three projectors (Publication IV).
Interaction Design
The Berlin Kompass supports embodied interaction—the application view can be panned by turning one’s shoulders to either the left or right, which refreshes the panorama accordingly. This gesture was planned to emulate the natural way in which a human looks around his or her surroundings.
This application has two kinds of user interface elements. The first is called an exit, which moves both users along a route. Exits are only visible in the tourist view, as only they can move along a route. When a tourist has a visible exit on the screen, he or she can activate this exit by walking towards the center of the projection (actually, the Kinect device located below the screen). This action moves both the tourist and the guide to the next panorama. The Kinect device is used to track both users’ movements.
Users can also use pointing gestures at the hotspot objects found in the panoramas. These objects offer vocabulary and contextual information about the surroundings, and they are always overlaid over landmarks.
Once one of these hotspots have been activated, a textual information box (e.g., “a modern office building”) describing the object becomes visible. This content is also played audibly to support the language learning aspects of the application. These utterances are output via speech synthesis. The hotspot information varies from single nouns to longer descriptions of the target object (e.g., “a building” versus “a gray office building with a sign on
……………
9
top”). This information can then be used to describe the surroundings to the other user.
Both users can see the panoramic view as a projection in front of them. The field of view (FOV) used by the application can vary based on the projection type and can be extended with multiple projectors and displays. For example, in Figure 3 three projectors were used to achieve a 160-degree view. Polys et al. (2005) stated that a larger FOV is more efficient with search-based tasks. Based on our results with Berlin Kompass (Publication II), it was also perceived as more satisfactory than interfaces with a lower FOV.
Berlin Kompass is a realistic collaborative VE that uses 360-degree panoramic photographs with embodied interactions. Sequential routes provide a good basis for both wayfinding and language learning experiments, and the collaborative aspects of the application encourage users to communicate and work together.
System Architecture
Berlin Kompass has four distinct components: 1) central logic, 2) graphics and voice service, 3) Kinect service, and 4) audio transmission service (Publication II). The overall program logic is handled by the central logic component. This component is responsible for receiving and sending messages between other system services. In addition, the communication between the clients is handled by this component. It also controls the activation of exits and dead ends.
The graphics and voice service handles the visual and auditory aspects of Berlin Kompass, and it is implemented on top of a graphic engine called Panda3D.1 This component handles the display of cylindrical panoramas, their FOV, and speech synthesis content. The Kinect service tracks the user’s physical location and skeletal joints. The data from these is then transformed into gestures, which are used to control the GUI of the system.
This includes the panning of the screen and movements from one panorama to another. This service also handles the pointing gesture while utilizing the Microsoft Kinect SDK. The audio transmission service handles the communication between the two installations. The service sends audio between the clients in User Datagram Protocol (UDP) packages. The Berlin Kompass’s application architecture can be seen in Figure 4.
1 https://www.panda3d.org
Figure 4. The Berlin Kompass’s system architecture. Remotely located users can interact using the built-in audio connection. The application’s statuses (tourist is moving, users have
reached their goal, etc.) are transferred via socket-based messaging. (Publication II) Wayfinding Scenario
Berlin Kompass supports two simultaneous users who must communicate and collaborate to complete a wayfinding task. Before starting this task, the user who acts as a tourist selects one of the three routes. All these routes start from the same location, but each has its own goal. The routes are based on real geographic locations around downtown Berlin. Once the route has been selected, both users are taken into the first panorama, and they can start communicating with each other. This communication is mediated via a headset. Because the application is designed for language learning, the users communicate in a predetermined language. In addition, contextual information is presented in this language. Currently, the application supports German, English, French, and Swedish.
As mentioned before, only the tourist can activate the exits and move along the route. Once an exit is activated, both users are transitioned to the next panorama. In each panorama, the tourist has three to four exits to choose from, and only one of these exits takes users closer to the goal. Therefore, it is crucial that users communicate with each other clearly.
Once an incorrect exit is chosen, both the tourist and the guide are transitioned to a dead end. After this, the tourist must describe the contents of an image to the guide, who then needs to pick the correct image from four options. This scenario is presented in Figure 5. If an incorrect image is selected, both users stay in the dead-end panorama. When the guide chooses the correct image, both users are transitioned back to the panorama where they got lost (i.e., where they activated the dead-end exit).
……………
11 Figure 5. In dead-end panoramas, the tourist describes his or her location (above) so the
guide can select the correct image from four options presented below (Publication IV).