Hasnat Malik
ENHANCING DRIVERS’ SITUATION AWARENESS IN SEMI-AUTONOMOUS VEHICLES
The Use of Alerts in Preparation for Handovers
Faculty of Information Technology and Communication Sciences M. Sc. Thesis December 2020
Hasnat Malik: Enhancing drivers' situation awareness in semi-autonomous vehicles: The use of alerts in preparation for handovers
M.Sc. Thesis Tampere University
Master’s Degree Programme in Human-Technology Interaction December 2020
Autonomous vehicles (AVs) have progressed considerably over recent years. However, before AVs are adopted and accepted on a large scale, various challenges need to be addressed. Some of the most pressing concerns are related to the transition period taking place from the Automated Driving System (ADS) to the driver in semi-autonomous vehicles (SAE Levels 3 and 4). The re- search focuses on the core components contributing to the transition period that consist of situa- tion awareness (SA), take-over request (TOR), and handovers.
The objective of the study is to investigate the collective impact of SA and perceived trust in automation on handovers while manipulating the SA alert. The similar use of auditory SA alert in enhancing the drivers’ situation awareness by providing information about the changing driving conditions has not been previously studied. The multimodal TOR was presented to help the par- ticipants to acquire situation awareness and prepare for the handover.
The study relied on experimental research that was conducted in a fixed-base driving simula- tor. The simulator mimicked the driving behavior of a Level 3 conditional automation vehicle. 14 participants took part in the study. The data used in this research comprises logged driving task data, a self-report perceived trust measurement questionnaire, and interview data. Both quanti- tative and qualitative data were utilized. The quantitative data from the self-report questionnaire was analyzed using the Wilcoxon signed-rank tests and the qualitative interview data was ana- lyzed using grounded theory.
The findings indicate that the use of both the SA alert and the TOR were effective. However, the quantitative data found that the SA alert did not elicit higher perceived trust compared to the driving tasks in which it was not presented. Nevertheless, the qualitative data suggest that the participants found the SA alert to be useful, as it enhanced their situation awareness and made them feel safe. Furthermore, the TOR was found to be necessary for informing the participants about the handover.
The results achieved in the study have an implication on the use of alerts for enhancing the driver’s situation awareness in semi-autonomous vehicles. However, further research is needed to learn more about their utility in various driving situations.
Key words and terms: Autonomous vehicles, Semi-autonomous vehicles, Handovers, Situation awareness, Driving simulator, Take-over requests, Trust in automation, Multimodal alerts
The originality of this thesis has been checked using the Turnitin OriginalityCheck service.
I would like to express my sincere gratitude to my supervisors Jussi Rantala and Roope Raisamo for guiding me every step of the way through this exciting journey.
I would like to thank Jaakko Hakulinen for helping us with all of the technical matters related to the driving simulator. I would also like to express my gratitude to Ahmed Farooq and Grigori Evreinov for lending us the USB connected control box for the ex- periment, which was a critical component.
Also, I would like to thank Visaxion for letting us use their simulator. Surely, without the simulator this project would not have been possible. Additionally, I would like to express my gratitude to Tampere University for providing me with all the resources and help needed for making this project possible.
Tampere, 15.12.2020 Hasnat Malik
1. Introduction ... 1
1.1. Thesis structure 3
2. Literature review ... 4
2.1. Autonomous vehicles 4
2.2. Different automation levels 5
2.3. Key concepts of the research 7
2.3.1.Trust 8
2.3.2.Situation awareness 9
2.3.3.Handovers 11
2.3.3.1 Different types of handovers 12
2.3.4.Non-driving related secondary tasks 13
2.3.5.Timing of take-over requests - the crucial element of handovers 14
2.3.5.1 Modality of take-over requests 15
2.3.6.Driver roles and partial automation 16
2.4. Theoretical framework 18
3. Research Methods ... 20
3.1. Experiment objective 20
3.2. Experimental setup 22
3.3. Experiment design 24
3.4. Driving tasks 25
3.5. Recruiting participants 27
3.6. Experiment procedure 28
3.6.1.Progression of driving tasks 29
3.6.2.After completing the driving tasks 31
3.7. Quantitative data 31
3.7.1.Obstacles and the distances driven 33
3.7.2.Self-report questionnaire 34
3.8. Qualitative data 35
3.9. Data analysis 36
4. Results ... 39
4.1. Quantitative data 39
4.2. Qualitative data 42
4.2.1.1 The effect of take-over request on the participants’ sense of safety 43 4.2.1.2 Perceived usefulness of the take-over request 44 4.2.1.3 Suggested improvements for take-over request used in this research 45 4.2.2.The role of situation awareness alert on alerting the driver about the changing
driving conditions 47
4.2.2.1 Situation awareness alert enhancing situation awareness and safety 47 4.2.2.2 Situation awareness alert causing confusion 48 4.2.2.3 The impact of situation awareness alert on the handover 49 4.2.3.The accuracy of situation awareness alert and take-over request 50 4.2.4.What are the perceived risks associated with autonomous vehicles? 51
4.2.5.Summary 52
5. Discussion ... 53 5.1. How does enhancing the driver’s situation awareness influence handovers in
semi-autonomous vehicles? 53
5.2. What is the impact of enhancing the driver’s situation awareness on
perceived trust in semi-autonomous vehicles? 55
5.3. What is the role of situation awareness alert and take-over request in
enhancing situation awareness of the driver? 57
5.4. Limitations 61
5.5. Future work 63
6. Conclusion ... 65 References... 69 Appendices
1. Introduction
Autonomous Vehicles (AVs) have come a long way since their inception. The advance- ment of artificial intelligence (AI) and machine learning along with the processing power of computers and the various sensors that work in conjunction to make autonomous driv- ing possible have progressed throughout the years. This has created opportunities for the technology to develop further and bring autonomous driving closer to reality. (Weber, 2014; Surden & Williams, 2016; Nguyen, 2019)
AI has had a major role in the development of AVs. The application of AI to trans- portation made progress when a German aerospace engineer Ernst Dickmanns displayed a proof-of-concept vehicle capable of autonomous driving at high speeds. The vehicle was a Mercedes van outfitted with cameras and sensors that helped with the steering, braking, and throttling. As a result of the project, European research organization EU- REKA launched the Prometheus project, which helped the advancement of camera tech- nology, software, and computer processing. (Weber, 2014; Nguyen, 2019)
Autonomous driving progressed considerably at the beginning of the 21st century when the US military announced the DARPA Grand Challenge, which was a long-dis- tance competition comprising of a 150-mile obstacle course. None of the vehicles that participated in the competition were able to finish the course. Nevertheless, the event incited innovation in the field and was considered a success. (Weber, 2014; Nguyen, 2019)
The arrival of autonomous vehicles is inevitable. Many of the world's leading auto manufacturers have either developed or are working on developing autonomous vehicle technology. Even traditional technology companies, such as Google are heavily involved in the development of AVs. While some companies like Google and Volvo are focusing on moving directly to fully autonomous vehicles, many other manufacturers are concen- trating on first manufacturing semi-autonomous vehicles. (Weber, 2014; Davies, 2017;
Eriksson & Stanton, 2017; Welch & Behrmann, 2018; Nguyen, 2019)
AVs are anticipated to bring many changes to the way we commute. They are pre- dicted to make roads safer by eliminating the likelihood of human error, which accounts for more than 90% of automobile crashes. AVs will not be prone to the effects of intoxi- cation, fatigue, and distractions on the road that humans are susceptible to. AVs are also anticipated to reduce emissions and increase driver's comfort. (Fagnant & Kockelman, 2015)
Moreover, AVs are predicted to have a major impact not only on transportation but also on the wider society, which consists of the economy and overall road safety. AVs are exclaimed to have the potential to save more than 900,000 lives in the next 10 years.
(MIT Technology Review, 2017)
Nonetheless, numerous issues surrounding AVs need to be addressed before people start accepting them. Some of the most pressing concerns comprise of trust, situation awareness (SA), handovers, and takeover requests (TORs) (Gugerty, 1997; Merat & Jam- son, 2009; Shaikh and Krishnan, 2012; Gold et al., 2013; Schoemig & Metz 2013; Miller et al., 2014; Strand et al., 2014; de Winter et al., 2015; Gold et al., 2015; Mok et al., 2015;
Endsley, 2017; Iqbal et al., 2017; McCall et al., 2019; Merat et al., 2018; Wan & Wu, 2018; Petersen et al., 2019). Trust influences the adoption and acceptance of AVs. It also has an effect on the misuse and disuse of the AV technology (Choi & Ji, 2015; Bazilin- skyy et al., 2018; Bellet et al., 2019). SA is a major factor concerning safety, especially in semi-autonomous vehicles where the responsibility of driving is shared between the Automated Driving System (ADS) and the human driver (Gold et al., 2015; Iqbal et al., 2017; Ruijten et al., 2018; SAE J3016-2018, 2018).
All of these are important issues because they have an impact on arguably the most critical aspect of semi-autonomous vehicles, which is the transition period that takes place between the human driver and the ADS. Situation awareness (SA), take-over requests (TORs), and handovers are an essential part of the transition period. (Iqbal et al., 2017;
McCall et al., 2019; Bellet et al., 2019; Petersen et al., 2019)
Various studies have addressed the major concerns surrounding semi-autonomous vehicles. The number of studies published related to the topics concerning the primary issues affecting semi-autonomous vehicles seems to be increasing, which suggests that they are highly relevant topics to study. (Merritt et al., 2013; Miller et al., 2014; de Winter et al., 2014, Gold et al., 2015; Kyriakidis et al., 2015; Iqbal et al., 2017; Häuslschmid et al., 2017; Banks et al., 2018; Petersen et al., 2019, Salminen et al., 2019)
Although numerous studies have discussed the fundamental issues related to semi- autonomous vehicles, however, they have not extensively investigated the impact of SA and perceived trust while varying the driving conditions, namely the weather and how they collectively affect handovers. I believe this is where the research gap lies, and it is what I aim to address in this research.
The research primarily focuses on the core components that contribute to the transi- tion period in semi-autonomous vehicles that consist of SA, TORs, and handovers. The advancement from semi-autonomous vehicles to fully autonomous vehicles is expected to take many more years due to legal, ethical, and technological barriers (Surden & Wil- liams, 2016; Bellet et al., 2019). Therefore, focusing on semi-autonomous vehicles is a lot closer to reality, as many of the current vehicles contain Advanced Driver-Assistance Systems (ADAS), which can automate many of the driving functions to assist the driver (Surden & Williams, 2016).
The ADAS functionality can be considered a precursor to semi-autonomous vehicles (Surden & Williams, 2016). Moving from ADAS powered vehicles to semi-autonomous
vehicles might be a logical and natural step, provided that the aforementioned issues are sensibly addressed (Merat & Jamson, 2009; Iqbal et al., 2017; Petermeijer et al., 2017;
Bellet et al., 2019; Petersen et al., 2019).
The study will answer the following research questions:
1. How does enhancing the driver’s situation awareness influence handovers in semi-autonomous vehicles?
2. What is the impact of enhancing the driver’s situation awareness on perceived trust in semi-autonomous vehicles?
3. What is the role of situation awareness alert and take-over request in enhancing situation awareness of the driver?
I will conduct an experiment in collaboration with Tampere University utilizing a driving simulator to acquire the data needed for answering the research questions. I will utilize quantitative and qualitative data for answering the research questions. I expect my research will contribute to the existing and expanding literature on autonomous vehicles.
1.1. Thesis structure
The thesis consists of 6 chapters in total. The second chapter introduces the technology that makes autonomous driving possible and examines the literature on the fundamental issues associated with semi-autonomous vehicles. The literature reviewed forms the the- oretical framework to the work that I carried out.
The third chapter discusses the research methods. The chapter talks about the nature of the experiments conducted for collecting the data and provides justification for the methods used. It describes the experimental setup, design, and the things the experiments comprised of. It also discusses the type of data analyzed, the methods used for analyzing the data and provides justification for it. The fourth chapter presents the results from both the quantitative and the qualitative data. The fifth chapter discusses and reflects upon the results attained. Moreover, the chapter presents the study limitations and proposes future research. The sixth and final chapter draws conclusion to the research and summarizes the work carried out in the study, its limitations and future research topics.
2. Literature review
This chapter introduces the technology that makes autonomous driving possible. The chapter presents the key concepts of related to the research. The chapter also discusses the most important aspects that are related to the operation of semi-autonomous vehicles and the impact they have on the human driver. In addition, the chapter contains review of the literature that influenced my research and builds a theoretical framework to the work that I carried out.
2.1. Autonomous vehicles
Autonomous vehicles (AVs) utilize various technologies to make sense of the world around them. Their ability to see things is one of the most critical aspects of the technol- ogy. It can be compared to the importance our eyes play when we are driving. AVs can perceive things by combining an array of different technologies that consist of lidar, radar, cameras, and ultrasound. These technologies work in conjunction to map the world in which AVs navigate and to distinguish between different objects. By working together, they help determine the velocity of an object and the distance to or from it. (Barnard, 2016; Surden & Williams, 2016)
The capability of vehicles to ascertain the velocity, angle or range of objects comes from radars. Automobiles have utilized this technology for approximately 20 years now.
It is the technology that makes adaptive cruise control and automatic emergency braking possible. It is immune to bad weather which is one of its strengths. However, the data that it returns is not precise, which is why you need multiple technologies to work simultane- ously. (Barnard, 2016; Surden & Williams, 2016)
Cameras give vehicles the ability to see lane lines and road signs (Surden & Williams, 2016). Unlike radar, ordinary cameras are vulnerable to bad weather. They find it difficult to identify light objects against bright skies, which was reported to be a factor in the Tesla autopilot related fatality that occurred in May 2016. Nevertheless, when weather condi- tions are good, meaning there is good light, they work exceptionally well and have very good resolution. This helps the vehicle distinguish between the various objects it per- ceives. (Barnard, 2016)
A solution to the problem that ordinary cameras suffer from, are thermal imaging cameras. Thermal imaging cameras use infrared technology to create images of beings and objects that emit heat. Objects and beings with a temperature above absolute zero emit infrared radiation and can therefore be detected by thermal imaging cameras. Ther- mal imaging cameras have a clear advantage over ordinary cameras, as they can see more clearly in challenging conditions, such as fog, smoke, haze, darkness, and sun glare.
(Dirjish, 2018)
Furthermore, AVs use Lidar (light detention and ranging) to build a 3-d map of the world around them. Lidar is usually located on top of the vehicle to provide an unob- structed view. (Barnard, 2016; Surden & Williams, 2016)
Ultrasound is used to determine close range distances. It works by emitting ultrasonic waves and detecting their return to find out the distance. These sensors are used in parking aid systems (Barnard, 2016; Davies, 2018). Cameras along with sensors collect large amounts of data that is processed in real time, which helps keep the vehicle traveling in the right lane and operating safely (Barnard, 2016; Surden & Williams, 2016). All the technologies that make autonomous driving possible, would not be able to achieve the feat if they would work alone. It is the unity of these technologies that makes autonomous driving possible (Surden & Williams, 2016; MIT Technology Review Insights, 2018).
The technologies mentioned pull the data. It is the vehicles computers that make sense of the data and distinguish between different objects. Artificial intelligence and machine learning are at the heart of this. In addition, connectivity is also a vital component of autonomous vehicles. It gives them access to the environment in which they operate in real time. It includes the weather, surface conditions, maps, other cars, latest traffic, and road infrastructure. This information is used to make decisions regarding various actions, such as braking or switching lanes. (Barnard, 2016; Gupton, 2016; Surden & Williams, 2016)
2.2. Different automation levels
Society of Automotive Engineers (SAE) have defined six levels of automation (SAE J3016-2018, 2018). Vehicles are classified into different automation levels depending on the types of automation systems they are equipped with. The distinction between the dif- ferent automation levels is presented below. (Hyatt & Paukert, 2018)
Level 0: No automation
All traditional cars essentially fall under this category. At this level, the responsibility of driving lies completely with the driver. Some cars that are classified as level 0, might have a fixed-speed cruise control system or blind spot warnings. However, it is the driver’s responsibility to be in command of every driving action.
Level 1: Driver assistance
Most of the contemporary cars that we see on the roads today fall into the SEA level 1 category of automation (SAE J3016-2018, 2018). For a car to be classified as level 1, it must have at least one advanced driver assistance system (ADAS), such as adaptive cruise control, which can maintain the vehicles own speed under certain situations. Other sys- tems that are included in ADAS consist of automatic emergency braking, automatic park- ing, and lane-keeping (Surden & Williams, 2016).
Level 2: Partial automation
For a vehicle to be classified as a level 2 automation vehicle, it must have two or more ADAS, such as adaptive cruise control, automatic emergency braking or active lane-keep assist. Two or more of these systems must work at the same time for a vehicle to be considered a level 2 automation vehicle. For example, if adaptive cruise control system would work in conjunction with automatic emergency braking, it would be classified as a true level 2 automation vehicle.
Level 3: Conditional automation
At this level of automation, you are operating at an autonomous level. The systems described in the previous three levels are there to assist the drivers. A vehicle equipped with level 3 automation can drive completely by itself on the highway. However, the driver must be ready to take on the task of driving if the conditions require to do so or there is a risk of a system failure.
This study focuses on Level 3 conditional automation.
Level 4: High automation
A vehicle equipped with level 4 technology can operate entirely on its own. It can be considered a fully autonomous vehicle. However, a vehicle operating at this level of au- tomation has some constraints. For instance, its operation can be limited to a geographical area via geofencing or it can be allowed to operate up to a certain speed limit. Level 4 automation vehicles come with a steering wheel and pedals, so that the driver could take control if a situation requires it.
Level 5: Full automation
Level 5 vehicles are fully autonomous. The vehicle is completely able to perform the task of driving and there is no need for the driver to intervene in any situation. At this level of automation there is not any need for a steering wheel or pedals. The vehicle can fully function without human involvement.
Below is an illustration of levels of driving automation that display the responsibility of the human driver and the Automated Driving System (ADS) in different automation lev- els. (SAE J3016-2018, 2018; Shuttleworth, 2019)
Figure 1. Levels of driving automation 2.3. Key concepts of the research
The major challenge with semi-autonomous vehicles is the transition period, which is the shifting responsibility of controlling the driving functions that happens between the hu- man driver and the ADS (Saffarian et al., 2012; McCall et al., 2019). Handovers are vital constituents of the transition period (Iqbal et al., 2017; McCall et al., 2019; Petersen et al., 2019). Moreover, situation awareness is one of the main factors that determines the success of the handover in terms of safety (Iqbal et al., 2017; McCall et al., 2019; Petersen et al., 2019).
Different types of alerts can be used to capture the driver’s attention prior to the hand- over and support his/her SA. Moreover, timing of the alerts is another important consid- eration that affects the driver’s SA and helps to prepare him/her for handovers. (de Winter et al., 2014; Iqbal et al., 2017; Petermeijer et al., 2017; Bazilinskyy et al., 2018; McCall et al., 2019; Petersen et al., 2019)
Also, the manner in which the alerts are presented to draw the attention of the driver are important to consider. They have a substantial role in conveying urgency to the driver (de Winter et al., 2014; Petermeijer et al., 2015; Petermeijer et al., 2016; Petermeijer et al., 2017; Bazilinskyy et al., 2018). The driver’s SA may be hampered because he/she might not be monitoring the driving environment due to being engaged in secondary non-
driving related (NDR) tasks, which are expected to increase with the increasing level of automation (de Winter et al., 2014; Iqbal et al., 2017; Banks et al., 2018).
Trust is a fundamental component in AVs as well, as it is a vital factor associated with their adoption and acceptance (Choi & Ji, 2015; Fagnant & Kockelman, 2015; Lee et al., 2015; Kaur & Rampersad, 2018; Hegner et al., 2019). Furthermore, trust along with SA have an imperative role in handover situations. The type of information presented for enhancing the drivers SA via alerts can increase his/her trust in automation, which can improve the handover performance (Petersen et al., 2019; Drexler et al., 2019). The sub- sequent chapters discuss these issues in more detail.
2.3.1. Trust
A homogenous definition of trust in the context of autonomous vehicles does not currently exist (Hoff & Bashir, 2014; Saleh et al., 2017). According to Petersen et al. (2019), the two most popular definitions of trust come from Mayer et al. (1995) and Lee & See (2004).
Mayer et al. (1995) define trust as “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party”. Lee & See (2004) define trust as “the attitude that an agent will help achieve an individual's goals in a situation characterized by uncertainty and vulnerability”. Accord- ing to the definition by Lee & See (2004) “An agent can be automation or another person that actively interacts with the environment on behalf of the person”.
Petersen et al. (2019) made use of both definitions and came up with a definition of trust applicable to AVs. They define trust “as the willingness of an individual to be vul- nerable to the actions of an AV based on the attitude that the AV will help them achieve their goals.” The definition coined by Petersen et al. (2019) was the most appropriate available for the work that I carried out.
Trust is an important construct in AVs. The potential users of AVs “have to place trust in the reliability, situation awareness and decision making of the system” that makes autonomous driving possible (Filip et al., 2016). Drivers will experience a drastic change from being in control of the vehicle to essentially being passengers and having no control in some situations in semi-autonomous vehicles. Some of the situations could present risks during which the safety and the well-being of the passengers lies with the vehicle.
The vehicle can also ask the driver to take back control in a situation that can be risky.
This requires a certain amount of trust in the AV. (Filip et al., 2016; Petersen et al., 2019) Trust is also one of the major factors influencing the adoption and acceptance of AVs (Choi & Ji, 2015; Fagnant & Kockelman, 2015; Lee et al., 2015; Kaur & Rampersad, 2018; Hegner et al., 2019). Lack of trust is one of the main barriers regarding the adoption of AVs (Kaur & Rampersad, 2018; Hegner et al., 2019). It also influences whether a
driver feels comfortable in giving up control to a machine (Hegner et al., 2019). Addi- tionally, the use, disuse and misuse of automation are factors that are influenced by trust (See & Lee, 2004; Hoff & Bashir, 2014; Lee et al., 2015; Filip et al., 2016).
Lack of trust affects the performance of the secondary NDR task as well. It can pre- vent the drivers from handing over the driving responsibility to the vehicle. It can also prevent the driver from fully focusing on the NDR task “because the driver is constantly monitoring the driving situation” (Petersen et al., 2019). Furthermore, trust can be influ- enced by situation awareness. It has been found that enhancing the drivers' situation awareness can have a positive impact on the trust experienced by the driver in automated driving. (Petersen et al., 2019)
Due to the above-mentioned factors trust is an integral element of semi-autonomous vehicles and will play a crucial part in their adoption and acceptance. Since the focus of my thesis is on semi-autonomous vehicles (SAE level 3), the aspect of trust cannot be ignored.
2.3.2. Situation awareness
The issue of situation awareness arises in automated systems that require a human oper- ator to monitor for prolonged periods of time and intervene if necessary, for instance, in case of a system failure (Endsley & Kris, 1995). Situation awareness is “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” as defined by Endsley (1995). She further divides situation awareness into three hierarchical levels:
Level 1 SA: Perception of the Elements in the Environment
Level 2 SA: Comprehension of the Current Situation
Level 3 SA: Projection of Future Status
In simple terms situation awareness means “knowing what is going on” (Endsley, 1995).
The most prevalent problem associated with the loss of situation awareness is the out- of-the-loop (OOTL) performance problem (Endsley, 2017; Merat et al., 2018). In the most severe cases, it can lead to deadly consequences (Endsley & Kris, 1995; Endsley, 2017). In the context of automated driving, it can be viewed as being in-, on- and out of the loop states during a period of automated driving. Drivers are assumed to be OOTL when their response time is impaired and is not functioning at the same level as manual driving. Merat et al. (2018) define the three states followingly:
In the loop - In physical control of the vehicle and monitoring the driving situa- tion
On the loop - Not in physical control of the vehicle, but monitoring the driving situation
Out of the loop - Not in physical control of the vehicle, and not monitoring the driving situation, or in physical control of the vehicle but not monitoring the driving situation
Relieving the driver from driving can aid the driver’s situation awareness, provided he/she actively monitors the road, while the vehicle operates in an automated mode. Com- pared to manual driving and driving with adaptive cruise control (ACC), highly auto- mated driving (HAD) reduces the driver’s workload (de Winter et al., 2014). de Winter et al. (2014) conducted a “meta-analysis and narrative review of simulator and on-road studies.” They found drivers can achieve a “super situation awareness” due to monitoring the environment while the vehicle operates in HAD mode. However, if the driver engages in secondary tasks, SA is impaired. If a critical event arises that requires the driver to take back manual control, the response of the driver will be slow, which poses a safety risk, as the lack of situation awareness can result in an accident (Ma and Kaber, 2005; Iqbal et al., 2017).
Driver distractions are also expected to increase with the increasing level of auton- omy. Drivers are more likely to engage in other activities, such as using their phones while the vehicle operates in automated mode. This is likely going to hamper the drivers' situation awareness. (Ma & Kaber, 2005; Iqbal et al., 2017)
Studies have shown that situation awareness of drivers decreases with the increasing level of automation. Increasing automation is also thought to impair the driving perfor- mance. It is possible that drivers will engage in demanding secondary tasks with the avail- ability of increasingly advanced automation, which will further hamper their driving per- formance and impact situation awareness. (Merat & Jamson, 2009; Schoemig & Metz 2013; Strand et al., 2014; de Winter et al., 2015; Iqbal et al., 2017)
The impact of situation awareness on trust and its implications on the performance of the secondary NDR task in the context of semi-autonomous vehicles has also been stud- ied. Petersen et al. (2019) discovered that increasing situation awareness can increase the driver's trust in automation. They manipulated situation awareness in three different lev- els: “no situation awareness, low situation awareness, and high situation awareness.” No information regarding the driving condition was provided in the no situation awareness condition, which was the control condition. A status update was provided in the low sit- uation awareness condition. The high situation awareness condition provided a status up- date and suggested a course of action.
One of the main aims of the Petersen et al. (2019) study was to determine whether SA helps build trust and has an impact on the performance of the secondary task. They discovered that trust has a strong positive impact on the secondary task performance in the condition that provided the most information related to situation awareness (high sit- uation awareness) (Petersen et al., 2019). It was an important finding, since drivers are
expected to engage more frequently in a variety of different secondary tasks as the level of automation increases (de Winter et al., 2014; Iqbal et al., 2017).
The driver must trust the automated driving system while focusing on performing secondary tasks, which can be an essential consideration depending on the importance of the secondary task. If the driver does not trust the ADS, it can hinder his/her performance of the secondary task, because he/she attempts to engage in both driving and the NDR task simultaneously. Moreover, situation awareness is a vital component aiding the driver in understanding what is going on in the environment in which the vehicle is operating.
It is particularly crucial for the driver to acquire adequate SA when taking back control of driving after being engaged in a secondary task. It has an impact on the safety and the quality of the handover. (Miller et al., 2014; Iqbal et al., 2017; Petersen et al., 2019)
Situation awareness and situational awareness are sometimes used interchangeably (Endsley, 1995; Iqbal et al., 2017; Petersen et al., 2019). I have used the term ‘situation awareness’ in this study.
2.3.3. Handovers
The most critical point in semi-autonomous vehicles is the control of the vehicle that shifts between the driver and the ADS. It can transition in two directions: from vehicle to driver and driver to vehicle. Takeover is referred to a situation where the ADS takes con- trol of the driving functions of the vehicle from the driver. Handover is referred to as the situation in which the driver is handed back control of the vehicle from the ADS to resume the responsibility of driving functions (Bellet et al., 2019). Although ‘takeover’ and
‘handover’ are sometimes used interchangeably, I have used the term handover to signify situations in which the vehicle hands the control back to the driver (McCall et al., 2019;
Bellet et al., 2019). It was more appropriate for the work that I carried out.
A handover situation arises, when a semi-autonomous vehicle senses that it has reached its operational design domain (ODD) (SAE J3016-2018, 2018). When that hap- pens, the vehicle generates an alert requiring the control of the vehicle to be taken over by the driver. The alert, which serves as a warning, is referred to as a Take-Over Request (TOR). Its job is to alert the driver to take back the control of the vehicle and provide adequate time in doing so. (Iqbal et al., 2017; Bellet et al., 2019)
Providing enough time to the driver prior to the handover is critical because he / she could be engaged in other activities that capture his / her attention. Handovers are an important aspect in semi-autonomous vehicles, as they have an impact on the drivers’
situation awareness. To guarantee a safe handover, it is paramount for the driver to have the necessary situation awareness before resuming responsibility of driving. (Iqbal et al., 2017; McCall et al., 2019; Petersen et al., 2019)
Semi-autonomous vehicles can monitor their environment for long periods of time, during which the driver might have disengaged, which makes sudden handovers hazard- ous (Iqbal et al., 2017; Victor et al., 2008). The lack of situation awareness can also lead to driving errors (Gugerty, 1997; Shaikh & Krishnan, 2012; Miller et al., 2014; Iqbal et al., 2017; Merat et al., 2018; McCall et al., 2019; Petersen et al., 2019).
The main challenge presented by handovers is the shifting responsibility of driving between the human driver and the ADS since it involves getting the driver’s attention back to driving after being inattentive. Such situations are tricky because they comprise of the driver’s situation awareness, trust in automation, TORs, and their timing; factors that are crucial for a safe handover. (Iqbal et al., 2017; McCall et al., 2019; Bellet et al., 2019; Petersen et al., 2019)
2.3.3.1 Different types of handovers
Handover situations can be divided into five different types as defined by McCall et al.
(2019):
- Scheduled handover
- Non-scheduled system-initiated handover - Non-scheduled driver-initiated handover
- Non-scheduled driver initiated emergency handover - Non-scheduled system initiated emergency handover
A scheduled handover occurs when a vehicle is about to enter a complex area where it is not capable of driving by itself. In such a case, the driver is notified in advance when the vehicle starts approaching the area, for instance, a certain distance or time prior to reaching the area. The notification to handover control to the driver occurs in the form of an alert. When handing control over to the driver, the vehicle must assume that the driver has adequate situation awareness and enough skills to resume the task of controlling the vehicle.
Non-scheduled system-initiated handover takes place when the system recognizes that operating the vehicle is beyond its functional limits, due to for instance, a sudden change in the traffic condition. In this type of situation, the driver might not have been expecting the control to be handed over to him / her. Regardless, the control must be handed over to the driver.
Non-scheduled system-initiated handover is used in this research. It is the handover that we emulate in the driving simulator experiment.
Non-scheduled driver-initiated handover is a situation where the driver decides to take control of the vehicle even if there is not a need for it. Some of the reasons for this
might be that the driver wants to experience the thrill of driving or the driver wants to take a different route, but the command is not being registered by the system.
Non-scheduled driver initiated emergency handover. This handover situation takes place when the driver has spotted a potentially disastrous situation in which the vehicle systems have failed to see and takes immediate control of the vehicle.
Non-scheduled system initiated emergency handover. This is a case in which the system is entirely at fault due to an internal system failure rather than an external threat. In this case the system is no longer able to function. If possible, the system notifies this to the driver. When faced with such a situation, the vehicle can at least stop itself safely if the driver is not able to or not willing to take over control.
2.3.4. Non-driving related secondary tasks
Multitasking is a common practice while driving and is the reason for driver distractions.
Various studies have recorded the damaging effects of multitasking, which includes in- teracting with in-vehicle systems, cell phone conversations and texting while driving (Iq- bal et al., 2017). The use of mobile phones particularly seems to be the most harmful.
Mobile phones are the reason for cognitive distraction, which occurs when drivers take their minds off the road and the surrounding situation. Cognitive distraction seems to have the biggest impact on driving behavior. It leads to slower reaction times (mainly braking), the inability to keep in the correct lane, and inferior steering performance (World Health organization, 2011). Distractions are expected to increase in semi-autonomous vehicles where driving responsibilities are assumed by cars more (Iqbal et al., 2017).
The increasing autonomous driving capabilities enable vehicles to operate on their own for extended periods of time. When a handover moment arises and the vehicle re- quires the driver to take control, it might be difficult for the driver to suddenly disengage from secondary task(s) and take control of driving the vehicle. A major factor that con- tributes to this difficulty is the lack of situation awareness of the drivers. Semi-autono- mous vehicles provide drivers the possibility of engaging in non-driving related activities, such as watching the news, checking email, or reading. When the vehicle suddenly asks the driver to take back control, the driver might have remnants of their preceding tasks, which impacts their situation awareness and can lead to fatal outcomes. (Iqbal et al., 2017) In traditional vehicles the driving task is typically considered to be the primary task, where the drivers focus is expected at all time. However, with the increasing capabilities of automation technology, the need for the drivers to take over control might decrease.
This might result in the NDR tasks the driver engages in as being the primary tasks with driving as a distraction (Hancock, 2013, 9 - 25; Iqbal et al., 2017)
2.3.5. Timing of take-over requests - the crucial element of handovers
It is likely that the drivers’ involvement with secondary tasks will become a common practice in semi-autonomous vehicles since they enable the driver to disengage from driv- ing for longer periods of time (Iqbal et al., 2017; Petersen et al., 2019). When the driver is engaged in a NDR task for an extended period and the vehicle reaches its operational design domain (ODD), an alert referred to as a take-over request (TOR) is generated by the vehicle to draw the driver’s attention back to driving and prepare him / her for a hand- over (Gold et al., 2013; Naujoks et al., Bazilinskyy et al., 2018; Epple et al., 2018).
The timing of TORs is vital because it impacts the driver’s SA and his / her ability to respond correctly when the control of the vehicle needs to be handed back to the driver (Gold et al., 2013; Iqbal et al., 2017). Moreover, correct timing of TORs is one of the most important determinants of a successful handover (Salminen et al., 2019). It has been investigated in various studies. In addition, different methods for generating TORs, such as presenting alerts with a specific lead time and pre-alerts have been explored (Gold et al., 2013; Mok et al., 2015; Iqbal et al., 2017; Epple et al., 2018; Wan & Wu, 2018).
Sudden handover situations in semi-autonomous vehicles (level-3) present more chal- lenges compared to vehicles with lower automation levels. A vehicle with level-3 auto- mation capabilities can monitor its environment for long periods of time, during which the drivers might engage in other activities instead of monitoring, while the vehicle drives by itself. Thus, it might be difficult for the driver to let go of the NDR tasks and take control of the vehicle with little or no time because of an immediate handover. The prob- lem of sudden handovers can be tackled with the use of pre-alerts or by providing drivers with adequate lead time. (Gold et al., 2013; Mok et al., 2015; Iqbal et al., 2017; McCall et al., 2019; Wan & Wu, 2018)
Iqbal et al. (2017) addressed the issue of handovers and measured handover perfor- mance with the use of pre-alerts and without them. They used two different types of pre- alerts: repeated burst and increased pulse. The pre-alerts took place 20 seconds before the handover. They found that in the cases where the pre-alerts were used, drivers looked up a lot earlier and did not rely on the visual input from the road at the last moment, which was the case in the instances where no pre-alert was used. There was hardly any difference between the 2 different types of pre-alerts regarding the preparation for the handover.
(Iqbal et al., 2017)
Priming drivers through pre-alerts can better prepare them for handovers as discov- ered by Iqbal et al. (2017) because drivers get adequate time to reorient themselves and acquire the necessary SA. It is an important aspect of driving, especially in semi-autono- mous vehicles, because it influences the driver’s reaction time. The modality and the tim- ing of the alerts are also vital given that they convey urgency. (Iqbal et al., 2017)
Pre-alerts offer various benefits: they provide drivers adequate time to disengage from the secondary NDR tasks, which can reduce mental workload and stress. This allows the drivers to be in a better condition to manage the handover. Pre-alerts also influence re- ducing the distraction effects of the NDR tasks. (Iqbal et al., 2017)
Lead time is another form of alerting the driver prior to the handover. It is similar to pre-alerts, as it also alerts the driver a certain time before the actual handover. Presenting drivers with a sufficient lead time can determine the outcome of a handover, as it has an impact on the safety and quality of the handover (Gold et al., 2013; Mok et al., 2015; Wan
& Wu, 2018). Lead time has a valuable role in getting the driver back into the loop (Gold et al., 2013; Endsley, 2017; Merat et al., 2018). It is crucial because it represents the time- to-collision (TTC) to the possible obstacle or the time until the system reaches its limits (Gold et al., 2013; Epple et al., 2018; Wan & Wu, 2018, Petersen et al., 2019).
Epple et al. (2018) identified that a minimum lead time of 5 seconds is needed for a successful handover. Gold et al. (2013) established that a lead time of 7 seconds is better than 5 seconds regarding the quality of the handover and influences safety. Allocating the extra 2 seconds for the lead time had an impact on the intensity of accelerations and de- celerations performed by the drivers as well as the general SA upon resuming control of driving.
Mok et al. (2015) found that lead times of 5 and 8 seconds were sufficient for a safe handover, which also influenced the trust of the participants compared to the shortest tested lead time of 2 seconds. It should be borne in mind that the time possible for deliv- ering the TOR is determined by the limited range of the sensors of the vehicle and their capability in predicting the boundaries of the system (Saffarian, et al., 2012; Gold et al., 2013; Wan & Wu, 2018).
Wan & Wu (2018) recommend a lead time of 10 seconds or longer for handovers.
However, a longer lead time does not necessarily lead to a more successful handover.
Drivers might not react to a longer lead time with the same intensity as a shorter lead time. In addition, providing an exceedingly long lead time might not be possible due to the limitations of the technology. (Saffarian, et al., 2012; Gold et al., 2013; Wan & Wu, 2018)
2.3.5.1 Modality of take-over requests
Situation awareness (SA) has a vital role in preparing the driver to take control of the vehicle during handovers in a safe manner. There are different ways to enhance the driver’s SA with the use of alerts and prepare him / her to take control of driving. Using different types of alerts, including multimodal alerts are the most common way to prepare the driver (de Winter et al., 2014; Iqbal et al., 2017; Petersen et al., 2019). In level-3 automation vehicles, the alerts can be delivered using different modalities, which also impact how drivers perceive them, as addressed in various driving simulator studies (Gold
et al., 2015; Iqbal et al., 2017; Ruijten et al., 2018; Banks et al., 2018 Petersen et al., 2019).
The use of multimodal alerts for delivering TORs in semi-autonomous vehicles has been extensively researched. Many studies have shown that multimodal alerts are a lot more effective compared to unimodal alerts in capturing the driver’s attention especially in situations where the driver is involved in secondary NDR tasks. (Petermeijer et al., 2015; Petermeijer et al., 2016; Petermeijer et al., 2017; Bazilinskyy et al., 2018; Holländer
& Pfleging, 2018; Yoon et al., 2018; Salminen et al., 2019)
Bimodal or multimodal alerts are the most effective when they are delivered at ap- proximately the same time. Moreover, the frequency and the intensity of the alerts influ- ence the perceived urgency (van Erp et al., 2015). Multimodal alerts have many benefits compared to unimodal alerts. If the driver misses an audio alert, due to, for instance, lis- tening to loud music, then other forms of alerts can come in handy and prove to even be crucial depending on the situation. Besides, multimodal alerts are perceived as more ur- gent than unimodal alerts (Holländer & Pfleging, 2018).
Multimodal alerts are known to evoke faster reaction times compared to unimodal and bimodal alerts. The most effective forms of multimodal alerts are comprised of audio, visual, and vibrotactile that are usually delivered simultaneously. (Diederich & Colonius, 2004; Van Erp et al., 2015; Petermeijer et al., 2016; Petermeijer et al., 2017; Bazilinskyy et al., 2018; Holländer & Pfleging, 2018)
2.3.6. Driver roles and partial automation
A study by Banks et al. (2018) suggests that partially automated driving is unwise. Using observations from an on-road study, Banks et al. (2018) investigated whether partially automated driving is a poor idea. They conducted thematic analysis of video data that was collected during a previous study by Eriksson et al. (2017) in which a Tesla Model S was used on the road in real traffic in Autopilot mode. The objective of the analysis was to emphasize the importance of autonomous functionality on driver behaviour. The study also aimed to identify specific areas that could show signs of compromised safety (Eriks- son et al., 2017; Banks et al., 2018). According to the SAE J3016-2018 (2018) framework the Tesla Autopilot is classified as level-2 partial automation.
The most common occurrences observed during the on-road drive were related to the audio and visual warnings that were presented in cases where the driver had hands off the wheel for 60 seconds or more while the Tesla Autopilot was active. When the hands of the driver were off the steering wheel for 60 seconds, a visual alert was presented. If the driver did not react to the visual alert, an audio alert was presented 15 seconds after the visual alert. Meaning, that in instances where both alerts were presented, the driver had had her/his hands off the steering wheel for 75 seconds. In cases where the drivers had
their hands off the steering wheel for a substantial period, they would have found them- selves in the role of driver not driving (DND). If the vehicle had reached its ODD limits while the driver was in the DND role, the consequences could have been catastrophic.
(Banks & Stanton, 2017; Banks et al., 2018)
Banks et al. (2018) reported that the initial visual alert (presented after 60s of hands- off driving) was not noticeable enough to capture one of the driver’s attention. The audio alert that was presented after the visual warning, did capture some of the drivers’ attention and prompted the driver to get his/her hands back on the steering wheel. However, it caused confusion. One participant did not react to any of the warnings and was hands-off for 77 seconds. The reason for it was that the participant failed to monitor the internal HMI presenting the alerts.
Although Tesla Model S comes equipped with collision warning systems that are de- signed to alert the drivers when facing potential hazards on the road, they cannot always be trusted to operate as one might assume (NTSB, 2017). Therefore, when driving such a vehicle (SAE level-2), it is essential for the driver to be alert even when he/she is in the DM (driver monitor) role and is monitoring while the vehicle is operating in an automated mode. (Banks & Stanton, 2017; Banks et al., 2018)
The Tesla crash that occurred in May 2016 resulting in a fatality was blamed for a prolonged period of distracted driving. Moreover, an in-depth report conducted by the National Transportation Safety Board (NTSB) regarding the crash states that Forward Collision Warning and Automatic Emergency Braking system that the Tesla is equipped with “is designed to recognize and detect slow, stopped, and decelerating vehicles when they are traveling ahead of the Tesla in the same lane.” (NTSB, 2017)
Banks et al. (2018) suggest that the driver should either be in a DD role or a DND role. DD is described as Driver Driving, which means that the responsibility for complet- ing basic, operational, tactical, and strategic tasks lies with the driver. DND is described as Driver Not Driving and refers to the ADS fully controlling the driving functions of the vehicle, which means that the driving tasks performed by the driver in the DD role would all be performed by the ADS. Banks et al. (2018) propose that DD and DND “are the only two viable options that can fully protect the role of the human within automated driving systems.” Due to the perception of high system reliability, a driver may not monitor the system as much as is required. Furthermore, humans are bad at carrying out sustained monitoring tasks, which makes decay in performance highly likely (Banks & Stanton, 2017; Banks et al., 2018).
The study by Banks et al. (2018) from which the data originated was conducted using a Tesla Model S operating in autopilot mode on real road conditions. It is a vehicle that is classified as level-2 (partial automation) (SAE J3016-2018, 2018). Although, it does demonstrate the behavior and attitude that people have regarding the autonomous vehicle
technology presently available, it does not necessarily mean that people will display sim- ilar behavior in level-3 vehicles with conditional automation.
Since level-3 vehicles can operate for longer periods of time on their own, the driver will have the possibility to disengage from driving and focus on secondary NDR tasks. If the vehicle reaches its ODD limits and the driver is alerted a certain time before reaching a potential collision, it can aid the driver by enhancing his/her situation awareness, which seems to be the greatest factor influencing the mental workload of drivers when shifting between secondary NDR tasks and driving (Gold et al., 2015; Iqbal et al., 2017; Petersen et al., 2019).
Moreover, as previously stated many different ways and methods for alerting the driver a certain period in advance about handing back control of driving functions have been researched (Petermeijer et al., 2015; Petermeijer et al., 2016; Petermeijer et al., 2017;
Iqbal et al., 2017; Bazilinskyy et al., 2018; Holländer & Pfleging, 2018; Yoon et al., 2018;
Salminen et al., 2019). They seem to be more effective than the systems used in Tesla Model S (Banks et al., 2018). Some of these will most likely make it to production vehi- cles with level-3 automation.
2.4. Theoretical framework
Based on the previous studies that I have examined in this chapter, I believe the key ele- ments of shifting the driver’s attention back to driving after a period of DND and prepar- ing for a handover in semi-autonomous vehicles are influenced by the following factors:
- Driver’s situation awareness - Take-over request
- Timing of take-over request - Multimodal alerts
- Trust in automation
Situation awareness of the driver is influenced by presenting him/her with a timely TOR before the handover. The timing of the TOR is crucial because the driver needs a certain amount of time to acquire the necessary situation awareness to prepare for the handover and reorient himself/herself back to driving. The timing also determines the outcome of the handover in terms of safety. The TOR should preferably be delivered using multimodal alerts, as they are more effective in capturing the drivers’ attention compared to unimodal alerts. Besides, enhancing the situation awareness of the driver can increase the driver's trust in automation, which has an impact on the driver’s engagement with the NDR task and handover safety.
Figure 2 presents all the related components of the theoretical framework used in this study. The text ‘On the loop’ refers to gradually bringing the participant back into the
DM role when the vehicle is operating in automated mode and the participant is engaged in the NDR task. It is achieved using an audio alert referred to as a situation awareness (SA) alert in this study, and it presents information about changing weather conditions.
Its purpose is to enhance the situation awareness of the participants.
Figure 2: Theoretical framework
I hypothesize that enhancing the participants’ situation awareness will lead to higher trust in automation (semi-autonomous vehicles), as the participants will be kept informed about what is going on in the environment in which the vehicle is operating. Moreover, they will more likely monitor the road while being engaged in an NDR task. Also, en- hancing the participants’ SA will lead to a safer handover. Based on what I hypothesize, I have developed the following research questions that I will answer through conducting experimental research utilizing a driving simulator.
1. How does enhancing the driver’s situation awareness influence handovers in semi-autonomous vehicles?
2. What is the impact of enhancing the driver’s situation awareness on perceived trust in semi-autonomous vehicles?
3. What is the role of situation awareness alert and take-over request in enhancing situation awareness of the driver?
3. Research Methods
This chapter presents the research methods employed in the study for collecting the data needed to answer the research questions. The chapter also presents the objective of the experiment and discusses the experimental setup, experiment design, and all the things that the experiment comprised of. It also presents the justification for using the research methods that were adopted for collecting the data. Besides, the experiment procedure is explained in detail. Furthermore, the chapter discusses the different types of data utilized and provides justification for it. Additionally, the data analysis is described in the chapter.
3.1. Experiment objective
Situation awareness is an important aspect in the realm of driving, particularly in semi- autonomous vehicles. There appears to be a lack of research that examines the relation- ship of situation awareness on trust and its effect on handovers in semi-autonomous ve- hicles (Petersen et al., 2019). According to my knowledge the effect of situation aware- ness on trust, while varying the driving conditions, particularly the weather, and measur- ing its impact on handovers has not been extensively studied. Helldin et al. (2013) con- ducted a driving simulator experiment in snowy conditions. They varied the intensity of the snowfall. However, they did not present information about changing weather condi- tions. In our experiment, we presented a verbal message using the auditory modality about the changing weather conditions in half of the driving tasks.
According to the definition of Merat et al. (2018) our experiment featured out-of-the- loop state in which the participants were “not in physical control of the vehicle, and not monitoring the driving situation” as they were asked to perform a secondary NDR task during the period of automated driving. By providing the information about the changing weather conditions while the vehicle was operating in automated mode, the objective was to enhance the situation awareness of the participants and measure its impact on perceived trust and handovers (Iqbal et al., 2017; Endsley, 2018; Petersen et al., 2019).
Also, the purpose of presenting information about the changing weather conditions was to keep the participants on the loop (Louw et al., 2015). Simulator study conducted by Cohen-Lazry et al. (2017) discovered that presenting situational information to the participants results in more glances to the road when the vehicle is driving in automated mode, and the participant is engaged in an NDR task. One of the main objectives of our research was to keep the participant engaged by having them monitor the road. We tried to accomplish it by delivering an auditory situation awareness (SA) alert in half of the driving tasks approximately 30 seconds before the TOR.
The information we provided via the SA alert can be considered as being Situation awareness of the Vehicle (SAV) (Filip el., 2016). It influences the perceived trust of the driver in the system since it is one way in which the vehicle displays awareness of the
environment in which it operates and projects its capabilities of driving by itself (Filip el., 2016; Petersen et al., 2019).
A driving simulator located at the Kauppi campus of Tampere University was the method used for acquiring the data for the study. Given the nature and purpose of the study, it was the only way possible for obtaining the data. Since AV technology, espe- cially comprising of level-3 conditional automation is relatively new and is being tested in driving simulator environments, it was not possible to acquire the data in any other way (Surden & Williams, 2016; Iqbal et al., 2017; Petermeijer et al., 2017; Wan & Wu 2018; Bellet et al., 2019; Petersen et al., 2019).
Although conducting an on-road study utilizing, for instance, a vehicle equipped with lower partial automation, such as a Tesla, might have been possible. However, it would have required a lot more planning and effort (Eriksson et al., 2017). Furthermore, it would not have been safe and ethical for the study to be carried out on an actual road due to the potential risks posed to the participants. Nonetheless, the risks might have been mitigated by having a safety driver present during the experiment as was done in the Eriksson et al.
(2017) study (Banks et al., 2018). However, data obtained using a vehicle that lacks con- ditional automation functionality would have perhaps lacked validity, given that the ob- jective of the study required the use of conditional automation capabilities (Bellet et al., 2019).
The experiment was conducted in a medium-fidelity simulator. Simulator fidelity is the extent to which the driving simulator mimics real-world driving. A driving simulator with high-fidelity consists of a full-feedback motion base and provides a “close to a 360°
field of view projected onto large screens”. A medium-fidelity simulator contains multi- ple displays and a limited view. It does not include a full motion base. A low-fidelity simulator has a fixed-base and usually consists of a single display and a “video game steering wheel” (Wynn et al., 2019). The simulator in which our study was conducted comprised of three planar projection surfaces providing 180 degrees field of view (FOV).
However, it lacked a motion base, therefore it was a medium-fidelity simulator (Wynn et al., 2019).
Driving simulators provide a safe setting for conducting experiments involving hu- man subjects (Eriksson et al., 2017). They present a highly controlled environment in which various scenarios and driving conditions can be tested at will. Also, collecting data is effortless in a driving simulator study compared to an on-road study. Moreover, testing dangerous scenarios in a simulator is also possible without the risk of causing injury to the participant. (de Winter et al., 2012)
There are, however, some disadvantages involving driving simulators. Simulators may not offer a realistic experience, especially if it is a low-fidelity simulator. Some peo- ple can also experience simulator sickness, which is another drawback of simulators (de
Winter et al., 2012). Moreover, the objective of the research and the type of simulator used regarding simulator fidelity can also impact the results achieved. High-fidelity sim- ulators do not always guarantee the most valid results. In some cases, low-fidelity simu- lators have achieved valid findings, whereas high-fidelity simulators have not. Therefore, the objective of the study can dictate the validity of the results depending on the simulator fidelity. (Wynn et al., 2019)
For testing the transfer of control from the vehicle to the driver and vice versa, driving simulators have been found to have a strong positive correlation with on-road driving (Eriksson et al., 2017). Additionally, Eriksson et al. (2017) concluded that “medium-fi- delity, fixed based, driving simulation is a safe and cost-effective method for assessing human-automation interaction, and in particular control transitions in highly automated driving.” Therefore, conducting the experiment for this study using a driving simulator was an obvious choice.
3.2. Experimental setup
The simulator environment was comprised of three 3300mm wide planar projection sur- faces. Vertically the projection area started from 350mm from the floor and was 2350mm tall. The display setup included Optoma EH515 projectors that had a 1920x1200 resolu- tion. The simulator was implemented by Creanex Oy. It was running a Vehicle version 1.9.0. The setup also consisted of a separate display for the desktop computer on which the simulator software was run. The desktop display was not visible to the participants.
The operating system of the desktop computer was Windows 10. Its specifications were the following:
Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz 3312MHz
8062MB RAM
NVIDIA GeForce GTX 1070
Moreover, the setup comprised of a multimodal feedback system. The system was based on a USB connected control box that was installed on the PC, which was running the driving simulator software. The haptic feedback was delivered via a cushion placed on the seat of the driving simulator. The dimensions of the haptic cushion were 50x40x5 cm. It comprised of 4 actuators that were embedded closer to the front side of the cushion.
The haptic signal was delivered using a sinusoidal wave frequency of 150Hz. The length of the haptic signal was 500ms. It included three 100ms bursts. There was a pause of 100ms between the bursts. Auditory feedback was delivered via a single (mono) active speaker, which was located on the right-hand side of the participants. Figure 3 displays the haptic cushion on the simulator seat as it was placed during the experiment. The yel- low rectangular shapes show the location of the actuators.
Figure 3. Haptic cushion
Figure 4 contains part of the simulator setup consisting of the steering wheel, the speaker, the gear shift lever, and the lever used for taking back manual control of the vehicle. Also, the acceleration and braking pedals are visible in Figure 4.
Figure 4. simulator setup
For recording the audio of the interviews, I used my mobile phone. The make and model of the phone was OnePlus 5. The video was recorded using a separate camera. The make and model of the camera was Sony HDR-XR500VE.
3.3. Experiment design
A within-group design with situation awareness (SA) as the independent variable was adopted for the experiment. Hence, the SA alert was manipulated. The reason for manip- ulating the SA alert was to learn whether it has an impact on the perceived participant trust in automation, and perceived safety during handovers. Thus, the dependent variables of the experiment were perceived participant trust and handover. The participants were asked to perform the secondary NDR task during all of the driving tasks as soon as the vehicle entered the automated driving mode. There were six driving tasks in total per- formed by each participant. Three of the driving tasks included a SA alert, while three did not. Therefore, we had two different conditions:
A. SA, NDR B. No SA, NDR
There were several reasons for implementing a within-group design: in a within- group design, it is not necessary to recruit as many participants compared to a between- group design. Moreover, in a within-group design, each participant is exposed to multiple conditions, which is what we wanted. Another benefit of adopting a within-group design is that it effectively isolates individual differences. Additionally, the objective of the study was not to compare the performance of one group of participants against another. Had
Gear shift lever
Manual mode lever
Speaker
that been the case then a between-group design would have been more appropriate. (Lazar et al., 2017, chapter 3)
To control the impact of the learning effect, which is usually a problem with a within- group design, ordering of the conditions was counterbalanced using a Latin Square de- sign. Moreover, the participants were provided adequate time to get familiar with the driving tasks by performing a test task before beginning the actual driving tasks. This was done to get the participants acquainted with the driving environment and to reduce the impact of the learning effect. Also, since each participant is exposed to multiple condi- tions in a within-group study, fatigue can become an issue due to the duration of the ex- periment session (Lazar et al., 2017, chapter 3). We observed this during the initial 3 sessions.
Changes were made to the experiment design after the first three experiment sessions, because we noticed the length of the session was approaching two hours, which was too long, as it introduced fatigue. It is generally recommended that the length of a single experiment session should be between 60 - 90 minutes (Lazar et al., 2017, chapter 3).
Also, one of the first three participants stated that the session was very tiring due to its length. Thus, we decided to make the experiment session more compact by reducing the time it took to complete the session. At first, the experiment session featured eight driving tasks, with each task lasting approximately 3 minutes and 40 seconds. After making changes to the duration of the experiment, the number of driving tasks were reduced from eight to six. However, the length of the driving tasks remained the same.
The experiment also featured a post driving task subjective questionnaire for measur- ing the participants’ perceived trust in automation. It was presented to the participants after completing each driving task. In addition, the experiment session consisted of a con- sent form, a background questionnaire, simulator sickness questionnaires (SSQ), and an interview. The interview took place at the end of the session after the participants had completed all of the driving tasks.
The participants were asked to fill in the SSQ twice, once before starting the driving tasks and again after completing them. The reason for administering the SSQ twice was to rule-out symptoms that could already be present before beginning the use of the simu- lator. Once the participants had completed the driving tasks, the SSQ was presented again to see if the participants had suffered from side-effects of using the simulator (Bouchard et al., 2007).
3.4. Driving tasks
Each driving task consisted of a driving event and changing weather conditions. The par- ticipants experienced three different events that required them to take control of the vehi- cle. Since the total number of driving tasks presented to each participant was six and the
