Dealing with Conflicting Information - Will Crews Rely on Automation

Proceedings of the Fifth Australian

Aviation Psychology Symposium,

November, 2000, Manly, Australia

Dealing with Conflicting Information - Will Crews Rely on Automation?

KATHLEEN L. MOSIER, JEFFREY KEYES, AND ROBERTA BERNHARD

San Francisco State University

Introduction

Automation Bias

The availability of sophisticated automated aids in modern aircraft feeds into a general human tendency to travel the road of least cognitive effort (Fiske & Taylor, 1994). Omission and commission errors resulting from automation bias, the tendency to utilize automated cues as a heuristic replacement for vigilant information seeking and processing, have been documented in professional pilots and students, in one- and two-person crews. Underlying causes of automation-related omission errors have been traced in part to vigilance issues. Crews count on automation to provide the most salient and reliable information about flight progress and system status, and often "miss" events that are not pointed out to them by automated systems. This tendency is exacerbated by the fact that operators often have incomplete or fuzzy mental models of how various modes of automation work (Norman, 1990; Sarter & Woods, 1992). Additionally, the opaque interfaces of many automated systems, which provide only limited information about actual status and criteria for conclusions, make it difficult for even vigilant decision makers to detect errors.

Pilots also rely on automation, particularly automated warning systems, to guide their actions, as is the case with GPWS (Ground Proximity Warning System) or TCAS (Traffic Collision Avoidance System) aural and visual displays. Automation-related commission errors result when pilots follow automated information or directives inappropriately (e.g., when other information in the environment contradicts or is inconsistent with the automated cue). Automation bias and resultant errors have been documented in current flight crew ASRS (Aviation Safety Reporting System) incident reports (Mosier, Skitka, & Korte, 1994), as well as in controlled studies using students and professional pilots as participants in solo or team configurations. In these studies, professional pilots were sensitive to the importance of correctness for critical flight tasks, and made fewer errors on events involving altitude and heading errors than frequency discrepancies (Mosier, Skitka, Heers, & Burdick, 1998; Mosier, Skitka, Dunbar, & McDonnell, in press). Training for automation bias reduced commission errors for students, suggesting the importance of early intervention and training on this issue (Skitka, Mosier, & Burdick, 2000; Skitka, Mosier, Burdick, & Rosenblatt, 2000). In another study, participants in a non-automated condition out-performed those using an automated aid during equivalent failure events (Skitka, Mosier, & Burdick, 1999).

Commission errors: Preference for action? It has been hypothesized that commission errors may be related to a tendency of pilots to take action in a given situation as a way of maintaining control. This notion was supported in an exploration of pilots' use of conflict probe displays (Cashion, Mackintosh, McGann, and Lozito, 1997). Several of the crews in that study (6 out of 20) chose to maneuver during an event in which the "rules of the road" suggested that action was not required of them, but rather of the other aircraft involved in the conflict. The tendency toward action makes sense for pilots, as proactivity has typically been associated with superior crew performance (e.g., Fischer, Orasanu, & Montalvo, 1993; Mosier & Chidester, 1991). Operators may feel that taking an action gives them more control over the situation, or they may be taking action as a way of dealing with a potential situation immediately to eliminate the risk it may develop into a problem and the workload involved in monitoring it.

Making information sources equally salient. Most of the studies mentioned above utilized low-, medium-, or high-fidelity flying tasks, and features of the information display may have fostered the tendency to rely heavily on automated information. For example, automated information is typically designed to be more available, compelling, and salient than other kinds of information. What happens when other sources of information are made equally salient to automated cues?

A series of paper-and-pencil scenario studies was conducted with student participants (Skitka, 1999). Scenario 1 was modeled after a simulator event used by Mosier, Palmer, and Degani (1992), and involved a decision about which one of two engines was more damaged and needed to be shut down. Automated monitoring devices indicated that one engine was severely damaged, whereas traditional system indices suggested that the other engine was the source of the problem. Reliability of each of the indicators as well as relative risk associated with making a mistake were systematically varied. In this situation, people on average responded in very rational ways. Under both high and low risk, they opted to go with the recommendation that had the highest probability of being correct; and when both sources of information had an equal probability of being right, they showed no systematic preference for one source over the other. In sum, when other sources of information were presented on a par, and with equal salience to automated recommendations, automation bias did not emerge.

Scenario 2 involved a car with the latest in on-board computers, and presented participants with conflicting information from the computer and the engine gauges. In this study, the source of an “action” recommendation was varied, as was the reliability of each of the sources and the risk involved in choosing each action. Results indicated significant main effects for action and reliability, and an action by risk interaction. In this case, in addition to a general tendency to choose action and follow reliable information, people were particularly likely to exhibit a preference for action – whatever the information source and whatever its reliability – when low risk was involved.

Scenario 3, a nuclear power plant scenario, was constructed so that there were risks associated with failing to act, rather than with acting. Again, the reliability of each information source (computer or gauges) and the source of the action recommendation were systematically varied. The results of this study conceptually replicated the findings of the airplane and car scenario studies, in that no systematic preference for automated information was observed. When information from non-automated sources was presented with equal salience to automated information, automation bias effects did not emerge. When risk varied as a function of inaction, rather than action, people again acted to minimize the potential hazard associated with their choices. Although there was a preference for action under both conditions of manipulated risk, the effect was much stronger when high risk was associated with inaction.

The present studies. The scenario studies described above suggest that the tendency toward automation bias may be at least in part a function of the way automated information is displayed. When individuals were made equally aware of conflicting information from all sources, they tended to be more analytical in their decisions, and displayed no systematic tendency to prefer automated cues over others. These findings suggest that in decision-making contexts such as the glass cockpit, it may be the display features of automated information, such as its salience and opaque interface, that causes it to be used as a heuristic, rather than the quality of the information itself. If all information in the cockpit was displayed equally compellingly, the phenomenon of automation bias, at least with respect to commission errors, might be mitigated - or even disappear. The present studies, which focused on regional, or Part 135 operations (i.e., commercially-operated aircraft with fewer than 50 passenger seats), were conducted as an initial exploration of this possibility.

An ASRS analysis of Part 135 reported incidents was conducted to gather preliminary information. It was intended to accomplish three tasks: a) to document the existence of issues or problems related to the use of automated and other information in Part 135 operations; b) to trace the patterns of action that pilots take in various types of incidents, especially when information conflicts are present; and c) to identify scenarios that would be good candidates for the scenario study which was to follow. In the scenario study, pilots were asked to choose between two sources of information that conflicted with each other. The information was presented in a paper-and-pencil format, to give both sources equal salience. The intent of this study was to determine whether a tendency toward automation bias would be a factor in pilot decisions when data were presented in a manner that gave information from all sources equal salience. We also wanted to examine the impact of perceived risk and action vs inaction on their choices.

ASRS Study

Method

Procedures. Data for the ASRS study were obtained from the ASRS incident database, and covered reports submitted between 1994 and 1998. Using several broad search queries, such as aircraft type, passenger operations, and automated displays or instruments (e.g., EFIS, integrated navigation), we created a preliminary sample of 1,200 reports that were submitted by Part 135 pilots. Each of these was screened for appropriateness for our study of conflicting information and/or automation-related errors, and candidate reports were subject to a second scrutiny. We were particularly interested in incidents involving conflicting information from different sources. Through this procedure, we ultimately created a sample of 189 ASRS reports involving some form of automation and/or information conflict. These reports were coded in two ways. First, the variable fields present in the ASRS database were entered directly into a data file to collate information on aircraft type, phase of flight, and incident type and resolution as coded by ASRS personnel. Second, the narratives entered by the pilots were coded to identify incidents in which information conflicts were present, and to track the actions that were taken in each case. Narrative coders cross-checked each others' work until 90% inter-rater agreement was attained. Incidents were coded with respect to the sources of information that were cited concerning the critical event, whether the sources provided consistent information or were in conflict, and how the incident was resolved. Factors such as fatigue, distraction, and risk level were coded whenever mentioned in the report.

Results and Discussion

Descriptive Information. The final sample of 189 reports represents approximately 16% of incidents reported by Part 135 pilots, demonstrating the presence of automation-related issues and incidents in regional operations. That number can be expected to increase as regional operators upgrade to more sophisticated aircraft types. Reports came from pilots of at least 13 different aircraft types. Breakdown of the data by aircraft type showed that the categories represented by the largest numbers of reports were "commercial, fixed wing" (n = 71), followed by Brasilia EMB-120 (n = 29), SF-340A/B (n = 29), and Beech 1900 (n = 20).[1] With respect to phase of flight, the largest number of incidents occurred during climb (n = 64), cruise (n = 40), descent (n = 31), and approach (n = 28).

Narrative Coding. The richest source of data for this study was found in the narratives, which offered candid information about the incidents in the words of the reporting pilots. From the narratives, incidents were categorized as: a) system/mechanical failures or faulty warnings (n = 67; 35.4%); b) traffic incidents (n = 71; 37.04%); and c) altitude, heading, or route failures or errors (n = 62; 27.51%). The most commonly reported cause for traffic incidents was ATC error (n = 20), and crew error and/or incorrect system set-up was cited most often as the source of altitude, heading or route failures (n = 47). Twelve incidents involved automation-related omission errors, and nine involved automation-related commission errors.

Whenever possible, we coded factors that pilots noted had contributed to the incident they were reporting. Distraction was cited as a contributing factor in 23.81% of the incident reports. Failure to double-check indicators and/or fatigue were occasionally cited as contributing to incidents (15.9% and 7.41% respectively). Distraction and failure to double-check indicators were most often associated with altitude, heading or route errors. Risk was mentioned in 9% of the reports.

In 103 reports (55%), pilots reported dealing with conflicting information from two or more sources. Of these, 70 involved some form of automated information that conflicted with another source. Sources coded included:

1.  Physical - visual or auditory cue

2.  Other human source (ATC, passenger, etc.)

3.  Checklists/handbooks

4.  "Steam gauges" - not electronic data

5.  Electric warning system (e.g., master caution warning lights, fire lights, oil lights, etc.)

6.  Electronic warning system

7.  Computer system (e.g., EFIS, EICAS, FMS, FMC)

8.  Autopilot

9.  TCAS I

10.  TCAS II

11.  Autofeathering system

12.  Weather detection system

13.  GPWS

14.  Unknown/unable to determine

Most conflicts involved instrument or electronic data that was contradicted by some human element, such as ATC information or a crew member's physical sensations (i.e., sees, hears, or feels something). Pilots tended to trust TCAS and their own senses over instruments or ATC information. A few incidents (n = 7) involved conflicts between traditional sources and electronic data (not including TCAS), and in all but one of these, crews followed the traditional source. Many of the incidents involving conflicting information were traffic incidents, and involved a conflict between TCAS and some other source (e.g., ATC or visual cues; n = 33). High risk was most often cited as a factor in conjunction with traffic incidents. Analyses indicated that, when TCAS information entailed taking evasive action, crews typically followed TCAS recommendations - even when visual information contradicted the need for the maneuver.

These incident reports document the existence of conflicts among information sources in the cockpit. It seems that any preference for automated information, however, is impacted by the action orientation of the information. In many cases, crews tended to follow their own senses or traditional sources of information rather than preferring electronic data. When TCAS was involved, however, crews generally followed its action directives, regardless of contradictory information. This strategy may be related at least in part to procedures prescribed in conjunction with TCAS directives, and also supports the notion of a "take action" tendency.

Scenario Study

Method

Scenario Development. Scenarios were created using incidents from the ASRS analysis and from previous research studies (Fischer, Orasanu, & Wich, 1995). Care was taken to ensure that scenarios were representative enough that they could be responded to by pilots of several different aircraft types. Each scenario conveyed a situation involving conflicting information from two sources: an automated source + either a human source or a traditional indicator. In each scenario, information from one source suggested making some change (action); information from the other source suggested maintaining status quo. Each scenario was followed by two decision options - for example: