Osa III: Uudet näyttökonseptit
6. The Ecological Interface Design Experiment (2005) – Qualitative
6.5 Results
6.5.2 Working practices of the crews
In the analysis of crews’ working practice the focus is to understand what kind of affordances the different display types provide to the operators and how the operators make use of these affordances. The question of interest here can also be formulated as how well the different display types communicate process state and the demands for action to the operators. The indicator for the communicative strength of the display is not the successfulness in process control but what ways of working are facilitated by the different displays. Ways of working represent the internal good of practice.
As has already been indicated we use two different types of indications here.
The first one refers to the persons’ conceptions of work, i.e. orientation. This indicator has a character of a stable background factor that is supposed to modu-late situation specific acting. The second indicator for practices is the habits of
action that we have identified on the basis of analysing each crews’ actual courses of actions in the different scenarios and when working with different display types. With regard to both indicators of practice (orientation and habit of action) we use the same evaluation scale, i.e. interpretative, confirmative, and reactive. This evaluation scale speaks of the adaptiveness of the behaviour (see section 6.3.4.2).
6.5.2.1 Crews’ work orientations
The orientation results were acquired by three independent evaluators on the basis of reading transcribed protocols of the orientation interviews of each process operator. Due to technical failure in recording the interviews of one turbine operator (T) (crew 3) and one reactor operator (R) (crew 2) were lost. It should also be noted that two reactor operators (R) served two crews (one the crews 6 and 5, the other crews 3 and 4). Because the analysis of process control and habit of action were mainly focused on the turbine operators’ action, especially the turbine operators’ orientation is relevant here.
As was indicated in the methods section orientation was conceptualised on the basis of five dimensions, under which a total of 12 indicators were used (bullets in the following list):
1) Conception of the object of activity:
• framing of own work.
2) Conception of the intrinsic constraints of work:
Information:
• type of reference to changes in process
• type of reference to the state of the process Procedures:
• reasons for using procedures
• limits of procedures.
3) Conception of knowledge and constructing knowledge:
• process feel
• conception of alarms
• role of collaboration in knowing
• conception of development.
4) Professional ethos:
• good process operator.
5) Personal sense of work:
• interesting in work
• agent role.
For each indicator descriptions were constructed that would indicate different types of orientation to work. The grading follows the generic background dimension of interpretative-reactive that was adopted as a dimension to indicate a basic epistemic attitude of the human being’s relationship to the environment.
Grading 2 was used for an interpretative attitude, 1 for a confirmative attitude, and 0 for a reactive attitude (the content are described in Table 14 in section 6.3.4.2). These ratings were used by three independent evaluators (A, B and C).
The inter-rater reliability was counted using Cohen’s Kappa (weighted Kappa).
The statistics indicates that a “moderate” agreement was achieved between each pair of raters A, B and C (A/B K=0.513; A/C K=0.587; A/C K=0.546).
Each rater’s results were included in the final result, which is provided in Table 26. The table indicates how many ratings of 2 (interpretative), ratings of 1 (confirmative) and ratings of 0 (reactive) each crew achieved with regard to the different evaluation elements. On the basis of the dominance of ratings we inferred the final evaluation of the crews’ orientation (last column in Table 26).
The crews’ orientations were classified to express either interpretative (dominance of ratings 2), confirmative (dominance of ratings 1) or reactive orientation (dominance of ratings 0).
Table 26. All ratings of three evaluators concerning crew members’ orientations and the final evaluation agreed for each crew (last column).
Role Crew No of 2 No of 1 No of 0 Final evaluation
T 1 22 8 6 Interpretative orientation
R 1 19 15 2
R 4 / 3 13 15 8 Confirmative orientation
T 4 12 15 9 T 6 5 20 11
R 5 9 9 18 Reactive orientation
R 6 / 5 6 15 15
T 2 5 12 19 T 5 2 15 19
Typical of the interpretative orientation was to connect the tasks of the daily work to more global connections and process phenomena and functions. In the confirmative orientation the daily work is seen as appropriate and active accomplishing of tasks. In the reactive orientation type work was considered as well mastered routines and actions to be launched on external demand. The results indicated that only the both operators of the crew 1 were rated to portray the interpretative orientation. Both operators of the crews 4 and 3, and also the
turbine operator of the crew 6 portrayed the confirmative orientation, which is basically quite ok. Both operators of the crew 5 and the turbine operator of crew 2 manifested reactive orientation. As Table 26 indicates, crews were typically homogenous with regard to orientation. This is according to expectation because orientation is an expression of a joint culture that a crew develops within its own community of practice.
6.5.2.2 Description of crews’ habits of action in all scenarios and display types
Habits of action were evaluated with the aid of a five-element analysis scheme.
Table 27 provides the indicators that were used in the evaluation. Indicators are behavioural markers for which we developed scenario-specific references. The scenario models were used as the source for defining the criteria. An example of the developed criteria is given in Table 27 below. It concerns the scenario In2.
Table 27. Habit of action indicators and scenario-based criterions for grading of Scenario IN2.
Indicator Criterions for:
Interpretative – confirmative – reactive Way of searching information
Search of information Questioning, comparative, cumulative – recording, relating – passive, based strongly on alarms (E.g.
monitoring drain switching, condense/feed water balance in effect increase)
Connecting changes to global functions Functional inferences vs. component level statements (E.g. condense/feed water balance (462VA5/VB5, 463VA4, 314VB4), continuation of effect increase, heat transfer efficiency (seawater temperature)
Connecting changes to task Anticipating, structuring task – focus on on-going sub-task – weak connection to task (E.g. drain switch, bypass, effect increase)
Observing effects of own operations Activeness of monitoring: active – medium – passive (E.g. manual operation of 463VA20 and 463VB19, reset of bypass, manual operation of 462VB5, requesting maintenance of 462VA5) Way of using operational possibilities
Way of managing Active agent, prompt actions,
taking responsibility – adjusting – lead by the process, slow actions (E.g. reset of bypass, fixing of failed components (462VA5))
Prioritising of tasks Focusing on relevant tasks: appropriate – medium – inappropriate (E.g. drain switch, reset of bypass, controlling FW flow by manual operation of 462VB5)
Use of adequate methods Adjusted to situation – use of generally valid methods – use inadequate methods (E.g. changing of FW pump, operation of 463VA20, manual operation of 462VB5)
Synchronisation of operations Not relevant
Taking into account the causes and the severity of disturbance in stabilisation / normalisation
Comprehensive – partial – deficient (E.g. controlling FW/condense balance to avoid shutdown)
Way of reasoning the cause of deviations Localising or clarifying the cause and necessary operations
Discussing the cause (type, place, size) – taking failure as given – minimal consideration of the failure (E.g. high level of 462EB1)
Way of using procedures
Way of using instruction in process control Plant start-up procedure: procedure use was not observable
Way of collaboration
Way of communicating Commenting on observations, intention towards common interpretation – informs of observations – minimal communication (Based on all material) Way of team working Proactive commenting of own aims and actions –
post-hoc informing of own aims and actions – self-sufficient acting (Based on all material)
Way of contacting outside CR Participative contacting – Requesting contacting – minimal contacting (Based on all calls to outside)
The courses of action constructed for each crew, as well the final debriefing interviews, were used as data for the evaluations. The evaluations were made together with two researchers who had analysed the courses of action. In the analysis, the turbine operators’ actions were more in focus because the video had been recorded from his point of view. The reactor operators’ actions, however, played a role in each task performance and had an effect on the evaluations, too.
The used rating scale was 2 (black), 1 (grey), 0 (white) in which 1 was considered as a norm indicating a confirmative way of taking into account the core task demands of process control work. Rating 2 was an upward deviation indicating interpretative account of the core-task demands of the process control work. Finally 0, i.e. reactive way of acting, was given when core-task demands, as reflected via the indicators, were partly neglected.
Table 28 provides the overall results of crew’s habits of action in all scenarios.
As can be seen there are differences in crews’ habits of action even though process control performance was seen to be equal (see Table 15). We have not made statistical analysis of habits of action (as we have done in some previous comparable studies) because we basically find quantification inadequate means to understand the differences in operators’ sense making of their work.