Heuristic evaluation of the usability of the Fortum IRD pilot

Three members of the VTT research team carried out a heuristic evaluation of the IRD-displays. Since the heuristic evaluation of the IRD-displays is based on screenshots and paper images of the displays, the evaluation will concentrate on basic design features of the displays. More complicated issues such as how these displays are integrated with the other displays (e.g. displays of the operation workstations and other large screen displays) in the control room and how they are controlled and managed are not considered.

The heuristic evaluation of IRD displays was carried out in January 2008.

Even though the design process was basically finished, some improvements were made after that date. They were mainly related to the features that were not as a target of the heuristic evaluation.

OverallUsability

Figure 26. Numeric evaluation of the main aspects of the usability of IRD-displays. For scales see Appendix N.

5.5.1 Numeric evaluation of the visual usability of the IRD displays The main part of the heuristic evaluation was a detailed qualitative assessment of the visual usability of the designed displays. These results are presented below in the next section (5.5.2). All the three evaluators also assessed the displays according to their visual layout and quality, consistency, familiarity and overall usability by using a Likert-scale from 1 (low usability) to 5 (high usability).

Overall, the visual usability of the display was thought to be at the moderate

level (see Figure 26). The IRD-displays received the lowest scores for readability, clarity and familiarity of notations. The highest scores were obtained on questions related to consistency and familiarity of terms and measures.

5.5.2 Qualitative assessment of the visual usability of the IRD displays Display normalization

The evaluators see that normalization of information that is shown in bar and trend graphs is a novel feature that may help operators to detect deviations very early. On the other hand, it is possible that operators lose touch of what the exact values are, which may be harmful in the stabilization of the system and diagnos-ing of the failure state. One thdiagnos-ing that might harden the use of mini-trend and bar graph information is that individual bar or trend elements do not have identifica-tion labels or other types of identifiers attached to them.

Layout and grouping of display elements

A huge amount of information is presented, and the display is full of detailed information. However it seems to be that only relevant data is displayed.

Overall, more empty space is needed. On the whole, information on the display is quite nicely grouped. For example, on the reactor side the direction of flow can be quite easily seen, and the key components such as the reactor and the steam generators can be easily distinguished from other components. On the other hand, the displays are cluttered on a close view, and the layout seems to be somewhat inconsistent. It can be said that the clarity of presentation varies from one part of the display to another. Especially at the turbine side many elements seem to float freely in the display, and their connections to other symbols remain unclear. In addition, the order of bar graphs and trends (indicating pressure, temperature and flow) that are related to particular components vary from component to component (e.g. TB20 vs. TH-tanks) which increases inconsistence.

The suggestion is that the style guide would provide guidance on how different symbols/elements should be built together and what should be their relation to other elements.

On a general level the physical structure of the system cannot be seen very clearly. The layout of the mimic diagram can be confusing since it does not always show logical relationships between components. Plant components presented along the mimic lines cannot be easily identified because of the lack of labelling, and the origin and destination points of pipelines are not labelled. It is disturbing that pipelines are presented only partially: They cease to exist abruptly or they cross each other. It can also be difficult to perceive to what component they are referring to. For example, in case of YB11, YB15, YB13, the pipelines surround trends and go through them. The inconsistent presentation

of the pipelines might be one reason why it is difficult to parse the display into meaningful subgroups. The meaning of the arrowhead attached to the pipelines is not clear; are they indicating the direction of flow or that the pipeline is continuing to the next short piece of pipe. Arrows at the end of the pipelines are also quite small, and therefore they may be difficult to see from the distance. Overall, the main structure and the main components of the process cannot be easily distinguished from each other. Nor is the direction of the process easily perceived.

The fact that all main functions, processes, systems and components that are typically shown on this kind of process display are not presented, makes it difficult to ‘read’ the display. Normally, they provide context to the critical information.

When this information is not displayed, essential contextual cues are lacking.

Use of colours

Overall, the colour set seems to be carefully selected: Alarms can be easily detected by colour, and colour coding has been efficiently used for segmenting the display. For example, background colouring helps to separate the display areas from each other. However, some colours may be difficult to distinguish from other ones. For example, the colour green or blue may be difficult to detect if the symbols are very small. The green colour is also used in a way that it indicates many different things. It is also used in many different symbols which have decreased its value and the colour change in single component is, therefore, not easily perceived. The colour yellow, used in the indication of alarm states, cannot be easily detected against the grey background. Also the contrast between the labels and the background is not high enough. For example, it is difficult to read grey letters that are located on a dark gray background.

Information presentation and aggregation of information

Some new forms to visualize information are used. For example, there are new symbols for the presentation of pressure, temperature and flow. Even though the pump, valve and tank symbols are familiar, their colouring is different. Mini-trends and bar graphs have not been presented in this way in other applications.

In some parts of the display it is difficult to interpret to which component the graphs are referring to. For example, it might be difficult to know what the eight trends are representing at the left hand side of the display (e.g. the condensing set with reheat, RB; the condensate, RN). Generally, the symbols are quite distinct, and, for example, pressure, flow and temperature symbols can be quite easily distinguished from each other. Alarms are also clearly connected to the target component/symbol. However, when connected to a pump symbol alarm frames obscure pump’s load information. Due to the density of the display, some components and elements might mask each other. For example, some numerical values are so close to each other that they can be read together (e.g. TH40 tank:

liquid level and pressure); also some labels can be read together (e.g. T-kansi,

Alik Max, Tmax, Kvara). In addition, some component labels are even located behind the symbols or digits.

Overall, symbols and alphanumeric characters are of sufficient size. Titles and labels and numerical characters can be easily seen and read. Some elements are too small, however, and detailed information cannot be seen from the distance.

Due to small size and occasionally occurring partial masking, elements of the graphs (e.g. set-point values, expected values and actual values) do not distinguish very well from each other, and they are not easily identified from distance. In order to identify many of the small components operators have to either remember/where they are located or they have to move closer. In style guide it is thought that one of the useful features of the IR-displays is that they can be used differently at different distances. To our opinion, this is a problem, not an advantage; since operators do not necessarily have an opportunity to walk closer to see clearly the details of the display. Neither have they any reason to walk there if they cannot accomplish operations through these displays. Overall, there is a large variation in element size, and too many sizes are used. For example, there are at least five font sizes in use, and the logic behind the use of this variation remains unclear. It also seems to be that the same type of components can be presented in two different sizes (e.g. diesel generators vs.

other generators).

Even though pump, breaker and turbine symbols are clearly distinguishable, some statuses of pumps may be difficult to perceive from the distance. For example, concerning Running 75 % speed, it may be difficult to perceive the white line from the background of the control room. Concerning the presentation of graphs, green diamonds expressing expected valve positions cannot be perceived from the distance. Also, the thin lines in the trend graphs are so faint that they are difficult to perceive.

Labelling is insufficient and used in an inconsistent way, and different types of display elements do not contain appropriate, distinct and unique labels. It is sometimes difficult to know what component the title or label is referring to since they are sometimes placed far away from their target element. Labels and identifiers are also inconsistently located around the element. Sometimes the label is on the left/right side of the component, sometimes above/below of it (e.g. the left corner of the display, titles of the reactor’s intermediate cooling system, TF and service cooling water system, VF). In some parts of the display the labels occlude other information, or they themselves are obscured. There are also some inconsistencies in the use of capital and lowercase letters in labels.

Therefore, it can be said that the hierarchy of the labels is not clear. Adequate labelling could also offer navigation links to the other displays and in this way help operators to move from one information source to other one. The style guide does not provide much guidance on these issues.

Some words are abbreviated in unconventional ways, and some abbreviations include punctuation. The logic of the use of abbreviations remains somewhat unclear. For example, some abbreviations are odd and apparently unfamiliar

(e.g. Raj Sauvak, ABO, Alik max), and some words are abbreviated even though there would be room for presenting the whole word.

Normalization and Dull Screen principle are well explained in the style guide, but otherwise the design rationale is in many ways deficient. For example, explicit rules are not given how the different shades of background colour and specific colours are used. There is an inconsistency between the Style Guide and the displays in the presentation of control rods in the Style Guide: ‘0’ is displayed at the top of the symbol, on the display ‘250’ is shown at the top.

5.5.3 Intermediate conclusion

To our opinion, it is important that in the evaluation of the new design the design principles are taken into account, and the prototype is evaluated in the light of these principles. On the other hand, we also think that it is fair to evaluate the new design by using the same criteria as for other types of displays especially if it is ever aimed to be implemented in real NPP control rooms.

It seems to be that in order to be able to apply the IRD design principles far too many compromises have been made. Perhaps a better solution would be that one display that is based on IRD principles would be designed for rapid detection of changes in the main process parameters. Overview displays, on the other hand, should be based on plant mimics, and IRD principles should not be applied in their design.