The experiments documented in the publications compiled for this dissertation were conducted to study the reasoning derived from the visualization and how this affects the clinical decision making process at an individual level. By applying quantitative and qualitative research, it was possible to compare visualizations in a clinical con-text and provided better understanding on what makes a good visualization. The studies provide documentation on the processes and activities involved in the visual reasoning and knowledge discovery that took place in the experiment sessions. By utilizing evaluation methods such as the insight-based and usability testing with cog-nitive walkthrough, the documented experiments can serve as blueprints for future studies.
To study and compare how different clinical data visualizations affect visual reason-ing (Publications I — IV).
The objective was achieved in all the publications. Publication I compared five vi-sualizations following insight-based methodology. The study collected the insights of non-medical experts and compared their values. In the study, participants evalu-ated the overall health of the modelled patient based on the data visualization. This exercise in judgement reflected the overall understanding that the participants were able to obtained from the visualization. The study demonstrates that the visualiza-tion affects visual reasoning. Visualizavisualiza-tions of the data represented by area size had a negative effect (poor performance and data understanding), while angular, tabular, and hGraph representations had a positive effect (deeper understanding of the data and the underlying conditions of the modelled patient).
Publications II and III studied the performance of the execution of tasks by par-ticipants in a laboratory. The objective was to evaluate the level of understanding the
participants gained from looking at the clinical data using the visualization software.
Additionally, usability experts conducted a heuristic evaluation of the system, and the results were positive. The visualization of the modelled clinical data affected the reasoning in a positive way when the tasks were performed correctly. The high com-pletion rate across participation indicates an overall positive result and demonstrates the assistive nature of the visualization in understanding complex clinical data.
Publication IV studied the insights formulated by clinicians using the timeline vi-sualization and the baseline representation. The values of the insights differed greatly depending on the visualization. It can be concluded that the timeline visualization affected visual reasoning positively by enabling clinicians to formulate hypotheses on the patient’s health condition.
To apply a methodology that studies visual reasoning and the decision making process in the assessment of clinical data visualizations (Publications I — IV).
All the publications achieved this objective. Publications I and IV applied insight-based methodology to clinical data visualizations. The methodology falls under the VDAR scenario[23]. It comprises a series of steps to collect insights formulated by participants during the experimentation. As described by North[24]and previously used in bioinformatics[25, 26, 27], it has a structure followed by a series of metrics to determine the degree to which the visualization supports hypothesis generation.
The studies detail the adjustments required to contextualize the methodology for clinical data.
Publications II and III detail the use of UP evaluations to study clinical visual-ization software. These evaluation methods are also recommended by Johnson and colleagues when studying EHR visualizations[76]. The cognitive walkthrough[83]
and heuristic evaluation[82]constitute structured usability methodologies. Addi-tionally, usability questionnaires measured the UP of the application. These ques-tionnaires “collect self-reported data” and analyse the data to produce usability met-rics. These questionnaires have been studied and analysed[76].
To develop and objectively measure the scalability and usability of software that vi-sualizes holistic clinical data (Publications II and III).
The objective was achieved. The software utilized the library hFigures
(avail-able in open source from the GitHub repositoryhttps://github.com/ledancs/
hFigures). The wellness dashboard used the library to present the modelled data of a patient at risk of developing T2D, who went through a wellness coaching program.
Publications II and III detail the usability methods employed to test the system. The methodologies fall under the UP scenario. The results indicate a positive outcome;
users were able to perform the majority of the tasks in a reasonable time. The cog-nitive walkthrough and heuristic evaluation were conducted with usability experts.
Usability questionnaires were also used, and the results indicate a positive outcome.
To study how visualizations affect the decision-making process (Publications I and IV).
The objective was achieved by comparing visualizations based on the correctness of the decisions made by the participants. In Publication I, non-medical users were asked to assess the overall health and wellness of the modelled patient. The visual-ization technique greatly affected the correctness of the decisions. For instance, the area-based visualization was difficult to grasp and as a result the participants were unable to formulate correct assessments to the same degree as with the other visual-izations.
In Publication IV, clinical experts formulated hypotheses and suggested treat-ments based on the data presented to them during the experitreat-ments. The publication details the different outcomes in terms of insight metrics and hypothesis generation.
These outcomes are the result of the participants’ assessments based on either the baseline representation or the timeline EHR visualization system.
To apply a methodology that studies visual reasoning and the decision making process to assess visualization software for clinical data with the participation of domain experts (Publication IV).
The objective was achieved by recruiting five professional psychiatrists, who took part in the analysis of real patient data. The study followed insight-based method-ology, and to date, is the only instance of this methodology applied to study the visualization of real clinical data with domain experts. The publication is as a use case of insight-based assessment which studied the impact of the visualization on the clinical decision making process.