Clinical decision support systems

The TBIcare decision support tool – aid for the clinician

Jyrki Lötjönen & Jussi Mattila,

VTT Technical Research Centre of Finland

1st Turku Traumatic Brain Injury Symposium Turku, Finland, 17–18 January 2014

Validation of the decision support tool

Ari Katila

University of Turku

Outline

This talk has been divided into the following sections:

• Introduction to decision support

• TBIcare decision support tool

• Validation

• Summary

Clinical decision support systems

• Clinical decision support systems (CDSS) help clinicians or other health professionals with their decision making tasks.

• CDSS techniques are typically

– knowledge-driven based on IF-THEN rules, or – data-driven based on artificial intelligence.

• There is a clear need for CDSS especially in complex diseases where the rules to diagnose are easily highly complex or fuzzy and subjective.

Evidence-based medicine

• Clinical decision making should follow the principles of evidence- based medicine defined as

– the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.

• The idea of data-driven medicine is to reveal this evidence from existing databases:

– continuously increasing amounts of data are acquired from patients, containing a lot of undefined and hidden information that could be better exploited in decision making, and

– enables personalised medicine by finding best evidence for each patient separately.

Why are so few CDSS used in reality?

Possible reasons include:

• Accuracy & robustness are not high enough:

– human body is a highly complex system and not enough data exists taking into account all the variability (personalised healthcare).

• Tools do not fit to clinical work-flows and meet clinical realities:

– difficult to use & time consuming – incomplete and imperfect data

• Tools using modern mathematical modeling are easily black-boxes to users.

• Data quantification is often challenging.

• Digital data are not easily available.

• Tools do not provide a holistic view by integrating heterogeneous data.

Requirements of CDSS in TBIcare

• A formal procedure was used to define application requirements.

• Experts were interviewed in Turku, Cambridge, and Tampere.

• Initial requirements

– current clinical protocols – general requirements

• Updated requirements and user scenarios (mockups & prototypes)

– process still continues

Clinical questions to be answered

1. Prediction of poor functional outcome at admission:

a) For mild TBI – persistent post-concussional symptoms at 3 months.

b) For moderate and severe TBI – unfavourable Glasgow outcome score (severe disability or worse) at 6 months.

2. Whether patients who are admitted to an intensive care environment are likely to have intracranial hypertension which is either

prolonged (> 7 days of ICP monitoring and therapy) or refractory (requiring third-tier therapies – induced hypothermia, sedation for metabolic suppression, and/or decompressive craniectomy).

3. Whether patients with TBI who are admitted to a hospital require monitoring in an intensive care environment.

4. Whether individual patients are more likely to respond to specific treatments (of ICP lowering methods, listed above).

Overview of the TBIcare tool

• Data viewer for patient and injury data

– Basic information, demographics, injury conditions

– Longitudinal data of events, measurements and treatments – Imaging data

• Data analysis and machine learning features – Fully automated image processing methods – Predictive models

• Outcome prediction from acute data

• Prolonged ICP monitoring prediction

• Additional supporting features – News feeds

– Learning resources

System overview

• Client-server system

• High performance server – Unified database design

• Data translated and transformed to a common framework

– Computationally demanding operations

• Data processing

• Imaging algorithms

• Predictive models

• Browser client

– Modern web technologies

• HTML5

• CSS3

• Heavy reliance on JavaScript

Injury overview

Graphs for longitudinal data

CT (and MRI) imaging

Koikkalainen: Radiology, 2008 & Mattila: JAD, 2011+2012 & One US patent + Patents pending

Decision support system developed builds a disease specific profile (list of

biomarkers with relevance) and shows the fit of patient to this disease (fitness).

Disease state index and disease state fingerprint

Predictive models

• Outcome prediction from acute injury data – Predicts good/poor outcome three to six

months after injury

• Need of prolonged ICP monitoring

– Predict whether the patient will need one week or longer ICP monitoring

• Disease State Index (DSI) and Disease State Fingerprint (DSF)

– Combine all heterogeneous data – Visualize in an interactive control

Additional supporting features

• Access to news feeds related to TBI

• TBI knowledge content

– Generic information about TBI – Clinical scale descriptions

• GCS, ISS, APACHE II, Marshall, etc.

– Criteria and guidelines

• Wiki

– Allows people to add and modify content in a collaboration with others

– Customization for hospitals/regions – Localization

– User’s guide

Validation

The validation is being realised on two tracks:

1) Off-line validation of computational methods using both retrospective and prospective data.

2) Validation of CDSS simulating the real clinical use:

• Studying differences in clinical decisions without and with using the tool.

Preliminary validation - Methods

• Outcome measure:

– Functional outcome after the TBI was measured with the Glasgow Outcome Scale (GOS).

– GOS was dichotomized as unfavorable outcome (GOS 1-3) and favorable outcome (GOS 4-5).

• Data sets: Data from patients with traumatic brain injury was collected in Turku University hospital and Cambridge University Hospital.

• 10 iterations of stratified 3-fold cross-validation were reported.

Preliminary validation - Results

AUC

All 0.89 0.85

Basic Measures 0.84 0.83

Physiological 0.41 0.60

Laboratory 0.76 0.66

Metabolomics 0.90 -

CT 0.85 0.77

Turku:

Basic Measures: age, gender, GCS, pupil reactivity, loss of consciousness, post traumatic amnesia, ISS, Marshall Grade All: Basic Measures + secondary insults, physiological measures, laboratory results,

Cambridge:

Basic Measures: age, gender, GCS, pupil reactivity, APACHE, ISS, Marshall Grade All: Basic Measures + secondary insults,

physiological measures, laboratory results, CT

Turku: Cambridge:

Preliminary validation - Results

VALIDATION - FIRST PHASE

• Recruited team for validation – 1 emergency physicians – 3 neurosurgeons

– 2 neurointensivists – 1 neurologist

– 1 nurse specialist

• Main focus primarily to determine

– Treatment path to ward / ICU / home – GOS 1 – 5

– ICP monitoring,

• yes / no

• Less than 7 days or more than 7 days

• Surgery required for evacuation of mass lesion

VALIDATION PHASE TWO

• Phase two repeated after two weeks

• Main focus again primarily to determine – Treatment path to ward / ICU / home – GOS 1 – 5

– ICP monitoring,

• yes / no

• Less than 7 days or more than 7 days

– Surgery required for evacuation of mass lesion

• With the aid from

– Image processing results and

– DSI analysis as contributing factors for one’s decisions.

• Interface close to the final software

http://dsf.vtt.fi/TbiCare - ValidationStudy

EXPERIENCES

• Missing data became irritating

– Pupillary size, asymmetry, reaction to light

• DSI – does it count the best value?

– Assessment during emergency care or transportation

• Missing timepoints as well

• CT image segmentation caused minor confusion as bone was segmented as blood

• Laboratory values 0 not evidently reliable

• APACHE and SOFA scores recorded as ER assessments

• Learning curve was rising pretty fast

• 20 minutes from the start, finally appr. 6 min

PHASE 1 (N=121) PHASE 2 (N=67)

Mean SD Median Mean SD Median

Age 3.4 1.2 3 3.4 1.0 3

Pre-Injury Health 2.7 1.2 3 2.4 1.1 2

Other Injuries 2.7 1.3 3 2.6 1.5 3

GCS 4.1 1.2 5 4.0 1.2 4

CT 4.3 0.8 5 4.4 0.7 4

Pupils And Other Neurology 3.2 1.3 3 2.9 1.3 3

Vital Functions 3.2 1.2 3 3.1 1.1 3

Laboratory 2.6 1.2 3 2.5 1.0 3

Delays 2.2 1.3 2 2.5 1.1 2

Treatment Measures 2.7 1.3 3 2.6 1.0 3

Imaging Tools 2.8 1.1 3

DSI Tools 3.0 1.1 3

• 1: Not at all, …, 5: Critically important

• Most influential measures in red

• DSI and Imaging Tools added in Phase 2 in blue

PRELIMINARY STATISTICS

Main challenges for decision support

• Data availability

– Much of the data not stored in electronic formats – Several ICT systems in use

• Data quality

– Manual data entry should be minimized – Input forms should be context sensitive

• Input validation

• Sanity checks – Data heterogeneity

• Data access

– Complex information systems

– Authentication and authorization issues

Summary

• TBIcare tool is among the first attempts to develop a holistic tool for helping clinicians in their decision making in TBIs.

• The challenge in TBIs is huge and the goal will be reached by gradual improvements.

• The preliminary results in TBIcare are promising and we believe that steps into right direction has been taken.

Acknowledgments

The credit from this work goes to the whole TBIcare team!