Comparison of Expert Opinion, Majority Rule, and a Clinical Prediction Rule to Estimate Distal Radius Malalignment

Kokoteksti

(1)UEF//eRepository DSpace Rinnakkaistallenteet. https://erepo.uef.fi Terveystieteiden tiedekunta. 2017. Comparison of Expert Opinion, Majority Rule, and a Clinical Prediction Rule to Estimate Distal Radius Malalignment Luokkala T Ovid Technologies (Wolters Kluwer Health) info:eu-repo/semantics/article info:eu-repo/semantics/acceptedVersion © Wolters Kluwer Health, Inc All rights reserved http://dx.doi.org/10.1097/BOT.0000000000001022 https://erepo.uef.fi/handle/123456789/5113 Downloaded from University of Eastern Finland's eRepository.

(2) Journal of Orthopaedic Trauma Publish Ahead of Print DOI: 10.1097/BOT.0000000000001022. Title page A) Title Comparison of Expert Opinion, Majority Rule, and a Clinical Prediction Rule. B) Authors. EP TE. 1 ) Toni Luokkala 1 , M.D.. D. to Estimate Distal Radius Malalignment. 2) Tapio Flinkkilä 2, M.D., Ph.D.. 3) Juha Paloneva 1,3 , M.D., Ph. D.. 4) Teemu V Karjalainen 1, M.D., Ph.D. Affiliations. 1) Central Finland Central Hospital, Department of Surgery, address: Keskussairaalantie 19, 40620, Jyväskylä, Finland. C. 2) Oulu University Hospital and University of Oulu, Department of. Orthopaedics and Traumatology, address: Kajaanintie 50, 90220, Oulu,. C. Finland. 3) University of Eastern Finland. A. C) Corresponding author Toni Luokkala, M.D., address: Keski-Suomen Keskussairaala, Keskussairaalantie 19, 40620 Jyväskylä, Finland tel: +358 14 269 5058, email: toni.luokkala@ksshp.fi D) Acknowledgements. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(3) The authors would like to thank Mr Tuomas Selander for statistical assistance. E) ) Conflicts of interests All of the authors hereby declare that they have no conflicts of interest to disclose. F) Source of Funding. G) Ethical approval. EP TE. This study was conducted after Institutional board approval.. D. None.. Presented in part at the 26th Meeting of Scandinavian Hand Society, Levi, Finland, 2nd April 2016.. Abstract. Objectives: To investigate the ability of individual surgeons (expert opinion [EO]) to. C. predict distal radius fracture healing above a threshold malalignment compared with the majority prediction of the group of surgeons (“majority rule,” [MR]) and a. C. statistically derived clinical prediction formula (Edinburgh wrist calculator [EWC]). Design: Comparative diagnostic study from prospectively collected data of consecutive. A. patients.. Setting: Two academic level 1 and one academic level 2 trauma centers. Patients/Participants: Eighteen surgeons assessed probability of healing above a threshold malalignment (often referred to as fracture “instability”) for 71 fractures based on radiographs taken initially and after closed reduction and cast application. The probability of losing alignment according to the EWC was dichotomized (likely to lose alignment ≥0.5 vs. unlikely <0.5).. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(4) Main Outcome Measures: Accuracy parameters of prediction of EO, MR and EWC Results: EWC and MR demonstrated higher accuracy (0.77 and 0.75, respectively) and sensitivity (0.95 and 0.79, respectively) compared to expert opinion (accuracy 0.66, sensitivity 0.58) for predicting healing above threshold malalignment. Reliability was higher for MR (kappa 0.88) than for EWC (kappa 0.63) or EO (kappa coefficient 0.44).. D. The negative predictive value of the EWC for healing above a threshold of malalignment was excellent (0.97).. EP TE. Conclusions: Surgeon opinion is not reliable or accurate for predicting loss of alignment of a distal radius fracture above a threshold malalignment after closed reduction and immobilization. Dichotomized EWC may be a useful tool in predicting loss of alignment (instability) of a distal radius fracture.. Level of evidence: Diagnostic Level II. A. C. C. Keywords: distal radius fracture; instability; malalignment; prediction rule; expert opinion, probability. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(5) 1 2 Introduction. 4. Distal radius fractures (DRF) can lose alignment after closed reduction and cast or splint. 5. immobilization. Fractures that heal above a certain threshold malalignment are commonly. 6. referred to as “unstable”. 1, 2, 3, 4 The ability to estimate the probability of healing with a. 7. specific amount of a malalignment would help patients decide between operative and. 8. nonoperative treatment. Several factors are associated with an increased likelihood of loss of. 9. alignment including age, dorsal angulation, metaphyseal comminution, and ulnar variance of. EP TE. D. 3. 10. the fracture. 4,5 Experts may interpret these risk factors variably, resulting in inconsistent. 11. estimates of healing above a threshold malalignment. The complexity of accurate prediction is. 12. observed in other fields. 6, 7. 13. The Edinburgh wrist calculator (EWC; http://www.trauma.co.uk/wristcalc), a clinical. 15. prediction formula based on statistical evaluation of 4000 fractures, can aid decision-making.. 16. A recent study validated the EWC in a different data set.8 In other fields, prediction rules. 17. have consistently demonstrated superior performance compared to expert opinion (EO).9. Majority rule (MR, also known as Condorcet’s jury theorem) was first described by Marquis. A. 19. C. 18. C. 14. 20. de Condorcet in 1785.10, 11 According to majority rule theory, in dichotomized problems with. 21. a correct option, the “wisdom” of the reviewers aggregates when the number of reviewers. 22. increases. This aggregation will only occur if the independent reviewers’ judgments are. 23. mostly correct (probability > 0.5 for the single judgment).. 24. 1 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(6) 25. The purpose of this study was to compare the accuracy and reliability of EWC, EO, and MR. 26. in predicting healing above a threshold malalignment (“instability”) of a distal radius fracture.. 27 Methods. 29. The research material was prospectively collected as a part of an epidemiological study. 30. comprising all distal radius fractures between January 2008 and December 2008 in the. 31. catchment area of Oulu University Hospital.12 After institutional review board approval, all. 32. these consecutive DRF radiographs were reviewed from a centralized radiological database.. 33. We included displaced and nondisplaced fractures that were both clinically and radiologically. 34. diagnosed acute (<7 days old), isolated distal radius fracture in an adult patient ≥18 years old.. 35. Primary standard anteroposterior and lateral radiographs before and immediately after closed. 36. manipulation and cast immobilization, as well as at 6-week follow-up visit at outpatient clinic. 37. needed to be available. All displaced fractures underwent closed reduction and cast. 38. application (non-displaced fractures were not manipulated). Fracture had to meet our. 39. standards for acceptable reduction after closed manipulation and/or application of cast: ≤0°. 40. dorsal angulation, ≤3mm ulnar variance, and ≥15° radioulnar inclination. Two independent. 41. investigators evaluated the radiographs and if they did not reach mutual consensus, the. 42. fracture was excluded. We excluded fractures that did not meet the criteria of acceptable. 43. final position after initial closed reduction or settled into a non-acceptable final position. A. C. C. EP TE. D. 28. 44. within 2 weeks. The threshold for final acceptable alignment on radiographs was ≤10° dorsal. 45. tilt, ≤2mm ulnar variance, ≥15° inclination, and ≤1 mm articular step or gap. Other exclusion. 46. criteria were: patients with missing data (i.e. if radiographs were not available after closed. 47. reduction and/or application of cast or at 6 weeks follow-up), and the presence of concomitant. 48. fractures (Figure 1, Table 1).. 49. 2 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(7) Nine orthopaedic surgeons (upper limb specialists and orthopaedic trauma surgeons) and 9. 51. hand surgeons who all regularly treated DRF at two tertiary and one secondary referral. 52. centers in Finland were asked their opinion if fracture in question would heal above or below. 53. the malalignment threshold (Table 2). Patient age, sex, and mechanism of trauma were. 54. provided with the radiographs. The participants evaluated the radiographs taken initially and. 55. after the closed reduction and cast application. Surgeons were blinded to the final radiological. 56. outcome. After 6 months, the same surgeons assessed the same fractures in an alternative. 57. order.. EP TE. D. 50. 58. MR was defined as the choice of more than 50% of the surgeons. Two surgeons (TL and TK). 60. assessed the probability of loss of alignment based on the EWC,13 which takes into account. 61. patient age, initial dorsal angulation, metaphyseal comminution, and ulnar variance of the. 62. fracture, as well as independence of the patient. The EWC yields a probability of healing with. 63. malalignment greater than the threshold between 0 and 1. The result of EWC was. 64. dichotomized to not likely to lose alignment if the EWC probability was <0.5 and likely to. 65. lose alignment if it was ≥0.5. The study that presented the EWC defined the ulnar variance by. 66. comparing it to the contralateral side, but we assumed a neutral variance.. 67. The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive. 68. value (NPV) of diagnosis of healing with greater than threshold malalignment were calculated. A. C. C. 59. 69. for each method for each fracture. The 95% confidence interval (95%CI) was calculated for. 70. each variable with Wilson’s method when applicable.. 71. The Kappa coefficient was used to calculate intrarater reliability for independent reviewers. 72. and MR and interrater reliability for independent reviewers and EWC. Fleiss’s kappa for. 73. multiple raters was used to calculate the interrater reliability for independent reviewers.. 74. Reliability of the estimates of independent reviewers compared to the actual instability status. 75. (fracture status at final follow up) of the DRF was calculated using kappa coefficient. Kappa 3 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(8) 76. coefficient results were grouped as follows: 0.00–00.20, negligible; 0.21–0.40, poor; 0.41–. 77. 0.60, moderate; 0.61–0.80, good; and 0.80–1.00, excellent. Orthopedic and hand surgeons. 78. were compared using independent-samples Mann–Whitney U-test. Values are presented as. 79. mean with 95%CI when applicable. We set the p value to <0.05 to indicate statistically. 80. significant results.. D. 81 Results. 83. A total of 229 distal radius fractures were identified and 158 fractures were excluded: 29 did. 84. not meet the study criteria of acceptable final position after closed reduction; 55 had. 85. alignment greater than acceptable limits during the first two weeks after reduction and were. 86. offered surgery; 48 had missing data; 8 had previous or other ipsilateral fractures; and for 18. 87. fractures there was reviewer disagreement about acceptable alignment after reduction. The. 88. remaining 71 fractures (22 with initially acceptable alignment that were not reduced and 49. 89. that were reduced) were treated with cast immobilization and included in the study (Figure 1).. 90. Of these 71 fractures, 34 (48%) healed with greater than threshold malalignment (they were. 91. “unstable”) and 37 (52%) did not (Figure 1, Table 1).. C. EP TE. 82. C. 92. The mean kappa coefficient for intrarater reliability was 0.58 (moderate) (95%CI 0.29 to. 94. 0.77) for EO and 0.88 (excellent) (95%CI 0.82 to 0.94) for MR. Fleiss’s kappa coefficient for. A. 93. 95. interrater reliability was 0.44 (moderate) for EO, and the standard kappa coefficient for. 96. interrater reliability of was 0.63 (good) (95%CI 0.53 to 0.94) for EWC. The experts agreed. 97. unanimously with each other eight times (11% of fractures) in the first round; of these. 98. unanimous agreements, six fractures were correctly judged. In the second round, the experts. 99. agreed ten times (14%), of which all were correctly judged. The mean kappa coefficient for. 4 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(9) reliability of EO compared to reality was 0.32 (poor) (95%CI 0.17 to 0.49). Reliability of the. 101. four experts was negligible (Table 3).. 102. The accuracy of prediction of healing above a threshold alignment (fracture “instability”) was. 103. limited for all techniques, but best for the probability calculator and MR was better than. 104. average expert opinion. All methods performed well with initially nondisplaced fractures, of. 105. which only 2 healed with greater than threshold malalignment (Table 4). There were no. 106. significant differences in accuracy, sensitivity, or specificity between orthopaedic and hand. 107. surgeons or between the first and second round.. EP TE. D. 100. 108 Discussion. 110. The purpose of this study was to compare the accuracy and reliability parameters of EWC,. 111. EO, and MR and to assess which method might be the most suitable for clinical use. None of. 112. the methods could accurately detect fractures more likely than not to heal with alignment. 113. above threshold. Expert opinion was the least reliable and accurate method to predict healing. 114. of greater than threshold malalignment. Therefore, EO of this prediction may not be useful for. 115. informing the patient and assisting with decision making between non-operative and operative. 116. treatment. The EWC was the most reliable technique. However, the PPV of the dichotomized. 117. EWC may not be high enough (0.56 and 0.59) for most patients to justify surgery as long as. 118. the alignment is acceptable in a cast. The high NPV give patients substantial confidence in. A. C. C. 109. 119. choosing nonoperative treatment when they have a lower than 50% probability of healing. 120. with greater than threshold malalignment using the EWC.. 121 122. Several things should be kept in mind when interpreting this study. Radiological. 123. measurements are imprecise, and thresholds for malalignment are arbitrary and variable. 124. between studies1,2. We used the same radiographic thresholds for acceptable initial reduction. 5 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(10) and acceptable final alignment as used in original development of the EWC. The accuracy of. 126. majority rule might have increased with more surgeon raters. With a mean accuracy of 59%,. 127. we might reach 90 percent accuracy with 80 expert opinions. But this is currently impractical. 128. for everyday fracture care. Our analysis dichotomized adequate final alignment, but the. 129. numerical estimates of the probability of healing above a threshold malalignment might be. 130. useful to patients. This study addresses radiological outcome rather than functional outcome.. 131. The radiological malalignment does not necessarily cause poor functional outcome, especially. 132. less active or more infirm people.14,15,16 Only two surgeons evaluated the data using the EWC,. 133. so the estimates of reliability might not be generalizable. Use of the EWC for all of the. 134. surgeons was not possible, as that might have affected the EO judgments. Therefore, another. 135. study with different fractures is needed to measure the interrater reliability of EWC., We. 136. excluded significant number of fractures that showed any early loss of threshold alignment. 137. within 2 weeks. Therefore, the results only apply to distal radius fractures with later loss of. 138. threshold alignment and to fractures that were within the threshold for final alignment after. 139. injury, or were reducible to the strict criteria for acceptable reduction. Patients and surgeons. 140. that accept less strict thresholds for post-reduction alignment may observe different results.. C. EP TE. D. 125. C. 141. The ability of experts to predict fracture behavior based on initial radiographs is also poor. 143. among scaphoid fractures.17 The arbitrariness of experts’ intuitive weighing of the factors can. A. 142. 144. be explained by several different mechanisms. First, routine clinical prediction is not based. 145. solely on the use of published predictive factors; rather, it is mixed with personal hopes and. 146. heuristics (mental short cuts) and shaped by one’s experience and opinions. Second,. 147. individuals tend to vary their behavior according to prior successes or failures.7 But we may. 148. not realize that after we deem the fracture “unstable” (likely to heal above threshold. 149. malalignment) and proceed to surgery, we will lose the opportunity to learn if our prediction. 6 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(11) was incorrect. This results in a self-fulfilling prophecy that feeds self-confidence. This is. 151. particularly problematic for young patients who are statistically less likely to lose alignment,. 152. but may be less satisfied with delayed loss of threshold malalignment leading to later. 153. surgery.18, 19 Facing potential dissatisfaction surgeons may be more predisposed to. 154. recommend surgery. On the other hand, because older patients are most likely to have. 155. fractures that displace but are more accepting of malalignment, we may be more comfortable. 156. with a choice of nonoperative treatment and we may monitor the alignment less frequently.. 157. This results in another kind of confirmation bias, since we may not learn from our incorrect. 158. judgments when older patients with fractures that are deemed unlikely to lose alignment. 159. might not be monitored routinely.. EP TE. D. 150. 160. This study establishes the ability of a probability calculator based on data from a large. 162. number of fractures to provide better estimates of healing with greater than threshold. 163. malalignment (“fracture instability”) than expert opinion. Given the good negative predictive. 164. value of the EWC patients with a probability of losing alignment less than 0.5 can feel. 165. comfortable with a choice for nonoperative treatment. This supports a potential strategy of. 166. giving patients the option of skipping a return visit and repeat radiograph when the fracture is. 167. well-aligned or well-reduced and risk of loss of alignment is low.. C. A. 168. C. 161. 169 170 171 172 173 174. 7 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(12) References. 176 177 178. 1.. Walenkamp MMJ, Aydin S, Mulders MA, et al. Predictors of unstable distal radius fractures: a systematic review and meta-analysis. J Hand Surg Eur Vol. 2016 Jun;41(5):501-15. 179 180. 2.. Walenkamp MMJ, Vos LM, Strackee SD, et al. The Unstable Distal Radius FractureHow Do We Define It? A Systematic Review. Jnl Wrist Surg. 2015 Nov;4(4):307–316.. 181 182 183. 2.. Leone J, Bhandari M, Adili A, et al. Predictors of early and late instability following conservative treatment of extra-articular distal radius fractures. Arch Orthop Trauma Surg. 2004 Jan;124(1):38–41.. 184 185 186. 3.. Wadsten MÅ, Sayed-Noor AS, Englund E, et al. Cortical comminution in distal radial fractures can predict the radiological outcome: a cohort multicentre study. The Bone & Joint Journal. 2014 Jul 1;96-B(7):978–983.. 187 188. 4.. Altissimi M, Mancini GB, Azzará A, et al. Early and late displacement of fractures of the distal radius. International Orthopaedics (SICO). 1994 Jan 1;18:61–65.. 189 190. 5.. Lafontaine M, Hardy D, Delince P. Stability assessment of distal radius fractures. Injury. 1989 Jul;20(4):208–210.. 191 192. 6.. Kahneman D, Tversky A. Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Economic Society. 1979 Jan 1;2(47):263–291.. 193 194. 7.. Meehl P. Clinical versus Statistical Prediction - A Theoretical Analysis and a Review of the Evidence. MInnesota: Echo Point Books and Media, LLC; 2013.. 195 196. 8.. LaMartina J, Jawa A, Stucken C, et al. Predicting alignment after closed reduction and casting of distal radius fractures. YJHSU. 2015 May;40(5):934–939.. 197 198. 9.. Grove WM, Zald DH, Lebow BS, et al. Clinical versus mechanical prediction: a metaanalysis. Psychol Assess. 2000 Mar;12(1):19–30.. 199 200. 10.. 201 202. 11.. 203 204. 12.. Flinkkila T, Sirnio K, Hippi M, et al. Epidemiology and seasonal variation of distal radius fractures in Oulu, Finland. Osteoporos Int. 2010 Oct 23;22(8):2307–2312.. 206 207. 13.. Mackenney PJ, McQueen MM, Elton R. Prediction of instability in distal radial fractures. J Bone Joint Surg Am. 2006 Sep;88(9):1944–1951.. 208 209 210. 14.. Arora R, Lutz M, Deml C, et al. A prospective randomized trial comparing nonoperative treatment with volar locking plate fixation for displaced and unstable distal radial fractures in patients sixty-five years of age and older. J Bone Joint Surg. C. C. EP TE. D. 175. Condorcet. Essay on the Application of Analysis to the Probability of Majority Decisions. Paris: l'Imprimerie Royale. 1785 Jan 1;(1).. A. Austen-Smith D, Banks JS. Information Aggregation, Rationality, and the Condorcet Jury Theorem. The American Political Science Review. 1996 Mar 1;90(1):34–45.. 205. 8 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(13) 211. Am. 2011 Dec;93(23):2146–2153. 15.. Grewal R, MacDermid JC. The risk of adverse outcomes in extra-articular distal radius fractures is increased with malalignment in patients of all ages but mitigated in older patients. YJHSU. 2007 Sep;32(7):962–970.. 215 216 217. 16.. McQueen MM, Hajducka C, Court-Brown CM. Redisplaced unstable fractures of the distal radius: a prospective randomised comparison of four methods of treatment. J Bone Joint Surg Br. 1996 May;78(3):404–409.. 218 219 220. 17.. Desai VV, Davis TR, Barton NJ. The prognostic value and reproducibility of the radiological features of the fractured scaphoid. J Hand Surg Eur Vol. 1999 Oct;24(5):586–590.. 221 222 223. 18.. Rodríguez-Merchán EC. Plaster cast versus percutaneous pin fixation for comminuted fractures of the distal radius in patients between 46 and 65 years of age. J Orthop Trauma. 1996 Dec 1;11(3):212–217.. 224 225 226. 19.. Merchan EC, Breton AF, Galindo E, et al. Plaster cast versus Clyburn external fixation for fractures of the distal radius in patients under 45 years of age. Orthop Rev. 1991 Dec 1;21(10):1203–1209.. 227. EP TE. D. 212 213 214. Figure Legends. 229. Figure 1. Flowchart of the fracture selection.. A. C. C. 228. 9 Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(14) Table 1. Fracture characteristics. 71. Age (mean). 18-86 (54.9). Male. 18-76 (44.3). Female. 18-86 (58.2). Male (%). 17/71 (23.9%). Female (%). 54/71 (76,1%). Stable (%):. 37/71 (52.1%). Unstable (%). 34/71 (47.9%). Initial displacement grade. 22. C. Non-displaced Displaced. EP TE. Sex. D. Number of fractures. 49. A1. A. A2. C. AO/OTA classification:. 17 11. A3. 10. B1. 3. B2. 0. B3. 0. C1. 12. C2. 15. C3. 3. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(15) Table 2. Surgeon demographics. Number of surgeons. 18. Orthopaedic surgeons. 9. Hand surgeons. 9 33-49 (41.4). Experience as a specialist, years (mean) Frequency of consulting on treatment. 1-15 (6.8) 0-10 (5). EP TE. on distal radius fractures/week, range. D. Age (mean). (mean). Does consult on treating posttraumatic. 14/18. A. C. C. conditions (malunion, ligament problems). Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(16) Table 3. Reliability of the estimates of independent reviewers compared to the actual instability status of the distal radius fracture Kappa - coefficient. Hand surgeon 1. 0.49. Hand surgeon 2. 0.41. Hand surgeon 3. 0.44. Hand surgeon 4. 0.16. Hand surgeon 5. 0.22. Hand surgeon 6. 0.30. Hand surgeon 7. 0.40. Hand surgeon 8. 0.40. Hand surgeon 9. 0.34. Orthopaedic surgeon 1. 0.37. Orthopaedic surgeon 2. 0.31. Orthopaedic surgeon 3. 0.49. Orthopaedic surgeon 4. 0.40. Orthopaedic surgeon 5. -0.01. Orthopaedic surgeon 6. 0.10. Orthopaedic surgeon 7. 0.13. Orthopaedic surgeon 8. 0.24. Orthopaedic surgeon 9. 0.49. A. C. C. EP TE. D. Surgeon. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(17) Table 4. Accuracy parameters summarized. Results present the ability of expert opinion, majority rule and Edinburgh wrist calculator to predict the radiological instability of distal radius fracture. Results are presented as mean with 95% confidence interval (95% CI) when applicable.. Majority Rule. (95% CI). Edinburgh wristcalc,. Edinburgh wristcalc,. Surgeon 1. Surgeon 2. ALL 0.66 (0.63 to 0.70). Sensitivity. 0.58 (0.52 to 0.65). Specificity. 0.75 (0.70 to 0.80). PPV. 0.68 (0.64 to 0.72). NPV. 0.65 (0.62 to 0.69). NONDISPLACED 0.82 (0.76 to 0.87). Sensitivity. 0.17 (0.05 to 0.28). Spesificity. 0.88 (0.82 to 0.94). PPV. 0.76. 0.95. 0.87. 0.73. 0.71. 0.71. 0.67. 0.56. 0.59. 0.83. 0.97. 0.92. 0.91. 0.91. 0.86. n/a. n/a. 0.00. 0.91. 0.91. 0.90. n/a. 0.00. 0.00. 0.00. 0.92 (0.91 to 0.94). 1.00. 1.00. 0.95. 0.68. 0.71. 0.71. C. NPV. 0.77. 0.79. C. Accuracy. 0.75. EP TE. Accuracy. D. Expert Opinion. DISPLACED. 0.59 (0.55 to 0.63). A. Accuracy Sensitivity. 0.59 (0.53 to 0.66). 0.79. 0.95. 0.91. Spesificity. 0.55 (0.49 to 0.61). 0.53. 0.55. 0.56. PPV. 0.72 (0.69 to 0.74). 0.71. 0.59. 0.63. NPV. 0.44 (0.40 to 0.48). 0.63. 0.94. 0.88. Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(18) D EP TE C C A Copyright Ó 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited..

(19)