Page - 44 - in Applied Interdisciplinary Theory in Health Informatics - Knowledge Base for Practitioners

because there were too many of them, or the alerts had low sensitivity or specificity). Was it instead that the cost of changing a clinical process (care process altered) was too high, perhaps because clinical staff were resource constrained, and had little capacity to make the changes needed? Table 2 provides an example of the calculations that can be made for expected utility, based on measurements of the probability and utility of each step in a value chain. It demonstrates that in this particular scenario, the problem lies in the implementability of decisions to improve practice. There is clear benefit in what the Feedback and Audit tool tells clinicians, and it is also clear that there is benefit in undertaking the recommended changes. There however is no ability to translate this feedback into effective real world actions. The main problem in this example is not with the technology, or the information it generates, but with the socio-technical context in which it is used. Consequently creating a better tool would still not change the outcome. Instead, more resources and leadership might be needed to action the information generated by the analytics tool. Table 2. Worked example of a value chain analysis for a computerized Audit and Feedback report. Probabilities are obtained by measuring real world event frequencies, Local utilities are obtained by measuring clinician value assessments at each step in the value chain, using a standardized measurement instrument. The expected utility for any path fragment is calculated from the utility of the node at the end of the path and the probabilities of every node in the path. Step 1: Interaction Step 2: Information received Step 3: Decision changed Step 4: Care process altered Step 5: Outcome changed Event probability 1.0 (1000/1000) 0.61 (614/1000) 0.62 (379/614) 0.08 (31/379) 0 (0/31) Utility 0.8 0.9 0.9 0.92 0.95 Local expected utility 0.8 (0.8 x 1.0) 0.55 (0.9 x 0.61) 0.56 (0.9 x 0.62) 0.074 (0.92 x 0.08) 0 (0 x 0.95) Path expected utility 0.8 (0.8 x 1.0) 0.55 (0.9 x 1.0 x 0.61) 0.34 (0.9 x 1.0 x 0.61 x 0.62) 0.028 (0.92 x 1.0 x 0.61 x 0.62 x 0.08) 0 (0.95 x 1.0 x 0.61 x 0.62 x 0.08 x 0) Analysis Utility of a accessing 1000 indicators is high because the there is a high expectation they will contain actionable information. Report length may reduce utility. Utility of receiving specific information from a report about abnormal indicators is high, but expected utility is lower as probability that any indicator is abnormal is moderate. Utility of decision to deal with an abnormal indicator is high, given likely benefit. Expected utility is lower as only some indicators are chosen. A collapse in expected utility at this stage occurs because most decisions in Step 3 do not translate into process changes in Step 4. The potentially high utility of process changes is entirely negated by the very limited process changes arising from Step 4. E.Coiera /AssessingTechnologySuccessandFailureUsing InformationValueChainTheory44