Page - 38 - in Applied Interdisciplinary Theory in Health Informatics - Knowledge Base for Practitioners

Table 1. Examples of measures that can be used to evaluate systems at different stages of the interaction value chain for information retrieval systems which search for documents, and telehealth systems which support the communication of patient information (n = number of). Decision theory provides us with a powerful and theoretically robust way of estimating the value we place on receiving new information. For example, if a new diagnostic test result changes a patient’s treatment and saves their life, then instinctively the value of that information is high. If a diagnostic test allows a patient to avoid a risky treatment and to follow a less risky but equally beneficial option, then the information’s value is based on those avoided risks. If a new diagnostic test result only confirms what is already most likely, and it triggers no change to treatment, then it might have a relatively low value. This Value Of Information (VOI) can be defined as the value we place on receiving particular data prior to making a decision [6]. We could calculate such a value in financial terms such as money saved or earned, or as patient expressed preferences. In other words, VOI is the difference between the value of persisting with the present state of affairs and the value to us of being able to embark on a new decision, influenced by new information. VOI is zero whenever obtaining new data does not change decisions or outcomes. VOI also has a decision-theoretic interpretation. Imagine for example that a patient undertakes a test, and will be given different treatments depending on the blood test result. Each of these two treatments will result in a different outcome for the patient. How do we determine the value of each outcome to the patient? A preference for one outcome over another can be represented with a quantitative value called a utility. A utility is a number between zero and one and the outcome with the highest utility is the preferred one. A utility value is thus a model of an individual’s preference for an outcome, expressed in numerical form, and can be derived by a number of different means. Common methods to estimate utilities include rating scales, standard gambles and estimating quality-adjusted life expectancy e.g. using a time trade-off [7] [8]. Next we need to consider that each of the two potential treatment outcomes is uncertain. A given treatment will not always have the same effect on different patients. So even if one outcome might have higher utility for a patient, we need also to consider how likely that utility will ever be realized. To do that we now calculate the expected utility e of making one choice over another, which is simply the product of its probability p and its utility u: Interaction Information Decision Care process Outcome Information retrieval system n queries made, n query reformulations n documents retrieved, precision and recall, document relevance n correct or incorrect decisions, decision velocity n and type of tests ordered, medications prescribed, cost of care Morbidity and mortality, Quality Adjusted Life Year (QALY) Telehealth system n conversations, call quality and time, user satisfaction Quality and quantity of patient level data shared n additional correct or incorrect decisions Health service utilization rates, travel costs Blood pressure, HbA1c, blood glucose etc., Morbidity and mortality, QALY E.Coiera /AssessingTechnologySuccessandFailureUsing InformationValueChainTheory38