Page - 167 - in Applied Interdisciplinary Theory in Health Informatics - Knowledge Base for Practitioners

3.3. Quantitative process evaluation of an A&F intervention in cardiac rehabilitation The third study illustrates how CT can be used to enrich quantitative process evaluations of A&F interventions. In this case CT was instrumental to understand the outcomes observed in a cluster randomised trial of e-A&F to improve cardiac rehabilitation [26]. The intervention involved local quality improvement teams receiving quarterly web-based feedback in combination with outreach visits. Feedback was given on 18 quality indicators and included benchmark comparisons. During the visits teams reviewed their feedback and selected indicators they wished to improve upon into their action plan–within the same web-based system. For each indicator that teams targeted for improvement, they were prompted to describe the problem, goal, and concrete actions on how to achieve the goal. During each outreach visit (corresponding to feedback cycles), teams reviewed the new performance scores and updated their action plan accordingly. The associated trial did not show any significant changes in either care processes or patient outcomes [27]. Following CT, the intervention’s ineffectiveness must have been either the result of the possibility that feedback indicating sub- benchmark performance still failed to convince recipients to change, actions were not completed, or completed actions were ineffective. We designed a two-part study to investigate the first gap. Part 1 was a laboratory experiment involving 41 individual cardiac rehabilitation professionals who were given two feedback reports in an adjusted version of the web-based feedback system. These professionals were asked to select indicators for improvement, based on the feedback. If their response was at odds with CT’s hypothesis (i.e. indicator’s performance score was below the benchmark but not selected for improvement; or the score was above benchmark and still selected for improvement), they were asked to explain their choice. Part 2 was a field study concerning a secondary analysis of the trial data, in which multidisciplinary teams selected indicators for improvement across multiple cycles of feedback. Regression techniques were applied to assess determinants of cardiac professionals’ intentions to improve practice. The principal findings were that performance scores and benchmark comparisons influenced intentions, but between one third and half of the time intentions were at odds with CT because professionals either disagreed with benchmarks; deemed improvement unfeasible; or did not consider the indicator an essential aspect of care quality. In addition, it revealed that intentions remained similar in subsequent feedback cycles (because actions were not completed) and that professionals prioritised improving data quality rather than care quality. This study contributed to the understanding of A&F in both the current intervention and in general, in the sense that it quantified how often the feedback loop stagnates and provided insight into the determinants and reasons for not following feedback. 3.4. Understanding the influence of A&F in pain management in intensive care In the previous example CT was used as a conceptual framework in a post-hoc analysis of decision processes. The final illustration also concerns work of our own research team and builds on the findings of the previous study, but in this case CT was used in the very design of the experiment. This study involved an e-A&F dashboard providing intensive care teams with periodic feedback on four pain management indicators [28,29]. Inspired by the cardiac rehabilitation study we recognised that healthcare professionals often already have beliefs about their clinical performance and feedback may fail to change those beliefs. We studied the extent to which those beliefs W.T.GudeandN.Peek /ControlTheory toDesignandEvaluateAuditandFeedback Interventions 167