Page - 23 - in Applied Interdisciplinary Theory in Health Informatics - Knowledge Base for Practitioners

Information Theory and Medical Decision Paul KRAUSEa,1 a Department of Computer Science, University of Surrey, United Kingdom Abstract. Information theory has gained application in a wide range of disciplines, including statistical inference, natural language processing, cryptography and molecular biology. However, its usage is less pronounced in medical science. In this chapter, we illustrate a number of approaches that have been taken to applying concepts from information theory to enhance medical decision making. We start with an introduction to information theory itself, and the foundational concepts of information content and entropy. We then illustrate how relative entropy can be used to identify the most informative test at a particular stage in a diagnosis. In the case of a binary outcome from a test, Shannon entropy can be used to identify the range of values of test results over which that test provides useful information about the patient’s state. This, of course, is not the only method that is available, but it can provide an easily interpretable visualization. The chapter then moves on to introduce the more advanced concepts of conditional entropy and mutual information and shows how these can be used to prioritise and identify redundancies in clinical tests. Finally, we discuss the experience gained so far and conclude that there is value in providing an informed foundation for the broad application of information theory to medical decision making. Keywords. Shannon entropy; Relative entropy; Conditional entropy; Mutual information; Medical diagnosis Learning objectives After reading this chapter, the reader will be able to: 1. Understand the basic concepts of information theory: information content; Shannon entropy; relative entropy. 2. Understand how these concepts can be applied to medical decision making at a general level. 3. Understand how the more advanced concepts, conditional entropy and mutual information, could provide deeper insights into the potential redundancies in laboratory tests. 1. Introduction to Information Theory Information theory has gained application in a wide range of disciplines, including statistical inference, natural language processing, cryptography and molecular biology. It covers the study of the transmission, processing, extraction, and utilization of information at a foundational, mathematical level. A fundamental goal of information 1 Corresponding Author: Paul Krause; E-mail: p.krause@surrey.ac.uk Making Applied Interdisciplinary Theory in Health Informatics P. Scott et al. (Eds.) © 2019 The authors and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI190108 23