AI for healthcare requires ‘smart data’ rather than ‘big data’

Norman Fenton gave a talk titled AI for healthcare requires ‘smart data’ rather than ‘big data’ to medics at the Royal London Hospital on 27 November. He explained the background and context for the PAMBAYESIAN project.

Norman's Powerpoint presentation

Book Launch at the Turing Institute

Some photos from last night's book launch event at The Turing Institute

Norman Fenton and Martin Neil

 

More photos

Book Launch event at The Turing Institute

On 14 November 2018 Norman Fenton and Martin Neil are hosting a reception at The Turing Institute to celebrate the launch of the Second Edition of their book "Risk Assessment and Decision Analysis with Bayesian Networks".

Slide show of the book 

A small number of places remain for people to register for the reception

Book Blog

New research published in IEEE Transactions makes building accurate Bayesian networks easier

One of the biggest practical challenges in building Bayesian network (BN) models for decision support and risk assessment is to define the probability tables for nodes with multiple parents. Consider the following example:

In any given week a terrorist organisation may or may not carry out an attack. There are several independent cells in this organisation for which it may be possible in any week to determine heightened activity. If it is known that there is no heightened activity in any of the cells, then an attack is unlikely. However, for any cell if it is known there is heightened activity then there is a chance an attack will take place. The more cells known to have heightened activity the more likely an attack is.

In the case where there are three terrorist cells, it seems to reasonable to assume the BN structure here:

To define the probability table for the node "Attack carried out" we have to define probability values for each possible combination of the states of the parent nodes, i.e., for all the entries of the following table.

That is 16 values (although, since the columns must sum to one we only really have to define 8).
When data are sparse - as in examples like this - we must rely on judgment from domain experts to elicit these values. Even for a very small example like this, such elicitation is known to be highly error-prone. When there are more parents (imagine there are 20 different terrorist cells) or more states other than "False" and "True", then it becomes practically infeasible.  Numerous methods have been proposed to simplify the problem of eliciting such probability tables. One of the most popular methods - “noisy-OR”- approximates the required relationship in many real-world situations like the above example. BN tools like AgenaRisk implement the noisy-OR function making it easy to define even very large probability tables. However, it turns out that in situations where the child node (in the example this is the node "Attack carried out") is observed to be "False", the noisy-OR function fails to properly capture the real world implications. It is this weakness that is both clarified and resolved in the following two new papers.

• Noguchi, T., Fenton, N. E., & Neil, M. (2018). "Addressing the Practical Limitations of Noisy-OR using Conditional Inter-causal Anti-Correlation with Ranked Nodes". IEEE Transactions on Knowledge and Data Engineering DOI: 10.1109/TKDE.2018.2873314 (This is the pre-publication version) 
• Fenton, N. E., Noguchi, T. & Neil, M, (2018). "An extension to the noisy-OR function to resolve the “explaining away” deficiency for practical Bayesian network problems", IEEE Transactions on Knowledge and Data Engineering, under review

The first paper (the online preprint version has just been published by the IEEE) shows how the problem is resolved by defining the nodes as 'ranked nodes' and using the weighted average function in AgenaRisk. The second paper shows that by changing a single column of the probability table generated from the noisy-OR function (namely the last column where all parents are "True") most (but not all) of the deficiencies in noisy-OR are resolved.

Hence the first paper provides a 'complete solution' but requires software like AgenaRisk for its implementation, while the second paper provides a simple approximate solution.

Acknowledgements: The research was supported by the European Research Council under project, ERC-2013-AdG339182 (BAYES_KNOWLEDGE); the Leverhulme Trust under Grant RPG-2016-118 CAUSAL-DYNAMICS; Intelligence Advanced Research Projects Activity (IARPA), to the BARD project (Bayesian Reasoning via Delphi) of the CREATE programme under Contract [2017-16122000003]. and Agena Ltd for software support. We also acknowledge the helpful recommendations and comments of Judea Pearl, and the valuable contributions of David Lagnado (UCL) and Nicole Cruz (Birkbeck).

Bayesian networks for trauma prognosis

There is an excellent online resource produced by Barbaros Yet that summarises the results of collaboration between the Risk and Information Management research group at Queen Mary and the Trauma Sciences Unit, Barts and the London School of Medicine and Dentistry. This work focused on developing Bayesian network (BN) models to improve decision support for trauma patients.

The website not only describes two BN models in detail (one for predicting acute traumatic coagulopathy in early stage of trauma care and one for predicting the outcomes of traumatic lower extremities with vascular injuries) but allows you to run the models in real time showing summary risk calculations after you enter observations about a patient.

The models are powered by AgenaRisk.

Links:

• http://traumamodels.com/
•  Perkins ZB, Yet B, Glasgow S, Marsh DWR, Tai NRM, Rasmussen TE (2018). “Long-term, patient centered outcomes of Lower Extremity Vascular Trauma”, Journal of Trauma and Acute Surgery. DOI:10.1097/TA.0000000000001956 
• Yet B, Perkins ZB, Tai NR, and Marsh DWR (2017). “Clinical Evidence Framework for Bayesian Networks” Knowledge and Information Systems, 50(1), pp.117-143.DOI:10.1007/s10115-016-0932-1 
•  Perkins ZB, Yet B, Glasgow S, Cole E, Marsh W, Brohi K, Rasmussen TE, Tai NRM (2015). “Meta-analysis of prognostic factors for amputation following surgical repair of lower extremity vascular trauma” British Journal of Surgery, 12 (5), pp. 436-450. DOI:10.1002/bjs.9689
• Yet B, Perkins ZB, Rasmussen TE et al.(2014). Combining data and meta-analysis to build Bayesian networks for clinical decision support. J Biomed Inform vol. 52, 373-385. http://dx.doi.org/10.1016/j.jbi.2014.07.018   http://qmro.qmul.ac.uk/xmlui/handle/123456789/23055
• Perkins ZB, Yet B, Glasgow S, Cole E, Marsh W, Brohi K, Rasmussen TE, Tai NRM (2015). “Meta-analysis of prognostic factors for amputation following surgical repair of lower extremity vascular trauma” British Journal of Surgery, 12 (5), pp. 436-450. DOI:10.1002/bjs.9689
  
• Yet B, Perkins ZB, Rasmussen TE, Tai NR, and Marsh DWR (2014). “Combining Data and Meta-analysis to Build Bayesian Networks for Clinical Decision Support” Journal of Biomedical Informatics , 52, pp.373-385. DOI:10.1016/j.jbi.2014.07.018
  
• Yet B, Perkins Z, Fenton N et al.(2014). Not just data: a method for improving prediction with knowledge. J Biomed Inform vol. 48, 28-37. http://dx.doi.org/10.1016/j.jbi.2013.10.012
• Yet B, Perkins Z, Tai N et al.(2014). Explicit evidence for prognostic Bayesian network models. Stud Health Technol Inform vol. 205, 53-57. http://dx.doi.org/10.3233/978-1-61499-432-9-53
• Perkins Z, Yet B, Glasgow S et al. (2013). EARLY PREDICTION OF TRAUMATIC COAGULOPATHY USING ADMISSION CLINICAL VARIABLES. SHOCK. vol. 40, 25-25.

It's finally arrived...

Still waiting to get our own copies of the second edition of the book, but one of our PhD students just received his copy, so it is real! Note that sample chapters and lots of other resources are available on the book's blog. The first edition (published Dec 2012) has 437 Google scholar citations, and many dozens of 5-star reviews on Amazon.

Second Edition of our book to be published 28 August 2018

From the back cover of the Second Edition:

************************************************

"The single most important book on Bayesian methods for decision analysts" —Doug Hubbard (author in decision sciences and actuarial science)  

"The book provides sufficient motivation and examples (as well as the mathematics and probability where needed from scratch) to enable readers to understand the core principles and power of Bayesian networks." —Judea Pearl (Turing award winner)  

"The lovely thing about Risk Assessment and Decision Analysis with Bayesian Networks is that it holds your hand while it guides you through this maze of statistical fallacies, p-values, randomness and subjectivity, eventually explaining how Bayesian networks work and how they can help to avoid mistakes.” —Angela Saini (award-winning science journalist, author & broadcaster)

Since the first edition of this book published, Bayesian networks have become even more important for applications in a vast array of fields. This second edition includes new material on influence diagrams, learning from data, value of information, cybersecurity, debunking bad statistics, and much more. Focusing on practical real-world problem-solving and model building, as opposed to algorithms and theory, it explains how to incorporate knowledge with data to develop and use (Bayesian) causal models of risk that provide more powerful insights and better decision making than is possible from purely data-driven solutions.

Features

• Provides all tools necessary to build and run realistic Bayesian network models
• Supplies extensive example models based on real risk assessment problems in a wide range of application domains provided; for example, finance, safety, systems reliability, law, forensics, cybersecurity and more
• Introduces all necessary mathematics, probability, and statistics as needed
• Establishes the basics of probability, risk, and building and using Bayesian network models, before going into the detailed applications

A dedicated website contains exercises and worked solutions for all chapters along with numerous other resources. The AgenaRisk software contains a model library with executable versions of all of the models in the book. Lecture slides are freely available to accredited academic teachers adopting the book on their course.

************************************************
Sample chapters are available on the book's website