menu

Friday, 22 June 2018

Bias in AI Algorithms


This is an update of a posting originally made on  18 Jan 2018 (see below for the update)

On 17 Jan 2018 multiple news sources (e.g. see here, here, and here) ran a story about a new research paper ‎ that claims to expose both the inaccuracies and racial bias in COMPAS - one of the most common algorithms used for parole and sentencing decisions to predict recidivism (i.e. whether or not a defendant will re-offend).

The research paper was written by the world famous computer scientist Hany Farid (along with a student Julia Dressel).

But the real story here is that the paper’s accusation of racial bias (specifically that the algorithm is biased against black people) is based on a fundamental misunderstanding of causation and statistics. The algorithm is no more ‘biased’ against black people than it is biased against white single parents, ‎ old people, people living in Beattyville Kentucky, or women called ‘Amber’. In fact, as we show in this brief article, if you choose any factor that correlates with poverty you will inevitably replicate the statistical ‘bias’ claimed in the paper. And if you accept the validity of the claims in the paper then you must also accept, for example, that a charity which uses poverty as a factor to identify and help homeless people is being racist because it is biased against white people (and also, interestingly, Indian Americans).

The fact that the article was published and that none of the media running the story realise that they are pushing fake news is what is most important here. Depressingly, many similar research studies involving the same kind of misinterpretation of statistics result in popular media articles that push a false narrative of one kind or another.

22 June 2018 Update: It turns out that now Microsoft is "developing a tool to help engineers catch bias in algorithms" This article also cites the case of the COMPAS software:
 "...., which uses machine learning to predict whether a defendant will commit future crimes, was found to judge black defendants more harshly than white defendants." 
Interestingly, this latest news article about Microsoft does NOT refer to the 2018 Dressel and Fardi article but, rather, to an earlier 2016 article by Larson et al: https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm From a quick inspection it does seem to be a more comprehensive study than the flawed Dressel and Farid article. But my quick impression is that the same fundamental misunderstandings statistics/causality are there. Given the great degree of interest in AI/bias, and given also that we were unaware of the 2016 study, we plan to do an update to our unpublished paper.

Our article (5 pages): Fenton, N.E., & Neil, M. (2018). "Criminally Incompetent Academic Misinterpretation of Criminal Data - and how the Media Pushed the Fake News" http://dx.doi.org/10.13140/RG.2.2.32052.55680  Also available here.

The research paper: Dressel, J. & Farid, H. The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4, eaao5580 (2018). 

Thanks to Scott McLachlan for the tip off on this story.

See some previous articles on poor use of statistics:

Wednesday, 20 June 2018

New project: Bayesian Artificial Intelligence for Decision Making under Uncertainty


Anthony Constantinou - a lecturer based in the Risk and Information Management Group at Queen Mary University of London - has been awarded a prestigious 3-year EPSRC Fellowship Grant £475,818 in partnership with Agena Ltd to develop open-source software that will enable end-users to quickly and efficiently generate Bayesian Decision Networks (BDNs) for optimal real-world decision-making. BDNs are Bayesian Networks augmented with additional functionality and knowledge-based assumptions to represent decisions and associated utilities that a decision maker would like to optimize. BDNs are suitable for modelling real-world situations where we seek to discover the optimal decision path to maximise utilities of interest and minimise undesirable risk.

A full description of the project can be found here. The EPSRC announcement is here.

Links

Thursday, 24 May 2018

The limitations of machine learning


Readers of this and our other blogs will be aware that we have long been sceptical of the idea that 'big data' - coupled with clever machine learning algorithms - will be able to achieve improved decision-making and risk assessment as claimed (see links below). We believe that a smart data approach that combines expert judgment (including understanding of underlying causal mechanisms) with relevant data is required and that Bayesian Networks (BNs) provide an ideal formalism for doing this effectively.

Turing award winner Judea Pearl, who was the pioneer of BNs, has just published a new book "The Book of Why: The New Science of Cause and Effect", which delivers essentially this same message. And there is a great interview with Pearl in the Atlantic Magazine about the book and his current views. The article includes the following:
As he sees it, the state of the art in artificial intelligence today is merely a souped-up version of what machines could already do a generation ago: find hidden regularities in a large set of data. “All the impressive achievements of deep learning amount to just curve fitting,” he said recently.
Read it all.

The interview also refers to the article "Human-Level Intelligence or Animal-Like Abilities?" by Adnan Darwiche. This is an outstanding paper (8 pages) that explains in more detail why we do not need to be over impressed by deep learning.

Links
    Norman gets his hands on Pearl's new book
    p.s. The second edition of our book:  Fenton, N.E. and M. Neil, "Risk Assessment and Decision Analysis with Bayesian Networks" (with foreword by Judea Pearl) will be available August 2018. See this space.

    Friday, 4 May 2018

    Anthony Constantinou's football prediction system wins second spot in international competition

     
    Anthony Constantinou

    QMUL lecturer Dr Anthony Constantinou of the RIM research group has come second in an international competition to produce the most accurate football prediction system. Moreover, the winners (whose predictive accuracy was only very marginally better) actually based their model on the previously published pi-ratings system of Constantinou and Fenton.





    Anthony's model Dolores was developed for the International Machine Learning for Soccer Competition hosted by the Machine Learning journal.

    All participants were provided with the results of matches from 52 different leagues around the world - with some missing data as part of the challenge. They had to produce a single model before the end of March 2017 that would be tested on its accuracy of predicting 206 future match outcomes from 26 different leagues, played from March 31 to April 9 in 2017.

    Dolores was ranked 2nd with a predictive accuracy almost the same as the top ranked system (there was less than 1% error rate difference between the two; the error rate was nearly 120% lower than the participants ranked lowest among those that passed the basic criteria).

    Dolores is  designed to predict football match outcomes in one country by observing football matches in multiple other countries.It is based on a) dynamic ratings and b) Hybrid Bayesian Networks.

    Unlike past academic literature which tends to focus on a single league or tournament, Dolores provides empirical proof that a model can make a good prediction for a match outcome between teams 𝑥 and 𝑦 even when the prediction is derived from historical match data that neither 𝑥 nor 𝑦 participated in. This implies that we can still predict, for example, the outcome of English Premier League matches, based on training data from Japan, New Zealand, Mexico, South Africa, Russia, and other countries in addition to data from the English Premier league.

    The Machine Learning journal has published the descriptions of the highest ranked systems in its latest issue published online today. The full reference for Anthony's paper is:

    Constantinou, A. (2018). Dolores: A model that predicts football match outcomes from all over the world. Machine Learning, 1-27, DOI: https://doi.org/10.1007/s10994-018-5703-7

    The full published version can be viewed (for free) at https://rdcu.be/Nntp. An open access pre-publication version (pdf format) is available for download here.

    This work was partly supported by the European Research Council (ERC), research project ERC-2013-AdG339182-BAYES_KNOWLEDGE
    The DOLORES Hybrid Bayesian Network was built and run using the AgenaRisk software.

    The full reference for the pi-ratings model (used by the competition's winning team) is:
    Constantinou, A. C. & Fenton, N. E. (2013). Determining the level of ability of football teams by dynamic ratings based on the relative discrepancies in scores between adversaries. Journal of Quantitative Analysis in Sports. Vol. 9, Iss. 1, 37–50. DOI: http://dx.doi.org/10.1515/jqas-2012-0036
    Open access version here.
    See also:

    Monday, 30 April 2018

    Bayesian Nets to Determine Impact of Agricultural Development Policy



    An interesting paper - describing use of Bayesian nets to determine impact of agricultural development policy on household nutrition in Uganda -  uses the new 'Value of Information' functionality developed in BAYES-KNOWLEDGE.

    Full reference:
    Cory W. Whitney Denis Lanzanova Caroline Muchiri Keith D. Shepherd Todd S. Rosenstock Michael Krawinkel John R. S. Tabuti Eike Luedeling (2018), "Probabilistic Decision Tools for Determining Impacts of Agricultural Development Policy on Household Nutrition", Earth's Future (Open Access) https://doi.org/10.1002/2017EF000765

    Tuesday, 17 April 2018

    Explaining Bayesian Networks through a football management problem



    Today's Significance Magazine (the magazine of the Royal Statistical Society and the American Statistical Association) has published an article by Anthony Constantinou and Norman Fenton that explains, through the use of an example from football management, the kind of assumptions required to build useful Bayesian networks (BNs) for complex decision-making. The article highlights the need to fuse data with expert knowledge, and describes the challenges in doing so. It also explains why, for fully optimised decision-making, extended versions of BNs, called Bayesian decision networks, are required.

    The published pdf (open source) is also available here and here.

    Full article details:
    Constantinou, A., Fenton, N.E, "Things to know about Bayesian networks", Significance, 15(2), 19-23 April 2018, https://doi.org/10.1111/j.1740-9713.2018.01126.x

    Wednesday, 14 March 2018

    Tuesday, 6 March 2018

    Two coins: one fair one biased

    Alexander Bogolmony tweeted this problem:


    If there is no reason to assume in advance that either coin is more likely to be the coin tossed once (i.e. the first coin) then all the (correct) solutions show that the first coin is more likely to be biased with a probability of 9/17 (=0.52941). Here is an explicit Bayesian network solution for the problem:


    The above figure shows the result after entering the 'evidence' (i.e. one Head on the coin tossed once and two Heads on the coin tossed three times). The tables displayed are the conditional probability tables defined for the associated with the variables.

    This model took just a couple of minutes to build in AgenaRisk and requires absolutely no manual calculations as the Binomial distribution is one of many functions pre-defined. The model (which can be run in the free version of AgenaRisk is here). The nice thing about this solution compared to the others is that it is much more easily extendible. It also shows the reasoning very clearly.


    Monday, 12 February 2018

    An Improved Method for Solving Hybrid Influence Diagrams

    Most decisions are made in the face of uncertain factors and outcomes. In a typical decision problem, uncertainties involve both continuous factors (e.g. amount of profit) and discrete factors (e.g. presence of a small number of risk events). Tools such as decision trees and influence diagrams are used to cope with uncertainty regarding decisions, but most implementations of these tools can only deal with discrete or discretized factors and ignore continuous factors and their distributions.

    A paper just published in the International Journal of Approximate Reasoning presents a novel method that overcomes a number of these limitations. The method is able to solve decision problems with both discrete and continuous factors in a fully automated way. The method requires that the decision problem is modelled as a Hybrid Influence Diagrams, which is an extension of influence diagrams containing both discrete and continuous nodes, and solves it by using a state-of-the-art inference algorithm called Dynamic Discretization. The optimal policies calculated by the method are presented in a simplified decision tree.



    The full reference is:

    Yet, B., Neil, M., Fenton, N., Dementiev, E., & Constantinou, A. (2018). "An Improved Method for Solving Hybrid Influence Diagrams". International Journal of Approximate Reasoning. DOI: 10.1016/j.ijar.2018.01.006  Preprint (open access) available here.
    UPDATE (22 Feb 2018): The full published version the paper is available online for free for 50 days here: https://authors.elsevier.com/c/1Wc6D,KD6ZG8y-

    Acknowledgements: Part of this work was performed under the auspices of EU project ERC-2013-AdG339182-BAYES_KNOWLEDGE

    Friday, 9 February 2018

    Decision-making under uncertainty: computing "Value of Information"


    Information gathering is a crucial part of decision making under uncertainty. Whether to collect additional information or not, and how much to invest for such information are vital questions for successful decision making. For example, before making a treatment decision, a physician has to evaluate the benefits and risks of additional imaging or laboratory tests and decide whether to ask for them. Value of Information (VoI) is a quantitative decision analysis technique for answering such questions based on a decision model. It is used to prioritise the parts of a decision model where additional information is expected to be useful for decision making.

    However, computing VoI in decision models is challenging especially when the problem involves both discrete and continuous variables. A new paper in the IEEE Access journal illustrates a simple and practical approach that can calculate VoI using Influence Diagram models that contain both discrete and continuous variables. The proposed method can be applied to a wide variety of decision problems as most decisions can be modelled as an influence diagram, and many decision modelling tools, including Decision Trees and Markov models, can be converted to an influence diagram.

    The full reference is:

    Yet, B., Constantinou, A., Fenton, N., & Neil, M. (2018). Expected Value of Partial Perfect Information in Hybrid Models using Dynamic Discretization.  IEEE Access. DOI: 10.1109/ACCESS.2018.2799527

    Acknowledgements: Part of this work was performed under the auspices of EU project ERC-2013-AdG339182-BAYES_KNOWLEDGE, EPSRC project EP/P009964/1: PAMBAYESIAN, and ICRAF Contract No SD4/2012/214 issued to Agena.

    Wednesday, 7 February 2018

    Lawnmower v terrorist risk: the saga continues


    Kim Kardashian's tweet comparing risk from lawnmowers v terrorists triggered the award and debate

    Yesterday Significance Magazine (the magazine of the Royal Statistical Society and the American Statistical Association) published an article “Lawnmowers versus Terrorists”  with the strapline:
    The Royal Statistical Society’s first ‘International Statistic of the Year’ sparked plenty of online discussion. Here, Norman Fenton and Martin Neil argue against the choice of winner, while Nick Thieme writes in support.
    Our case, titled “A highly misleading view of risk”, was an edited version of a paper  previously publicised in a blog post that itself followed up on original concerns raised by Nicholas Nassim Taleb about the RSS citation and the way it had been publicised. The ‘opposing’ case made by Nick Thieme was essentially a critique of our paper.

    We have today published a response to Nick’s critique.


    Links:

    Monday, 5 February 2018

    Revisiting a Classic Probability Puzzle: the Two Envelopes Problem


    Many people have heard about the Monty Hall problem. A similar (but less well known and more mathematically interesting) problem is the two envelopes problem, which Wikipedia describes as follows:
    “You are given two indistinguishable envelopes, each containing money, one contains twice as much as the other. You may pick one envelope and keep the money it contains. Having chosen an envelope at will, but before inspecting it, you are given the chance to switch envelopes. Should you switch?”
    The problem has been around in various forms since 1953 and has been extensively discussed (see, for example Gerville-Réache for a comprehensive analysis and set of references)  although I was not aware of this until recently.

    We actually gave this problem (using boxes instead of envelopes) as an exercise in the supplementary material for our Book, after Prof John Barrow of University of Cambridge first alerted us to it. The ‘standard solution’ (as in the Monty Hall problem) says that you should always switch. This is based on the following argument:
    If the envelope you choose contains $100 then there is an evens chance the other envelope contains $50 and an evens chance it contains $200. If you do not switch you have won $100. If you do switch you are just as likely to decrease the amount you win as increase it. However, if you win the amount increases by $100 and if you lose it only decreases by $50. So your expected gain is positive (rather than neutral). Formally, if the envelope contains S then the expected amount in the other envelope is 5/4 times X (i.e. 25% more).
    In fact (as pointed out by a reader Hugh Panton), the problem with the above argument is that it equally applies to the ‘other envelope’ thereby suggesting we have a genuine paradox. In fact, it turns out that the above argument only really works if you actually open the first envelope (which was explicitly not allowed in the problem statement) and discover it contains S. As Gerville-Réache shows, if the first envelope is not opened, the only probabilistic reasoning that does not use supplementary information leads to estimating expectations as infinite amounts of each envelope. Bayesian reasoning can be used to show that there is no benefit in switching, but that is not what I want to describe here.

    What I found interesting is that I could not find - in any of the discussions about the problem - a solution for the case where we assume there is a finite maximum prize, even if we allow that maximum to be as large as we like. With this assumption it turns out that we can prove (without dispute) that there is no benefit to be gained if you stick or switch. See this short paper for the details:
    Fenton N E, "Revisiting a Classic Probability Puzzle: the Two Envelopes Problem" 2018, DOI10.13140/RG.2.2.24641.04960

    Thursday, 18 January 2018

    Criminally Incompetent Academic Misinterpretation of Criminal Data - and how the Media Pushed the Fake News



    On 17 Jan 2018 multiple news sources (e.g. see here, here, and here) ran a story about a new research paper ‎ that claims to expose both the inaccuracies and racial bias in one of the most common algorithms used for parole and sentencing decisions to predict recidivism (i.e. whether or not a defendant will re-offend).

    The research paper was written by the world famous computer scientist Hany Farid (along with a student Julia Dressel).

    But the real story here is that the paper’s accusation of racial bias (specifically that the algorithm is biased against black people) is based on a fundamental misunderstanding of causation and statistics. The algorithm is no more ‘biased’ against black people than it is biased against white single parents, ‎ old people, people living in Beattyville Kentucky, or women called ‘Amber’. In fact, as we show in this brief article, if you choose any factor that correlates with poverty you will inevitably replicate the statistical ‘bias’ claimed in the paper. And if you accept the validity of the claims in the paper then you must also accept, for example, that a charity which uses poverty as a factor to identify and help homeless people is being racist because it is biased against white people (and also, interestingly, Indian Americans).

    The fact that the article was published and that none of the media running the story realise that they are pushing fake news is what is most important here. Depressingly, many similar research studies involving the same kind of misinterpretation of statistics result in popular media articles that push a false narrative of one kind or another.

    22 June Update

    Our article (5 pages): Fenton, N.E., & Neil, M. (2018). "Criminally Incompetent Academic Misinterpretation of Criminal Data - and how the Media Pushed the Fake News" http://dx.doi.org/10.13140/RG.2.2.32052.55680  Also available here.

    The research paper: Dressel, J. & Farid, H. The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4, eaao5580 (2018). 

    Thanks to Scott McLachlan for the tip off on this story.

    See some previous articles on poor use of statistics:

    Thursday, 11 January 2018

    On lawnmowers and terrorists again: the danger of using historical data alone for decision-making


    The short paper and blog posting we did last week generated a lot of interest, especially after Nicholas Taleb retweeted it. An edited version (along with a response from a representative of the Royal Statistical Society) is going to appear in the February issue of Significance magazine (which is the magazine of the RSS and the American Statistical Association). In the mean time we have produced another short paper that explores further problems with the 'lawnmower versus terrorist risk' statistics - in particular the inevitable limitations and dangers of relying on historical data alone for risk assessment:
    Fenton, N.E., & Neil, M. (2018). "Is decision-making using historical data alone more dangerous than lawnmowers?", Open Access Report DOI:10.13140/RG.2.2.20914.71363. Also available here.

    Wednesday, 3 January 2018

    Are lawnmowers a greater risk than terrorists?


    Kim Kardashian, whose tweet comparing the threats of lawnmowers and terrorists, led to RSS acclaim
    In December 2017 the Royal Statistical Society (RSS) announced the winner of its “International Statistic of the Year”. The statistic was simply "69" which it said was "the annual number of Americans killed, on average, by lawnmowers - compared to two Americans killed annually, on average, by immigrant Jihadist terrorists".  The full RSS citation says that the statistic tweeted by Kim Kardashian ‘highlights misunderstandings of risk’ and ‘illuminates the bigger picture’. Unfortunately, we believe it does exactly opposite as we explain in this brief paper:
    Fenton, N.E., & Neil, M. (2018). "Are lawnmowers a greater risk than terrorists?" Open Access Report DOI:10.13140/RG.2.2.34461.00486/1 
    As you can see from the tweet by Taleb, this use of statistics for risk assessment was not universally welcomed.


    See update to this story here.