Thursday 24 May 2018

The limitations of machine learning

Readers of this and our other blogs will be aware that we have long been sceptical of the idea that 'big data' - coupled with clever machine learning algorithms - will be able to achieve improved decision-making and risk assessment as claimed (see links below). We believe that a smart data approach that combines expert judgment (including understanding of underlying causal mechanisms) with relevant data is required and that Bayesian Networks (BNs) provide an ideal formalism for doing this effectively.

Turing award winner Judea Pearl, who was the pioneer of BNs, has just published a new book "The Book of Why: The New Science of Cause and Effect", which delivers essentially this same message. And there is a great interview with Pearl in the Atlantic Magazine about the book and his current views. The article includes the following:
As he sees it, the state of the art in artificial intelligence today is merely a souped-up version of what machines could already do a generation ago: find hidden regularities in a large set of data. “All the impressive achievements of deep learning amount to just curve fitting,” he said recently.
Read it all.

The interview also refers to the article "Human-Level Intelligence or Animal-Like Abilities?" by Adnan Darwiche. This is an outstanding paper (8 pages) that explains in more detail why we do not need to be over impressed by deep learning.

    Norman gets his hands on Pearl's new book
    p.s. The second edition of our book:  Fenton, N.E. and M. Neil, "Risk Assessment and Decision Analysis with Bayesian Networks" (with foreword by Judea Pearl) will be available August 2018. See this space.

    1 comment:

    1. AI Patasala provides an advanced platform for Machine Learning Training in Hyderabad. This program is designed to those who want to pursue employment in computer science or machine learning platforms.
      Machine Learning Training Hyderabad