Norman Fenton at Maths in Action Day (Warwick University)

Today Norman Fenton was one of the five presenters at the Mathematics in Action Day at Warwick University - the others included writer and broadcaster Simon Singh and BBC presenter Steve Mould (who is also part of the amazing trio Festival of the Spoken Nerd which features Queen Mary's Matt Parker). The Maths in Action day is specifically targeted at A-Level Maths students and their teachers.

Norman says:

This was probably the biggest live event I have spoken at - an audience of 550 in the massive Butterworth Hall (which has recently hosted Paul Weller and the Style Council, Jools Holland) - so it was quite intimidating. My talk was on "Fallacies of Probability and Risk" (the powerpoint slides are here). I hope to get some photos of the event uploaded shortly.

Butterworth Hall (hopefully some real photos from the event to come)

Bayesian Networks for Risk Assessment of Public Safety and Security Mobile Service

A new paper by Matti Peltola and Pekka Kekolahti of the Aalto University (School of Electrical Engineering) in Finland uses Bayesian Networks and the AgenaRisk tool to gain a deeper understanding of the availability of Public Safety and Security (PSS) mobile networks and their service under different conditions. The paper abstract states:

A deeper understanding of the availability of Public Safety and Security (PSS) mobile networks and their service under different conditions offers decision makers guidelines on the level of investments required and the directions to take in order to decrease the risks identified. In the study, a risk assessment model for the existing PSS mobile service is implemented for both a dedicated TETRA PSS mobile network as well as for a commercial 2G/3G mobile network operating under the current risk conditions. The probabilistic risk assessment is carried out by constructing a Bayesian Network. According to the analysis, the availability of the dedicated Finnish PSS mobile service is 99.1%. Based on the risk assessment and sensitivity analysis conducted, the most effective elements for decreasing availability risks would be duplication of the transmission links, backup of the power supply and real-time mobile traffic monitoring. With the adjustment of these key control variables, the service availability can be improved up to the level of 99.9%. The investments needed to improve the availability of the PSS mobile service from 99.1 % to 99.9% are profitable only in highly populated areas. The calculated availability of the PSS mobile service based on a purely commercial network is 98.8%. The adoption of a Bayesian Network as a risk assessment method is demonstrated to be a useful way of documenting different expert knowledge as a common belief about the risks, their magnitudes and their effects upon a Finnish PSS mobile service.

Full reference details:

Peltola, M. J., & Kekolahti, P. (2015). Risk Assessment of Public Safety and Security Mobile Service. In 2015 10th International Conference on Availability, Reliability and Security (pp. 351–359). IEEE. doi:10.1109/ARES.2015.65

Yet another flawed statistical study attracts massive unquestioning attention

The Guardian, 29 Sept 2015

A very widely reported story in today’s news (see, for example, the report in the Guardian and this Press release) claims that companies in which there is at least one female executive on the Board (‘gender diverse’ companies) in the US, UK and India outperform companies with male-only executives by a staggering US$655 billion per year. The story is based on a study by Grant Thornton whose representative Francesca Lagerberg concludes:

“The research clearly shows what we have been talking about for a while: that diversity leads to better decision-making”.

As is typical when the results of a statistical study fit a popular narrative, the story attracted massive, unquestioning attention. Unfortunately, while I am sure that most people agree that greater gender diversity in the Boardroom is a worthy objective, based on the ‘full report’ – and in the absence of other data - Lagerberg's claim is simply not supported. In fact, the study exemplifies some of the classic misuses of statistics that we wrote about in the first chapter of our book and highlights yet again the need for proper causal/explanatory models to be used in statistical studies such as these*.

Moreover, using the data in Lagerberg's study it is possible to construct a simple causal model (a Bayesian network) that replicates the results but with provably opposite conclusions: diversity decreases performance.

The full report and BN model are provided here. The model can be run in the free version of AgenaRisk.

*Making such an approach both universally feasible and acceptable is the major objective of BAYES-KNOWLEDGE.

Why target setting leads to poor decision-making

Norman Fenton is the co-author of an article in Nature published today that addresses the issue of improved decision-making in the context of international sustainable development goals. The article pushes for a Bayesian, smart-data approach:

We contend that target-setting is flawed, costly and could have little — or even negative — impact. First, targets may have unintended consequences. For example, education quality as a whole suffered in some countries that diverted resources to early schooling to meet the target of the Millennium Development Goal (MDG) of achieving universal primary education.

Second, target-setting inhibits learning by focusing efforts on meeting the target rather than solving the problem. The milestones are easily manipulated — aims such as halving deaths from road-traffic accidents can trigger misreporting if the performance falls short or encourage underperformance if the goal can be exceeded.

Third, it is costly: development partners will have to reallocate scant resources for a 'data revolution' that will cost an estimated US$1 billion a year.

We advocate a different approach. Governments and the development community need to embrace decision-analysis concepts and tools that have been used for decades in mining, oil, cybersecurity, insurance, environmental policy and drug development.

The approach is based on five principles:

1. Replace targets with measures of investment return
2. Model intervention decisions
3. Integrate expert knowledge
4. Include uncertainty in predictive models
5. Measure the most informative variables

Recommendations include the following:

It is a common mistake to assume that 'evidence' is the same as 'data' or that 'subjective' means 'uninformative'. Decision-making should draw on all appropriate sources of evidence. In developing countries where data are sparse, expert knowledge can fill the gaps. For instance, in our assessment of the viability of agroforestry projects in Africa, we used our experience to set ranges on tree-survival rates, costs of raising tree seedlings and farm prices of tree products.
 ....
Decision theorists and local experts will have to work together to identify relevant variables, causal associations and uncertainties. The most widely accepted method of incorporating knowledge for probability assessment is Bayes' theorem. This updates the likelihood of a belief in some event (such as whether an intervention will reduce poverty) when observing new evidence about the event (such as the occurrence of drought). Bayesian analyses — incorporating historical data and expert judgement — are used in transport and systems-safety assessments, medical diagnosis, operational risk assessment in finance and in forensics, but seldom in development. They should be used, for example, to evaluate the relative risks of competing development interventions. 
 ....
Decision-makers .. should employ probabilistic decision analysis, for example Monte Carlo simulations or Bayesian network models. Provided that such models are developed using properly calibrated expert judgement and decision-focused data, they can incorporate the key factors and outcomes and the causal relationships between them. For instance, simulations for evaluating options for building a water pipeline could take into account rare 'what-if' scenarios, such as a hurricane during development, and predict (with probabilities) the time and cost of implementation and the benefits of improved water supply.

The statistics of sex

Sir David Spiegelhalter (left) and Norman Fenton

Norman Fenton, 28 April 2015

Last night I attended the launch of David Spiegelhalter's book "Sex by Numbers"** at the Wellcome Collection in London, which is currently also hosting an exhibition on Sexology.

What makes David's book a very good read is that it not only presents intriguing insights into a broad range of sexual activities, but it does so in a way that explains in lay terms the good, bad and ugly of the underlying statistical methods as well as some of the maths. This includes things like erroneous reporting of sexual habits that falls into the category of prosecutors' fallacy***. There are hundreds of different numbers about sex presented and most get a star rating (ranging from 1* to 4*) based on their reliability; so, for example, the surprisingly high number 48% (births that were formally 'illegitimate' in 2012 in England and Wales) is in the most reliable category (4*), while the number 84% (women emotionally unsatisfied with their relationship) is in the least reliable category (1*). The numbers for favourite sexual positions as presented in the following table are rated as 2*:

	Women	Men
Man on top	48%	25%
Woman on top	33%	45%
Doggy	15%	25%

To give a feel for the range of numbers the book provides those 80% of 25-34 year-olds who have engaged in oral sex in the last year will be interested to know that 3% is the proportion of recommended daily zinc intake contained in an average ejaculation.

**David was one of my co-presenters on the BBC documentary Climate Change by Numbers. The other co-presenter was Hannah Fry, whose book published in February is called "The Mathematics of Love". I deny the rumours circulating that my next book is to be called "The Risks of Marriage"....

***see here for background on the prosecutors' fallacy

The risk of flying

Norman Fenton, 26 March 2015

I have just done an interview on BBC Radio Scotland about aircraft safety in the light of the GermanWings crash - which now appears to have been a deliberate act of sabotage by the co-pilot*. I have uploaded a (not very good) recording of it here (mp3 file - it is just under 4 minutes) or here (a more compact m4a file)

Because this type of event is so rare classical frequentist statistics provides no real help when it comes to risk assessment. In fact, it is exactly the kind of risk assessment problem for which you need causal models and expert judgement (as explained in our book) if you want any kind of risk insights.

Irrespective of this particular incident, the interview gave me the opportunity to highlight a very common myth, namely that “flying is the safest form of travel”.  If you look at deaths per million travellers then, indeed, there are 50 times as many car deaths as plane deaths. However, this is a silly measure because there are so many more car travellers than plane travellers. So, typically, analysts use deaths per million miles travelled; with respect to this measure car travel is still 'riskier' than air travel, but the death rate is only about twice as high as plane deaths. But this measure is also biased in favour of planes because the average plane journey is much further than the average car journey.

So a much fairer measure is the number of deaths per passenger journey. And for this, the rate of plane deaths is actually three times higher than car deaths; in fact only bikes and motorbikes are worse than planes.

Despite all this there is still a very low probability of a plane journey resulting in fatalities - about 1 in half a million (and much less on commercial flights in Western Europe). However, if we have reason to believe that, say, recent converts to a terrorist ideology have been training and becoming pilots then the probability of the next plane journey resulting in fatalities becomes much higher, despite the past data.

*I had an hour’s notice of the interview and was told what I would be asked.  I was actually not expecting to be asked about how to assess the risk of this specific type of incident;  I was assuming I would only be asked about aircraft safety risk in general and about the safety record of the A320.

Postscript: Following the interview a colleage asked:
"Did you have the mental issues of the co-pilot on the radar when you replied? "
My response: Interesting question. A few years back we were involved extensively in work with NATS (National Air Traffic Safety) to model/predict risk of mid-air collision over the UK airspace. In particular NATS wanted to know how the probability of a mid-air collision might change given different proposals for changes to the ATM architecture (e.g. ‘adding new ground radar stations’ versus ‘adding new on-board collisions alert systems’). Now - apart from three incidents in the late 1940’s which all involved at least one military jet - there has not been any actual mid-air collisions over UK airspace (so negligible data there) and the proposed technology was ‘new’ (so no directly relevant data there) but there was a LOT of data on "near misses" of different degrees of seriousness  and a LOT of expert judgment about the causes and circumstances of the near misses. Hence, we were able with NATS experts to build a very detailed model that could be ‘validated’ against the actual near miss data. What is very interesting are what factors NATS needed in the model. The psychological state and stress of air traffic controllers was included in the model as were certain psychological traits of pilots. It turns out that certain airlines were more likely to be involved in a near-misses primarily because of traits of their pilots.

Statistics of Poverty

Norman Fenton, 2 April 2014

I was one of two plenary speakers at the Winchester Conference on Trust, Risk, Information and the Law yesterday (slides of my talk: "Improving Probability and Risk Assessment in the Law" are here).

The other plenary speaker was Matthew Reed (Chief Executive of the Children's Society) who spoke about "The role of trust and information in assessing risk and protecting the vulnerable". In his talk he made the very dramatic statement that

"one in every four children in the UK today lives in poverty"

He further said that the proportion had increased significantly over the last 25 years and showed no signs of improvement.

When questioned about the definition of child poverty he said he was using the Child Poverty Act 2010 definition which defines a child as living in poverty if they lived in a household whose income (which includes benefits) is less than 60% of the national median (see here).

Matthew Reed has a genuine and deep concern for the welfare of children. However, the definition is purely political and is as good an example of poor measurement and misuse of statistics as you can find. Imagine if every household was given an immediate income increase of 1000%  - this would mean the very poorest households with, say, a single unemployed parent and 2 children going from £18,000 to a fabulously wealthy £180,000 per year. Despite this, one in every four children would still be 'living in poverty' because the number of households whose income is less than 60% of the median has not changed.  If the median before was £35,000, then it is now £350,000 and everybody earning below  £210,000 is, by definition, 'living in poverty'.

At the other extreme if you could ensure that every household in the UK earns a similar amount, such as in Cuba where almost everybody earns $20 per month then the number of children 'living in poverty' is officially zero (since the median is $240 per year and nobody earns less than $144).

In fact, in any wealthy free-market economy whichever way you look at the definition it is loaded not only to exaggerate the number of people living in poverty but also to ensure (unless there is massive wealth redistribution to ensure every household income is close to the median level) there will always be a 'poverty' problem:

• Households with children are much more likely to have one, rather than two, wage earners, so by definition households with children will dominate those below the median income level.
• Over the last 20 years people have been having fewer children and having them later in life, which again means that an increasing proportion of the country's children inevitably live in households whose income is below the median (hence the 'significant increase in the proportion of children living in poverty over the last 25 years').
• Families with large numbers of children (> 3) increasingly are in the immigrant community (Asia/Africa) whose households are disproportionately below the median income. 

Unless the plan is stop households on below median income from having children (also known as eugenics), the only way to achieve the stated objective of 'making child poverty history' (according to this definition) is to redistribute wealth so that no household income is less than 60% of the median (also known as communism). Judging by some of the people who have been pushing the 'poverty' definition and agenda it would seem the latter is indeed their real objective.