Why target setting leads to poor decision-making

Norman Fenton is the co-author of an article in Nature published today that addresses the issue of improved decision-making in the context of international sustainable development goals. The article pushes for a Bayesian, smart-data approach:

We contend that target-setting is flawed, costly and could have little — or even negative — impact. First, targets may have unintended consequences. For example, education quality as a whole suffered in some countries that diverted resources to early schooling to meet the target of the Millennium Development Goal (MDG) of achieving universal primary education.

Second, target-setting inhibits learning by focusing efforts on meeting the target rather than solving the problem. The milestones are easily manipulated — aims such as halving deaths from road-traffic accidents can trigger misreporting if the performance falls short or encourage underperformance if the goal can be exceeded.

Third, it is costly: development partners will have to reallocate scant resources for a 'data revolution' that will cost an estimated US$1 billion a year.

We advocate a different approach. Governments and the development community need to embrace decision-analysis concepts and tools that have been used for decades in mining, oil, cybersecurity, insurance, environmental policy and drug development.

The approach is based on five principles:

1. Replace targets with measures of investment return
2. Model intervention decisions
3. Integrate expert knowledge
4. Include uncertainty in predictive models
5. Measure the most informative variables

Recommendations include the following:

It is a common mistake to assume that 'evidence' is the same as 'data' or that 'subjective' means 'uninformative'. Decision-making should draw on all appropriate sources of evidence. In developing countries where data are sparse, expert knowledge can fill the gaps. For instance, in our assessment of the viability of agroforestry projects in Africa, we used our experience to set ranges on tree-survival rates, costs of raising tree seedlings and farm prices of tree products.
 ....
Decision theorists and local experts will have to work together to identify relevant variables, causal associations and uncertainties. The most widely accepted method of incorporating knowledge for probability assessment is Bayes' theorem. This updates the likelihood of a belief in some event (such as whether an intervention will reduce poverty) when observing new evidence about the event (such as the occurrence of drought). Bayesian analyses — incorporating historical data and expert judgement — are used in transport and systems-safety assessments, medical diagnosis, operational risk assessment in finance and in forensics, but seldom in development. They should be used, for example, to evaluate the relative risks of competing development interventions. 
 ....
Decision-makers .. should employ probabilistic decision analysis, for example Monte Carlo simulations or Bayesian network models. Provided that such models are developed using properly calibrated expert judgement and decision-focused data, they can incorporate the key factors and outcomes and the causal relationships between them. For instance, simulations for evaluating options for building a water pipeline could take into account rare 'what-if' scenarios, such as a hurricane during development, and predict (with probabilities) the time and cost of implementation and the benefits of improved water supply.