|
Henry Prakken |
Norman Fenton, 17 July 2016
There has been an important development on the use of
Bayes in the Law in the Netherlands, with what is possibly the first
full Bayesian analysis of a major crime in an appeal court there.
The case, referred to as the “Breda 6”, was the
1993 murder of a Chinese woman in Breda in her son’s restaurant.
Six young people were convicted of the crime and sentenced to up to 10
years in
jail (all have since completed their sentences). In 2012 the advocate
general recommended the case be looked at again because it centred on
confessions which may have been false.
In the review of the case a Bayesian argument supporting the prosecution case was presented by Frans Alkemade (
Update: see
below about some concerns Frans has about this article). Frans is
actually a physicist who previously used Bayes to analyse a
drugs trafficking case, concluding in a report commissioned by the
prosecution, that there was not enough evidence for a conviction (the
suspect was acquitted). The court requested that Henry Prakken
(professor in Legal Informatics
and Legal Argumentation at the University of Groningen) respond to the
Bayesian argument.
In June, while he was preparing his
response, I met Henry at the
International Conference on AI and the Law in San Diego. Henry told me
about the case and Alkemade's analysis for which the guilty hypothesis
was
"At least some of the six suspects were involved in the crime, which
took place after 4:30 on the night of 3 and 4 july 1993, and which
included the luring of the victim to the restaurant by at least some of
the female suspects".
Alkemade interpreted "involved" in a weak way and Henry said:
"..it
could even be no more than just knowing about the crime. In fact, one
of my points of criticism was that this guilt hypothesis is not useful
for the court, since it is consistent with the innocence of any of the
individual suspects (and even with the collective innocence of all three
male suspects)."
Among other things, Alkemade focused on two pieces of evidence:
- A report by the Criminal Intelligence Unit (CID) of the Dutch
police, saying that they had received information from "usually
reliable" sources identifying two of the male defendants and one of the
female defendants as being involved in the murder**.
- The subsequent discovery that two of the female defendants (not
mentioned in the CID report and who supposedly knew the three defendants
mentioned in the CID report) worked next door to the murder scene.
From Henry's description of the analysis, it seemed that
Alkemade did not account for all relevant unknown variables and
dependencies*** (also see the update), such as the possibility that
the anonymous tip-off may have been both malicious and prompted by the
fact that
the caller knew the defendants worked next door to the murder scene
(making the tip-off more believable). This would mean that the
combination of the two pieces of evidence would not have been such an
incredible coincidence if the defendants were innocent. So in that sense
the Bayesian argument was over-simplistic. On the other hand it was
also too complex for lawyers to understand since it was presented 'from
first principles' in the sense that all of the detailed Bayesian
inference calculations were spelled out. For the reasons we have
explained in detail
here
it seemed like a Bayesian network (BN) model would be far more
suitable. I therefore produced - in discussions with Henry - a generic
BN model to reason about the impact of anonymous evidence when combined
with other evidence that can influence the anonymous tip-off (the model
is
here and can be run using the
free AgenaRisk software).
The
intention was not to replicate all of the features of the case but
rather to demonstrate the impact of missing dependencies in Alkemade's
argument. Indeed, with a range of reasonable assumptions, the BN model
pointed to a much lower probability of guilt than suggested by
Alkemade's calculations.
Henry presented his response in court last week. He said:
"The court session was sometimes frustrating, since the
discussion was fragmentary and sometimes I had the impression that the court felt
it was confronted with a battle of the experts without the means to
understand who was right."
In my view this case confirms
our claim
that presenting a Bayesian legal argument from first principles (as
Alkemade did) is not a good idea.
The very fact that people assume it is necessary to do this for Bayes to
be accepted is actually the reason (ironically) that there will
continue to be very strong resistance to accepting Bayes in the
courtroom. Why? Because it means you are restricted to ludicrously
over-simplistic (and normally flawed) models of the case (3 unknowns
maximum) because that is the limit of the Bayesian calculations you can
do by hand and explain clearly. Our proposed solution is to model (and
run) the case properly using a BN and report back on the results stating
in lay terms what the model assumptions were and how sensitive the
conclusions are to different prior assumptions.
**Henry told me that there was something funny with the CID report, as
also noted by the advocate general. According to the CID report, the
anonymous informant had also accused the defendants mentioned in the
report of committing several other crimes, but in the investigations
preceding the revision case the police investigators had not been able
to find any confirmation of these other crimes, not even reports of
these supposed crimes to the police by the supposed victims. As the
advocate general stated in 2012, this casts doubt on the reliability of
the CID informants.
***My colleague Richard Gill - who
knows Alkemade - says that Alkemade was careful to define his pieces of
evidence in such a way that he thinks that he can justify the
independence assumptions which he needs in order to at least
conservatively bound the Likelihood ratio coming from each piece of
evidence in turn.
UPDATE 21 July 2015: Frans has
contacted me stating a number of concerns
about the above narrative and provided a number of technical insights
that I was not aware of.
As an expert witness in a case that is still under trial, he does not
feel free to discuss any details in public, but once the trial has
finished I will provide an updated report that incorporates his
comments. What I can confirm, however, is that in order to do the
calculations manually Frans could not model dependencies between
different pieces of evidence - a major limitation - although he did make
clear the limitations and pitfalls of this in his report.
See also: