Crime statistics: how many warnings do you need?
If you've been following the news this week the following might have caught your attention:
Sun, 13 May 2013
Daily Mail, 13 May 2013
Off the back of a Sun Freedom of Information (FOI) request to the 43 police forces in England and Wales, it was reported that one in five suspected rapists or murderers last year was born abroad.
According to the newspaper, 93 out of 555 murder suspects (17%) were foreign nationals compared to 632 out of 3,436 charged with rape (18%). It almost goes without saying that the figures actually suggest one in six of either is a foreign national. Unfortunately, the Sun don't link to the FOI responses they received, so it's impossible for a reader to follow the numbers to the source.
But that's not actually the issue here. Full Fact has seen media outlets using 'mass FOIs' to gather statistics before. While there's nothing wrong with gathering crime figures from individual police forces to inform local policy, problems start to arise when they are used to compare with other police forces.
The reason for this is that police forces record crime differently. All police forces have to submit crime data to the Home Office, but the only requirement is to send a count of the number of offences which they have spotted over a given time period. Only offences on the Home Office's 'Notifiable Offence List' need to be included.
For this reason, data on the characteristics of offenders - such as nationality - isn't captured centrally and might or might not be recorded locally - the extent and nature of the recording will vary across each police force.
We can actually trace back the Sun's FOI requests to similar ones listed in 'disclosure logs' on police force websites - these are where previous FOI responses are published. Both the Metropolitan Police and North Yorkshire Police's responses to the request about murder and rape charges for offenders contained the warning:
"Police forces in the United Kingdom are routinely required to provide crime statistics to government bodies and the recording criteria is set nationally. However, the systems used for recording these figures are not generic, nor are the procedures used locally in capturing the crime data. It should be noted that for these reasons this force's response to your questions should not be used for comparison purposes with any other response you may receive." [emphasis added]
However, this warning isn't always appended to FOIs. Dyfed-Powys police's response didn't contain any note of caution.
To find out more about these warnings, Full Fact decided to play the newspapers' game. Last year, we sent out FOIs to all 43 police forces in England and Wales, asking how many recorded offences in the area were known to have been committed by foreign nationals.
In addition, we explicity asked:
"Please provide guidelines or advice as to whether the data you release for this request can be used for comparison with other police forces, and whether the data you release can be conglomerated with that of other forces to provide a single national picture. If not in either case, please specify the reasons why."
Out of 35 police forces responding to the request, 20 sent us exactly the same warning as the Met Police and North Yorkshire would have sent to the Sun. A further 6 said the figures could be used for comparison purposes, but only if other forces used the same definitions [of nationality, for instance] in their records.
The map on the right shows which forces had cautioned us about how we could use the statistics (the larger warning sign are for those forces sending us the quote above, while the smaller signs represents those forces giving more bespoke guidance).
That said, there isn't perfect consistency. One force simply said the information was there for us to use, and another said any reply to our enquiry was suitable for release into the public domain.
However, given that the police forces themselves don't seem to agree over the limitations of this data, we need to be a little cautious about how we treat stories such as these. It's difficult to see how simply summing up the numbers across every force in the country can produce reliable estimates of crime statistics. National police recorded crime data is available from the Office for National Statistics, which controls for these variations, giving a more reliable picture. While it can't give us answers about the nationality of offenders, neither can we rely on the answers we're being given in the press.