With the Commons vote on the controversial Academies Bill due this evening, the Sutton Trust has fanned the flames of the school funding debate by suggesting that richer independent schools give a smaller proportion of their income to bursaries for poorer students than less well-off institutions.

However the findings of this morning’s report – Fee Remission and Bursaries in Independent Schools – have not been accepted by everyone.

The claim:

The Independent Schools Council (ISC), which represents England’s 1,260 private education establishments, has questioned the factual basis of the Sutton Trust's claims.

Rudolf Eliott Lockhart, Head of Research at the ISC, argued in a press release that the report was predicated on “out-of-date, incomplete and mismatched data.”

Having already highlighted problems in a Centre for Policy Studies report last week, Full Fact decided to investigate this latest contentious publication.

Analysis:

The Sutton Trust’s report, which found that those private schools in The Times' top 70 of high-earning establishments spent 4.3 per cent of their income on bursaries for poorer stundents, compared to 7.2 per cent by those ranked 211 to 280, was compiled by the Institute for Education Policy Research at Staffordshire University.

According to the ISC however, it draws these conclusions from data pertaining to 331 schools in 2007/08. This, it claims, compares unfavourably to its own research covering 447 schools in 2009/10.

Out-of-date?

We contacted the report’s author Peter Davies to ask whether or not this indicated a weakness in its method. He told us that as his team had used audited financial reports lodged by independent schools with the Charities Commission, there was necessarily a delay involved.

“These reports have to be compiled by the school for the previous year, audited by accountants and then published by the Charities Commission,” he said. “We then spent months analysing the data for our report – it is a lengthy process.”

Nevertheless, Mr Davies asserted that his report’s findings remain relevant as there had been little change amongst the top independent schools offering bursaries to poorer students.

He pointed to research conducted by accountants Howarth Clarke Whitehill which he claimed “doesn’t show a dramatic change in the number of schools offering bursaries rather than scholarships.”

Unfortunately for Professor Davies this isn’t an opinion shared by Howarth Clarke Whitehill’s Tim Baines, who compiles the research he mentions.

He told Full Fact that “the most recent survey shows a continuing very strong move from scholarships to bursaries across the board.”

He added: “One could surmise that, since the larger/top schools are traditionally the ones with the largest scholarship funds, that the switch was likely to be greatest in those, and this would accord with my own general view that the top schools are switching the most.”

Incomplete and mismatched?

The second premise of the ISC’s critique rests upon the assertion that the data sets used by Peter Davies and his team were incomplete and mismatched.

The ISC argues that whilst scholarship data was drawn from 187 schools in the Sutton Trust report, only 147 were used to provide information on bursary awards. The Council claims that by using a larger sample, it’s own findings were more reliable.

However this conclusion is not as easy to reach as the ISC might like.

As Professor Davies pointed out, the size of the sample isn’t necessarily the most important indicator of its reliability.

He told us: “We didn’t include schools that didn’t publish data on the amount spent on scholarships and bursaries because we didn’t want a self-selecting sample.”

“100 or so schools don’t provide such a breakdown in their accounts, including some of the richest such as Eton. We were of the opinion that it was implausible that they hadn’t considered how favourably such information might reflect on themselves,” he explained, adding: “If the ISC have more complete information they should publish it, but so far they haven’t. We thought it best to be transparent.”

Mr Davies is supported in this belief by the survey watchdog the Market Research Society.

A spokesperson told us that “generally the strength of a survey depends more upon its being properly weighted to be representative of its subject, rather than its sample size.”

This is a point conceded by Tim Baine, who notes that whilst it does make the survey more “reliable”, an issue remains as to whether or not the financial reports “contain enough information to enable any conclusion to be drawn.”

Conclusion:

The ISC's own research into the funding of bursaries and scholarships in private schools is a valuable contribution to the present debate, and its findings make an interesting counterpoint to the conclusions drawn in the Sutton Trust report.

That said, Full Fact does not believe that Professor Davies' findings can be completely undermined on the basis of data not made available to his study. Without access to the relevant fieldwork, the ISC's survey is not directly comparable to the data compiled by the Sutton Trust, and any relative value judgements made on either study are difficult to independently sustain.

However question marks about the relevance of data that is over two years old in the Sutton Trust report have not been fully exorcised, meaning that elements of the ISC's critique should be taken on board by the Sutton Trust.

Full Fact has raised these issues with the ISC, and their response has been published on the Full Fact blog.