Department for Education cautions over own free school performance claims

In June we wrote about a press release issued by the Department for Education which claimed that:

"Free schools are twice as likely to be 'outstanding', with 21% of open free schools rated 'outstanding' compared to 10% of all schools under the same framework".

We had repeatedly asked the Department for the details behind the claim, but without reply. So we asked the UK Statistics Authority to look into it - and raised concerns about the problems with the free school data being used to make the claim.

The conclusion of the Authority was that "it is arithmetically correct to say that the proportion of free schools judged as outstanding is higher than the proportion of all schools so judged", but said there were a number of problems with the comparison.

The Department has now published a report addressing the questions we raised. It outlines a number of difficulties with the comparison and states, of the inspection figures for all schools to which the free schools were compared:

"The findings cannot be interpreted as a balanced view of the quality of education nationally"

That's because Ofsted only carries out inspections where they're likely to have most impact, so outstanding and good schools are under-represented within any one year of Ofsted inspections.

It also says that, since inspectors have only visited a minority of free schools so far,

"Caution should therefore be taken when drawing conclusions about the performance of all open free schools and when comparing free schools to other schools".

At the time of the release, Ofsted had inspected and published reports for 62 free schools, while overall around 20,000 schools are subject to Ofsted inspection.

The release says that nevertheless "Ofsted inspection grades provide a valuable source of information, in the absence of attainment data, to begin to judge the performance of free schools".

That's a more modest claim than what was said in the press release.