The attainment gap in Scotland’s schools

“If you are a bright child living in a poorer area in Scotland, you are two years behind the same child living in a more affluent area.”

David Mundell, 16 February 2017

This more or less reflects the findings of a recent study by the Sutton Trust, which the Scotland Office confirmed is the source of the claim.

It’s actually referring to the family economic background of the children, rather than the area they live in, and it specifically relates to 15 year olds. Different measures may well show different results.

High-achieving well-off pupils in Scotland perform better at science, reading and maths than high-achieving poor pupils, according to the research.

It analysed data from 2015 from the PISA international survey of 15 year olds’ educational performance. The survey specifically seeks to test children’s “functional ability”, so how well they can use their science, maths, and reading skills in “real life” situations.

The gap in test scores between the two groups was translated as 31 months for science and maths, and 26 months for reading.

So poorer pupils were somewhere between two and three years ‘behind’ their better-off peers, according to the research. That’s actually a slightly smaller gap than is the case in England and compared to the average of the mainly wealthy countries covered by the PISA figures.

There are many variables in play to produce this kind of estimate. You need to choose who counts as rich and who counts as poor. You need to choose who counts as a high-achiever. You need to be happy that the test scores are an accurate reflection of ability. And you need to assume, based on those test scores, how far ‘behind’ one pupil is from another.

For instance, the measure of deprivation used in this report isn’t the same measure that’s commonly used in Scotland. It compares the 25% most deprived pupils with the 25% least deprived, based on ranking them by their background via a questionnaire. The measure used in Scotland identifies small geographic areas of deprivation based on a range of different factors like health and people’s incomes.

Another assumption is that a difference of about 30 points on PISA tests is equivalent to about a year’s attainment. That’s how we get from test scores to a claim about how far behind some children are to others. This isn’t a precise science, and previous years’ surveys actually said 40 points was equivalent to a year.

A final note of caution is with PISA figures themselves. They’re based on a sample of 15 year olds, so there’s some uncertainty in the figures. They also don’t tell us why some countries, schools or pupils perform better than others.

None of this is to say the headline claim is a bad estimate, especially if you agree with the choices the research has made on these questions. But it’s because of this uncertainty that we can’t say the findings are definitive.