How do you know if a survey is accurate?

22 March 2012

Flick through your newspaper of choice on any given day of the week and chances are that you might happen upon a story generated by a survey.

Typically, these stories will claim to have discovered something about an aspect of society that lies within the area of interest of the organisation commissioning the research.

Full Fact has been looking into a couple of surveys of this ilk this month. The first appeared at the beginning of March, and suggested that 'yob culture' cost British businesses £9.8 billion, based upon a survey of business owners by One Poll.

Back in January we also came across reports suggesting that Cardiff was the unhappiest city in the UK. This finding came from a study commissioned by guidebook publishers Lonely Planet

So how do we know if these findings tell us anything useful about the happiness and wealth of the UK?

The answer, of course, lies in the data that each survey unearthed.

However when we asked each polling company to provide us with the tables detailing their findings, we were told that it wasn't available without the client's consent.

This seemed contrary to the Market Research Society's Code of Conduct, which says:

According to clause B.50, MRS members:

"must comply with reasonable requests to make available to anyone the technical information necessary to assess the validity of any published findings from a research project."

We therefore got in touch with the MRS, who confirmed that this principle applied in these cases. While Opinion Matters have now provided us with the relevant data, we've also asked the MRS to clarify and restate that survey data should be available for the public to assess the research behind publicly made claims.

So next time that you wonder about a story written on the back of a survey, remember that you can ask any MRS member for a closer look at the relevant information.

Full Fact fights bad information

Bad information ruins lives. It promotes hate, damages people’s health, and hurts democracy. You deserve better.