April 17, 2012 • 12:52 pm

“A survey released by the Charities Aid Foundation revealed that nearly nine out of 10 charity executives believe the plans will severely hit gifts from major donors.”

The Guardian, 12 April 2012

Last week the Guardian, Daily MailHuffington PostMetro and City A.M. all reported that 9 out of 10 charity executives believed that the Government’s proposals to cap tax relief on would have a negative impact on charities. This was based on a survey conducted by the Charities Aid Foundation (CAF).

But how accurate are these figures and are they representative of the charity sector as a whole?

Analysis

A summary of the CAF’s findings can be found in a CAF press release released on 12 April. The figure of 9 out of 10 can be seen at the very top of the release. The CAF was kind enough to supply us with the details of this survey and they wanted us to point out that the survey was conducted in a relatively short time, from 4-11 April.

The sample

12,340 charities registered with the Charity Commission were invited to take part in the survey, of whom 120 responded – a response rate of around one per cent.

The CAF’s headline “nine out of 10″ claim refers veriously to “top charity executives” “chief executives” and “senior executives”. All major new outlets made the same claim based on the CAF press release. However, looking at the last question, this doesn’t seem to accurately represent the participants of the survey.

65.8 per cent of the respondents said they were at “Executive Level”. The rest were senior managers, mangers and ‘other’. So to imply these are all “top execs” doesn’t properly report the sample.

The questions

The 120 respondents were asked a variety of questions about the proposed cap, one of which asked them whether they agreed with a series of statements. One of these was:

“I believe this will have a negative impact on the value of donations from Major donors.”

88.2 per cent of respondents agreed with this statement and this is where the figure of 9 out of 10 comes from, which was accurately reported by the news outlets.

However the CAF press release also said:

“Charity executives also expressed deep concern about the potential impact on their own charity’s income, with half of those expecting a reduction in income predicting their major donations would fall by at least 20 per cent.”

This figure comes from respondents’ views about the effect of the cap on their own charities. 53 per cent, or just over half, did indeed predict their donations would fall by at least 20 per cent.

However this refers only to those charities that expected any reduction in their income. Out of all respondents only 42.5 per cent thought there would be a reduction while 45.8 per cent said there would be no change, and 11.7 per cent did not know.

Only 27 people, or 22.5 per cent, out of the whole sample thought there would be a reduction of donations from major donors to their own charities by at least 20 per cent.

While the press release does mention that the figure of just over half relates to those who though there would be a reduction it fails to put this in context of the survey as a whole.

 

Furthermore, the press release fails to mention that more respondents (45.8 per cent) thought the proposed cap would not change to the amount they receive from major donors.

Participant understanding

Another concern with the survey is the responses to the very first question. This asked respondents whether they agreed with the following statements:

“I am clear on the details of this cap in tax reliefs.”

“I am clear on the implications for the charity sector of this cap in tax reliefs.”

24.2 per cent disagreed with the first statement while 22.4 per cent disagreed with the second statement. This means that nearly a quarter of respondents were not clear on the details of the cap and just over a fifth were not clear on the implications for the charity sector of this cap.

Clearly then a sizable minority of respondents do not understand the proposed caps effect on charities, which potentially undermines the survey’s usefulness.

Representative sample

There are also problems with the representativeness of the sample size.  One question asks respondents what their charity’s total income is. The responses can be seen in this table:

Income Band

Charities

% of charities

£0

4

3.3

£1 – £100,000

57

47.5

£100,001 – £500,000

29

24.2

£500,001 – £1,000,000

10

8.3

£1,000,001 and above

16

13.3

Don’t know/can’t say

4

3.3

We can compare these to similar figures provided by the Charity Commission from the last quarter of 2011 – the most recent data available.

It is important to note of course that the Charity Commission data covers only charities registered with the commission. Types of charities that are not registered with the commission include those that have income less than £5,000, those with bases outside of England and Wales and ‘Excepted and Exempted’ charities which include scout groups, most English universities and national museums and galleries.

We are unsure whether the sample used by the CAF contains any charities not registered with the Commission but we can still make some attempt to compare the two.

Based on the charities register in the last quarter of December 2011 there were 161,649 charities registered with the Charity Commission. A breakdown based on their income band can be seen in this pie chart:

In the CAF survey 50.8 per cent of charities had an income of £0-100,000. According to the Charity Commission 82.17 per cent of charities had income of £0-100,000 (the blue and red segments combined). In the CAF survey 21.6 per cent had incomes over £500,000. According to the Commission, six per cent had incomes over £500,000.

Based on this comparison the survey seems to over-represent those charities with larger incomes and under-represent those with smaller incomes. 

It is worth noting though, that larger charities also have a disproportionate share of charitable income and consequently spend more.

As this pie chart shows, while charities with incomes over £5 million account for just 1.08 per cent of all charities, they do have 66.87 per cent of charitable income. Conversely charities with incomes less than £10,000 only have 0.42 per cent of all income.

Therefore, while the survey may over-represent wealthier charities in terms of number of charities, these wealthier charities have the overwhelming majority of total charitable income and spending.

It is also worth noting that many small charities are given grants by larger charities. Therefore, it is possible that donations larger than £50,000 trickle down to smaller charities via grants from larger ones.

Conclusion

The figures quoted in various news outlets are all accurate based on the Charity Aid Foundation’s press release.

However the press release itself fails to adequately explain the findings of the survey. To begin the release with “Nearly nine out of ten top charity executives” misrepresents the respondents of the sample who were not merely those at executive level.

In addition, many of the figures shown are presented without the wider context of the sample and the questions asked – such as the number of respondents that actually expected not to be hit or didn’t know, and their understanding of the implications of the Government’s policy.

This combined with the small eventual size of the survey and some possible concerns with the representativeness of the sample significantly undermine the usefulness of the survey, so we cannot be so bold as to say that nine out of 10 ‘charity executives’ are as concerned as the CAF make out.

Share this article:
By • 12:52 pm
A non-profit company (no. 6975984) limited by guarantee and registered in England and Wales. © Copyright 2010 - 2014 Full Fact