In TSRC we are often asked about emerging evidence on changes consequent on the "big society" agenda of the government. This presents us with a number of challenges.

 

The first is that of obtaining "real time" information. Many small-scale surveys claim to monitor the pulse of the voluntary sector. Unfortunately these are often open-access, web-based surveys. Respondents tend to be unrepresentative (not least because some will grind axes remorselessly), and results are invariably not directly comparable. And unless all surveys are conducted in a consistent manner and repeated over time, you can't tell whether they are measuring things reliably and consistently. The recent audit by Civil Exchange shows that we do have quite a big society, but inevitably most of the data on which it draws allow little to be said about change from 2010.

 

So what kind of sources are available? On organisations, TSRC has compiled from the Charity Commission register a panel of around 70,000 organisations which we can track since the mid-1990s. This can be used to look at measures of the financial performance of the charitable sector – for example, whether or not the proportion of organisations reporting an increase or decrease in income or expenditure has changed over time. We can also look at regulatory data to give us an idea of whether new organisations are being founded and whether the rate of closure of organisations is increasing or decreasing. On volunteering and charitable giving, there are reliable national surveys – the Citizenship Survey (a replacement for which is now being commissioned) and the Living Costs and Food survey - which can be used successfully to look at change over time in the proportions of people volunteering or giving money to charity.

 

There are time lags with all of these but they measure the same things in a reasonably consistent fashion. A recent report from the think tank ResPublica suggested that it is "somewhat narrow-minded to quantify healthy communities by volunteering statistics, public service delivery or economic activity alone".  I think conscientious researchers are always aware of the limitations of large-scale data sources, but if we are serious about assessing change over time then we need an appropriate and robust baseline.

 

What sort of changes are important? The annual release of volunteering statistics from the Citizenship Survey usually resulted in some fairly uncritical commentary about whether or not headline figures had gone up or down. That's a very limited view – for example, what about whether social and geographical gradients in volunteering rates were changing, and what about measures of the amount of effort people contribute? We can also do much more from the surveys about the "mixes" of activities (charitable giving, volunteering, civic engagement) that people undertake.  There's also the problem of attributing causality. It was said, for example, during the new Labour years, that public investment in volunteering had increased without any detectable increase in volunteering rates. The temporal coincidence is undeniable – but isn't it possible that the situation could have been worse without the Government’s  efforts? We don't have a counterfactual against which we can benchmark what would happen in the absence of policy.

 

A final question is about timescale – will we notice a detectable and significant change in behaviours and/or in the resources and distribution of organisations? The voluntary sector could expand rapidly simply through the transfer of public sector functions to non-profit organisations. On the other hand the number of organisations could go down if funding pressures drive organisations to merge. If we want to look at individual behaviours, then ideally we want to control for a number of characteristics – for example, when measured at the same age, are people born after the Second World War volunteering at the same rate as those born before it? To answer these questions we need large-scale longitudinal databases, not policy-making by anecdote, and we need to sustain the research effort over a number of years. The kind of changes envisaged by advocates of the “big society” may take years to be manifest and we need to ensure that we can track them over time.