The government's independent evaluation of the National Citizen Service, published last week, boasts some impressive headline findings.
For every £1 spent on the summer programme for 16-year-olds, up to £2 is generated in social return, it says. Collectively, the 8,500 young people who took part last year gave 200,000 hours of time to community projects and 77 per cent of participants said that after taking part in the programme, they were more likely to help out locally. Overall, the government predicts that benefits worth £400m could be reaped by communities as a result of spending £200m on the programme by 2014.
But dig behind the headline figures and the expected savings become less clear. Sonia Sodha, head of policy at The Social Research Unit, a charity that calls for policy-makers to invest in programmes supported by solid evidence, says that it is difficult to make claims of such savings on the basis of the latest evaluation. "The trick is doing things that will save the state and taxpayer money over the longer term, and where you can see a clear link between, say, young people on the programme and an increase in their results at school," she says. "That link hasn't been established yet."
Craig Morley, chief executive of The Challenge Network, the charity that will provide about 10,000 of the 30,000 available NCS places this summer, admits that it is hard to use the evaluation to make long-term assumptions about the programme's success.
"The short and medium-term outcomes are looking encouraging, but we really need to see how this is panning out over the long term, which will take years of study."
He adds, though, that a lot of projects would be happy if they were getting a £2 social return for every one £1 invested.
Sodha says the NCS is "a great innovation" and is encouraged that the government is researching the programme in a robust way. But she says it is difficult to justify spending such large sums on the programme by citing the latest findings: "In terms of using that research as the basis for scaling up the programme and investing significant sums of government money, we'd exercise caution. It's really difficult to say after just one year how well a programme will perform over the long term. After a one-year period, the results might be great, but that might not necessarily be the case over five years."
Last August, Vinspired, the youth volunteering charity set up by the Labour government, produced its own social impact report, based on a three-year study. It found that it generated a social return on investment of almost £6 for every £1 invested. Yet the Office for Civil Society cut its budget from £37m in 2010/11 to just £1.25m in 2011/12.
Morley will not comment on whether the NCS programme is better than other schemes available, but says that further funding needs to be allocated to other programmes: "I think that money should be invested wherever we see good quality youth and community schemes - whether that be local authority services or other types of youth schemes."
So is this a classic case of policy-based evidence - in other words, commissioning a piece of research to support a policy that has already been decided upon? Morley doesn't think so. He believes that the evidence available justifies further investment in the programme: "The short-term evidence is really powerful. What makes this scheme stand out is that we're bridging divides and bringing local young people together from a local area who wouldn't normally spend time together."