The latest independent evaluation of the government-backed National Citizen Service, which is delivered by a range of organisations including many charities, is a seminar in positivity.
The findings from researchers at Ipsos Mori reveal a highly popular youth scheme – no less than 97 per cent of participants who took part in 2013 would recommend the programme. It also shows the programme helped young people to improve a range of skills.
Given that each of the 40,000 places cost the public purse around £1,600, it is good to hear that the young people had a good time and felt that they learned something new. But equally it leaves one to wonder how the young people would have responded if asked the question "would you have preferred to have £1,600 paid into your bank account rather than attend the NCS scheme?"
The latest study does little to address the broader question of what it is so unique about the programme that it justified £62m of Cabinet Office funding in 2013. The study, for example, leaves us none the wiser about whether young people who have been through the programme actually go on to volunteer more than their peers, rather than just say they will. Sadly, because the study only researches those who have recently completed the programme, these bigger questions remain unanswered.
The researchers say in the report that a longitudinal study will be undertaken to determine the longer term impact on those who took part in 2013. But given that 2013 was the third year that the programme had been funded by central government, it begs the question why the latest evaluation did not include a report looking back at the long-term impact on those who took part in 2011 and 2012? A previous Cabinet Office-funded study conducted by NatCen into the 2011 and 2012 pilot programmes did start a longitudinal study, which found, significantly, that while the NCS scheme had led to improvements after a year, some of these appeared to be short term. It also concluded that taking part in NCS made little difference to the amount of time they volunteered.
A spokeswoman for the Cabinet Office tells me in a statement that the previous longitudinal study was not continued because "follow-ups with small data sets and control groups become more difficult over time due to the fact that people drop out of the sample group".
She says that the latest evaluation deals with perceptions of the programme and its impact, and will include a long-term study. "This approach rates highly against the Nesta Standards of Evidence," she says.
This year’s study also shows that almost 40,000 young people went through the programme in 2013, but there is no mention of the fact that the government had initially said it hoped that around 50,000 would take part or which parts of the country struggled to fill their places – omissions that seem to imply that there was little place for negativity in the study.
On this point, the Cabinet Office spokeswoman says that the latest evaluation "is not designed to test overall NCS performance against targets".
The current government wants 90,000 young people to take part in NCS in 2014. Based on this year’s placement costs, it would cost the Cabinet Office in the region of £144m, although theoretically the scheme might be pulled if there’s a change of government.
No doubt the NCS programme provides a huge amount of value and makes a lasting difference to some participants, but it’s difficult to conclude that from the latest evaluation.
It will be interesting to see which of the political parties believe it warrants the levels of investment currently promised, given the evidence.
This article was first published on the Third Sector blog