Knowing which of the methodologies to use is the first hurdle, says the chief executive of the Small Charities Coalition in an online commentary
Impact measurement is one area where there’s a danger of getting carried away and we, particularly small charities, need to keep our feet firmly on the ground if we’re not to divert valuable time and resources to chasing our tails.
As we’ve learned more about demonstrating the value of what we do, both in the sector and beyond, we’ve seen a level of complexity and sophistication introduced that belies what is at the heart of impact measurement – simply demonstrating how much of a difference we’ve made.
Impact measurement and reporting has given rise to research, debates, standards, frameworks, membership organisations and models (more than 1,000, according to the Inspiring Impact Group, which provides advice for voluntary sector organisations on impact measurement). One model of impact measurement in particular, social return on investment, is spawning a small industry of consultancies offering organisations the key to ‘selling their impact’.
Don’t get me wrong – there is much about impact measurement that can only be beneficial, particularly if an organisation gets to grips with it itself and doesn’t contract it out. This might seem the most cost-effective way to address the problem, but the greatest benefits to be had are from boards and teams rolling their sleeves up and worrying away at what changes they want to see happen and how they’re going to prove they’ve done it. It can help boards focus and allocate resources more effectively, it can ensure that teams really understand what’s happening with services and improve them, and it can lead to clearer, more transparent and accountable communication with all manner of audiences – everyone wants to see change for the better and everyone wants to see charitable funds being devoted to this change.
The fact that funders ask for demonstration of impact is a good driver, and certainly there is an appetite among even the smallest of charities to do it, and do it right. Over the four years we’ve been around we’ve seen an increased awareness and interest from our members for learning how to monitor, evaluate and demonstrate impact. When we consulted our members in the summer of last year they identified it as the third most pressing skills gap they had.
There is a wealth of good stuff out there (the NCVO, Acevo and the Charities Evaluation Service, to mention a few, have some great practical resources) to help, and our members have benefited hugely from the partnerships we have with the CES and IBM for training. Of huge value, too, is getting practical advice and guidance from someone who’s more knowledgeable and hearing what others have learned from their experiences. There are pockets of activity in the sector that are trying to get us to get our collective act together. In the short term, the Charity Finance Directors' Group' work will help with impact reporting principles and the Inspiring Impact Group’s 10-year programme is valuable, not only for recognising that learning and change on this will take time, but for pulling together some bright minds and approaches to inspire development.
But there are dangers for small and new charities. Being aware of and knowing which of the 1,000 or more methodologies out there to use and which will best suit a particular charity’s size and type of work is the first hurdle and opens up the risk of time and energy being spent on going down a blind alley. Measuring social impact is probably not the best use of time and money if you’re a rabbit sanctuary.
Similarly, funders who favour one particular methodology run the risk of forcing a charity down a path that doesn’t best suit its work or that lands it with more than one methodology to follow for different funders.
Funders also need to be clear about their expectations – a small charity might overestimate what a funder is asking for in terms of demonstrating impact and could potentially commit to more work or longitudinal studies that the funder isn’t necessarily expecting. Some examples from funders of the type of impact measurement required would help enormously.
Small charities are sometimes just too small to be able to do quantitative measures justice. And in boiling down evidence of change to a handful of quantitative (quite often financial) measures, which many impact methodologies favour, a wealth of evidence of transformation in people’s lives can be lost. It can all be a bit clinical and miss out on the story that can be told about individuals. Qualitative information can be of much greater value than 'just' a case study. Good funders recognise and value this.
So whether it’s possible for impact measurement to develop in a way that doesn’t result in small charities drowning in terminology and methodology that’s ill fitted to what they do remains to be seen. Hopefully, we can keep it simple – just principles that give some structure and consistency to what is addressed but with flexibility to accommodate difference in the how. More hand holding along the way and learning what works and what doesn’t from others, but generally not getting carried away.
Cath Lee is chief executive of the Small Charities Coalition