When Matt Stevenson-Dodd, the chief executive of Street League, floated the idea of being completely open about what the charity wasn’t getting right in its annual reports, he admits the idea was met with some trepidation.
"We had a very interesting board meeting before we decided to do this," he says. "But it had started to dawn on me that you shouldn’t really be 100 per cent successful as a charity. If you are, you’re either miracle workers, you’re working with the wrong people or you’re not telling the truth."
A report, The Value of Small, recently published by the Lloyd’s Bank Foundation, says that many small charities are struggling to measure their impact and demonstrate it to potential funders. Many lack the capacity to implement the formal approaches to monitoring and evaluation that commissioners require, it says. Instead, they capture case studies reflecting how they’ve helped individual service users.
"This type of evidence is not afforded the same weight as formal quantitative output," it adds.
Stevenson-Dodd has been working to improve impact reporting at Street League for the past eight years. The charity, which helps disadvantaged young people to move into employment using sport, started by improving the data collected on the young people it works with – including the backgrounds participants come from, the types of jobs they go into and whether they are still employed after six months. But, crucially, it also finds out how many drop out along the way and why.
This data would eventually turn into a real-time impact dashboard on the charity’s website, which was launched in November 2017.
The dashboard is updated monthly, using data the charity has been using internally since 2015 after spending years ensuring it was clean and reliable. Stevenson-Dodd says the technology behind it is actually quite simple and took about three months to design and test. The data, from an Excel spreadsheet, is visualised using Microsoft Power BI, a programme that was already included in the charity’s £300-a-year Pro Office package. The team taught themselves to use it by watching YouTube tutorials.
Tracking the data has been transformational for the charity, Stevenson-Dodd says: "We knew some good stuff was happening, but we couldn’t put our finger on it. What we learned from the data was that some of the stuff we do is amazing, but some of the stuff we do isn’t working. We change our approach all the time now – it’s not a gut feeling; it’s based on reality."
The feedback so far has also been very positive, he adds, and 357 other organisations have backed Street League’s Call for Clarity campaign, which promotes transparency in the sector.
Andy Ratcliffe, the chief executive of Impetus-PEF, a funder that Street League worked with on its reporting, says organisations need to focus first on impact data that is useful internally.
"I often get organisations coming to us saying they want support demonstrating their impact – particularly to funders," he says. "I really push back on that – you measure stuff to improve impact, not to prove impact. We start with really simple questions, such as who are you trying to work with and help? Where are you trying to get them to and how?"
As a funder, what Impetus-PEF looks for is organisations with good potential, with leaders and trustees who are open to change and who are prepared to invest the time and effort needed for improving impact, Ratcliffe says.
"It’s not good enough to ‘feel’ things work if you’re not challenging yourself to actually look at who you’re working with and where they’re ending up," he says. Sometimes, he adds, the data can throw up difficult decisions – it might suggest the organisation should stop working with particular groups of beneficiaries because it isn’t best placed to help them, or that it needs to add new elements to its programmes that increase costs.
Jane Steele, director of evidence and learning at the Paul Hamlyn Foundation, agrees that the starting point for a charity of any size should be the information the organisation needs about its own work: "What do its staff need to do their jobs well? What does the board of trustees need? What does the charity need to be able to provide feedback and accountability to its own beneficiaries?"
Steele says transparency and openness have become more central to how the foundation works in recent years, but she understands why small charities might be nervous about the idea.
"It’s a slow process of trying to change the culture and the way we all work," she adds. "Funders need to lead that and help organisations gradually feel able to have those conversations. They need to feel a bit more confident that, if they do share things more, it’s not going to create problems for them."
Ratcliffe also thinks funders need to lead by asking charities the right questions and embracing flexibility.
"Rather than make the charity collect data to fit into your particular box, funders should say to the charity: ‘What is your overall purpose and how are you telling us you’re achieving it?’" he says. "On the flip side, funders should also be challenging statistics and celebrating charities that give you the real numbers."
And it seems there is an appetite among funders to do just that. Stevenson-Dodd says he called a number of them before the charity launched its first transparent annual report in 2016.
"I said: ‘We’re going to talk up front about what we haven’t got right, which is going to make the charity look like it’s not working quite as well as you might have thought it was. How do you feel about that?’" he says.
"And all of them, without exception, said: ‘This is exactly what we want to see. We want to know what difficulties you’re facing and whether we can help or not.’"