The Motor Neurone Disease Association funds research into the disease by King's College.
New Philanthropy Capital this year berated charities for their poor record on measuring performance. Paul Jump talks to three organisations that are trying to improve
* A Monitoring & Evaluation Framework
The British Heart Foundation's prevention and care division accounts for about 30 per cent of the charity's spending. It was set up to monitor specific projects, but its managers felt they lacked an objective method of comparing the charity's wide range of activities.
So the team asked Charities Evaluation Services to help it develop a bespoke monitoring and evaluation framework.
"We started by going back to basics and looking at what we do," says Nicki Cooper, programme director in the prevention and care division.
Workshops with staff, beneficiaries and healthcare professionals helped the division to break down its overall aim of playing a leading role in the fight against heart disease into three strategic aims - desired areas of change, such as improving care for people affected by heart disease. Corresponding to these aims were outcomes that projects should contribute to, such as changing people's behaviour to minimise their risk of getting heart disease.
Three strategic objectives - ways of achieving the strategic aims - were then defined. These were providing resources, funding external organisations and providing direct services.
Key aims were also developed for each of the strategic objectives, many of which - such as increasing people's knowledge of cardiovascular health - are applicable to all of the division's work. These were further boiled down into more practical key objectives, such as providing a central hub of information, and concrete outputs such as providing a heart helpline.
Further workshops generated a range of indicators for each output, relating to how well it achieved the desired outcomes.
Cooper says having a set of pre-defined indicators and outcomes that can be applied to all of the division's projects has introduced more rigour to project planning and monitoring, and saved the division's 80 staff from having to duplicate work each time they set up a new project. A full-time evaluation manager helps them think through what to monitor and how to do it, typically with a mixture of qualitative and quantitative techniques.
The division's senior managers also expect the framework, introduced in April, to allow them to rigorously rate the relative success of different projects by providing a set of defined outcomes, applicable across all the division's work.
"If one project is much more expensive than another but three times as likely to make people change their behaviour, it might be better value in terms of contributing more to our overall objectives," says Cooper.
* Focus on Key Performance Indicators
When Adrian Bagg became chief executive of disability charity the Papworth Trust in 2006, the former vice-president of oil and shipping company A.P. Moller-Maersk brought with him a wide range of experience of performance measurement in large private sector companies.
According to David Martin, director of strategy and marketing at the charity, that experience has transformed the expanding trust into an organisation that is run like a business and assesses its performance on a range of issues, including customer satisfaction and the number of people it gets into jobs according to a battery of key performance indicators.
"We decided two years ago to be more transparent about how and why we do things," says Martin. "We want to maximise the impact of every pound we spend. Our annual income is £17m, which is a lot of money. If you are going cap-in-hand to grant-making trusts and bidding for contracts, you have to be able to show how good you are at spending their money."
Martin admits that devising the right KPIs has not been easy. Eighty per cent of the original indicators, which are reassessed every year, have been altered or replaced since the programme began. "We realised there were some things we were measuring only to make us feel good or to show the trustees we had ticked a box," he says.
Any qualitative information needed for the KPIs is gathered largely from staff surveys. "We don't do external surveys because they cost money," says Martin. "We are trying to develop customer satisfaction surveys that we can use without employing an external agency."
The trust does not hire anyone to 'chase stats', but one person in the accounts team is tasked with turning the monthly figures into charts.
The figures are discussed at bimonthly board meetings and senior managers use them as the basis of quarterly business reviews with managers from each operational area to talk about what is and is not working. This, says Martin, is an inclusive, two-way process that offers staff the opportunity to explain why things have not worked.
But Martin admits the new regime has not met with universal approval among the trust's staff. "Some saw it as something that wasn't for charities," he says. "But as they have become involved, they have realised it can be quite useful. And some of our staff can be quite far from the coalface: this gives them a sense of the real impact they are having."
Motor Neurone Disease Association
* Focus on core competencies
When the MND Association first tried to improve its performance measurement, chief executive Kirstine Knox admits the charity made heavy weather of it. "We spent our time thinking about what indicators to measure," she says. "You end up counting what is easy to count. We were data-rich but wisdom-poor."
So Knox turned to the Advanced Performance Institute, better known for working with corporate giants such as Shell, BP and AstraZeneca. Bernard Marr, chief executive of the API, sat the charity's board and senior management down and asked them what outcomes they particularly wanted to achieve and which areas they needed to excel in.
"Before you measure, you have to know what to measure," Knox says. "It all seems very obvious in retrospect, but we found it challenging to think about these questions: you have to get into a different mindset."
After coming up with a list of seven core competencies - such as understanding the needs of beneficiaries, empowering staff and volunteers, and being able to work in partnerships - the charity then considered the governance, financial, human and IT 'enablers' that would allow it to achieve them.
This allowed it to draw up what Marr calls "the most useful management tool of the past 100 years": a strategic map that represents the relationships between the desired outcomes, core activities and enablers. Marr then devised a set of key performance questions to allow the charity to pinpoint where it was on the map.
Knox says the new approach has had a significant impact on the way the charity operates. A perceived need to mitigate strategic risk led to the establishment of a recession-watch panel "so we see the cliff before we fall off it". Another question about whether the charity's human and financial resources were aligned in the right way prompted a mapping of MND services in London and the allocation of more resources to gaps in provision.
Marr says charities could go through similar processes without facilitators, provided they refrain from making their top-level priorities too detailed and complex. They also need to avoid being fixated on quantitative data. "It is too easy to say it is too complex to measure charities' performance because you don't have nicely measurable outcomes such as shareholder value," he says. "Qualitative data such as surveys can be very helpful, but a monthly survey of a sub-set of employees, volunteers or beneficiaries might be much more useful than the annual survey many organisations do."
