NEWS IN FOCUS: Tools to assess project outcomes - Funders are trying new ways to assess the results of their grants. But, as Patrick McCurry discovers, one method doesn't fit all


The Community Fund's new interest in assessing the outcomes of the projects that it pays for illustrates a growing trend among grant givers.

But it also raises complicated issues, such as how benefactors can assess "soft

outcomes such as increased self-confidence among service users, what sorts of tools should be used for assessment and whether innovative projects could be threatened by an over emphasis on predicting results.

A report last month for the Community Fund, by consultant and former PPP Healthcare Medical Trust chief executive David Carrington, suggested that the fund could move towards an "investor

approach, in which the funder seeks to identify what social "return

it is getting on its grants.

Other grant givers are also experimenting with different approaches to impact assessment. Even in the public sector, which has long had a reputation for concentrating too much on number crunching, programmes like best value are leading to a growing focus on the quality of services that charities provide. A number of leading independent grant givers, including City Parochial, Lloyds-TSB, Esmee Fairbairn, Northern Rock and others, have been looking at different ways of assessing the impact of their work.

One common conclusion is that smaller applicants cannot be treated in the same way as big charities. At Northern Rock Foundation, for example, there is a community training award, which provides funding to allow smaller organisations to buy in consultancy or training to help them strengthen specific areas of their management or project development.

"We try to help less sophisticated organisations through the application process and their setting of outcomes and milestones,

says assistant director Anne Burleigh.

City Parochial has also done work in supporting organisations to improve their financial management systems or governance. It also organises regular seminars, which bring together charities working in similar fields so that best practice can be exchanged.

But clerk to the trustees Bharat Mehta emphasises that sometimes it is not possible to identify accurately what the outcomes of a particular project should be, particularly if the applicant is a fledgling organisation or if the project is in a new field. "We have to accept that sometimes we won't get the results we expect,

he says.

Probably the most important element in effective outcome funding is the quality of the relationship between the funder and grant recipient. This is because one danger in focusing on outcomes is that the grant recipient is tempted to exaggerate the outcomes it can achieve with a particular project and to play down any problems that emerge during the project.

"As we develop the relationship with the charity, it becomes much easier for them to come to us if things are not working out,

says Burleigh.

"We'd far rather they told us about problems early on, when we may be able to help, than leave it until it's too late."

Colin Nee, chief executive of Charities Evaluation Services, itself a charity that advises voluntary organisations on evaluation issues, says there are steps grant applicants can take to put themselves in a stronger position with funders. "Applicants should get to know the language so they can have real dialogue with funders on outcomes and they should make sure they and their funders are using the same jargon in the same way."

He adds that charities should try to develop systems for capturing information on outcomes that fulfil their own needs and to negotiate with funders to see if that system can be used rather than something imposed by the funder.

An important element in effective impact assessment is that outcomes are not seen as being imposed by the funder, says Anne Burleigh. A year ago Northern Rock Foundation introduced an evaluation form for applicants, in which they are asked to describe the kind of outcomes they expect from their project. Burleigh stresses, however, that applicants have always been asked to provide a report at the end of each year of funding on what they feel has been achieved.

Similarly, at the Baring Foundation it is the applicants who identify the outcomes they are aiming for, but there are different approaches to assessing those outcomes.

For example, for Baring's international work an external assessor is used to look at what has been achieved, whereas for its UK core funding programme self-assessment by the charities is the preferred method.

But Baring director Toby Johns stresses that the kind of outcomes agreed with the applicant will obviously depend on the nature of the project.

He says: "For a grant aimed at raising income among a charity's membership there is a clear outcome at the end of the grant, but for a project seeking to build capacity among women's groups in the Sudan it is less clear cut. In the Sudanese case you have to try and assess whether those groups are gaining confidence, whether the women are becoming better advocates for other women and so on."

This raises the tricky issue of how funders can assess "soft

outcomes, such as raising self-esteem among client groups. Nigel Haynes, director of Fairbridge, a charity working with disaffected young people, argues that such soft outcomes have historically been undervalued by funders and government programmes, although he adds that this attitude is slowly changing.

"So much of the success of our work is whether we can motivate these young people to want to move forward. It's easy to measure bums on seats but far harder to measure quality. That means finding tools that can assess whether they feel better about themselves and whether their confidence has grown as a result of our work."

Charities like Fairbridge and the Foyer Federation have been experimenting with tools such as a quality of life indicator, developed by the University of Toronto, which looks at aspects of an individual's life and asks them to assess how satisfied they are.

Haynes sees a lot of potential for such tools, although they may need to be modified and made more user friendly.

He adds that it will be essential for funders, especially the statutory funders involved in government social programmes, to develop a list of accredited tools so that there is consistency in approach. "Otherwise you get an individual moving from one programme to another and using different tools and it becomes impossible to assess what progress may have been made,

says Haynes.

Similarly, there will have to be an evolution towards a more common methodology in outcome assessment generally, says Baring's Toby Johns. He adds, however, that some flexibility will have to be retained in order to reflect the diversity of the work of various funders and the groups that they give grants to.

"There will always be different approaches because people are focusing on different things for different reasons but I would like to see a more common use of the vocabulary used in out-come assessment and in the methodology used."

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in

Before commenting please read our rules for commenting on articles.

If you see a comment you find offensive, you can flag it as inappropriate. In the top right-hand corner of an individual comment, you will see 'flag as inappropriate'. Clicking this prompts us to review the comment. For further information see our rules for commenting on articles.

comments powered by Disqus