Systematic reviews can sometimes yield disappointing results

Significant investment and resources could help to improve such situations, writes John Mohan

John Mohan
John Mohan

In her Third Sector column, Caroline Fiennes recently called for greater use by the sector of systematic reviews of literature. A recent review in the online journal BMC Public Health of what's known about volunteering and public health demonstrates the advantages and challenges of this.

The researchers sought to find "experimental and cohort studies comparing the physical and mental health outcomes and mortality of a volunteering group to a non-volunteering group". Experimental studies assign people randomly to one group that receives a "treatment" and one that does not; cohort studies track groups of individuals over time. Few studies they found met these demanding standards.

Initial searches in this case yielded more than 9,600 potential sources, the abstracts of all of which were read by two separate researchers. Only 40 were deemed worthy of detailed scrutiny, but the process gives confidence that what has survived represents genuine attempts to answer the question.

The evidence pointed in contradictory directions and the authors sought to tease these out. They concluded that findings were inconsistent, with "limited robustly designed research" and not much that could "guide the development of volunteering as a public health intervention". They acknowledged the inherent challenges: volunteering was "essentially incompatible with the notion of a randomised intervention and evaluation".

These messages will not find universal approval in the sector, and that's why we need independent work, on the lines pioneered by the Cochrane collaboration in medical decision-making. And systematic literature reviews require significant resources. Is this another case of an academic saying that research demonstrates the need for more research? No.

Done properly and independently, an agreed programme of systematic reviews could prevent wasteful investment in socio-economic reinvention and duplication. Many people will endorse that, but putting together the necessary resources will require significant investment and collaboration.

Before commenting please read our rules for commenting on articles.

If you see a comment you find offensive, you can flag it as inappropriate. In the top right-hand corner of an individual comment, you will see 'flag as inappropriate'. Clicking this prompts us to review the comment. For further information see our rules for commenting on articles.

comments powered by Disqus