This article has been amended; see final paragraph
Commissioning in England and Wales is failing at every level and squeezing small and medium-sized charities out of service provision, research by the Lloyds Bank Foundation has found.
The report, Commissioning in Crisis, published today, says examples of poor practice in commissioning "far outweighed the good", and calls for the government to introduce a target for public spending with small and medium-sized charities.
The report collates anonymous evidence from charities with annual incomes of between £25,000 and £1m about their experiences of competing for more than 120 different tenders from local authorities, clinical commissioning groups, police and crime commissioners and central government departments.
More than half of the charities that contributed to the report were unsuccessful in their bids and even those that secured the contracts said they had faced many difficulties in the process, the report says.
Small charities included in the report describe being excluded from bids through backroom deals with larger providers, being denied information crucial to submitting a successful bid and being told to partner with larger providers that then dropped them at the last minute, leaving them unable to find another partner.
Paul Streets, chief executive of the Lloyds Bank Foundation, said in a statement: "This report should be a real wake-up call to commissioners of local public services.
"Small charities are struggling to respond to bureaucratic, complex and inappropriate requests by commissioners.
"When it comes to commissioning services, it seems common sense has failed."
The report identifies a lack of understanding among commissioners of the needs of service users and the nature of service delivery, and a failure to consult with and learn from the experiences of local providers already delivering the services – which, the report says, leads to the commissioners choosing the cheapest provider, which may not necessarily meet the needs of service users.
It says that bid specifications might actively exclude small charities through unnecessarily large contracts, or demanding that bidders have previously delivered an even bigger contract than the one on offer or a high minimum income.
Small charities are also struggling to find the time and resources to comply with excessively demanding bidding processes to unrealistically tight timescales, against a background of shifting goalposts and a lack of communication, the report says.
This poor practice also encourages "predatory" behaviour among larger providers, further excluding small charities, the report says.
"All of the evidence shared in this report unequivocally demonstrates that the current approach to commissioning is in crisis," it says. "It is not designing or delivering effective services for all.
"Commissioners are not challenging the status quo to take on radical change. Effective collaboration is lacking. Needs and how to meet them are not understood. Resources are not being targeted effectively.
"In short, commissioning is failing at every level."
Streets said: "The report identifies that poor commissioning practices, far from being isolated issues, are widespread.
"It also reveals a system that routinely discriminates against small charities on the basis of their size, preventing them from competing fairly, leaving them deprived of funding or at the mercy of larger providers, many of which can behave unscrupulously.
"Small charities have the local knowledge and expertise needed to deliver targeted support to local people. Being failed by current commissioning processes and the public funding that follows means they are at risk of closure, with communities likely to lose critical support at a time of growing need."
The report acknowledges that commissioners are also facing challenges in the form of reduced budgets and increasing demand for services.
But to improve the situation, it recommends that commissioners increase their understanding through consultation with experts, take a proportionate approach to what they ask of services, how they are funded and how agreements are made, and place more emphasis on the social and long-term value of service on offer.
It also calls on government to introduce a target for public spending with small and medium-sized charities, in consultation with the sector.
"There needs to be a clear indication of the desired direction of travel for working with organisations with an income of under £1m," the report says.
It adds that government should further support commissioners by challenging poor commissioning and improving transparency throughout the commissioning process and delivery of contracts.
The case study about Multiple Choice initially said that there had not been a proper transition process. This sentence has been removed and a statement by Leeds City Council has been added.