A report from the Office of the Deputy Prime Minister Select Committee claims that the policy can increase racial segregation instead of promoting social cohesion.
It says that grants to voluntary organisations should normally require that they work across community boundaries.
But voluntary sector leaders have said that black and minority ethnic voluntary groups could go to the wall if the recommendation is adopted by the Government.
The 'social cohesion' report from the ODPM's Select Committee considers how to avoid a repetition of the race riots in Oldham and Bradford in 2001.
It states that local authorities should develop policies which identify when it is appropriate to fund an organisation that works with just one cultural group. "Grant conditions should otherwise require that voluntary and community organisations work across community boundaries," the report states. "Every effort should be made to ensure that community centres cater for a range of cultural groups rather than separate centres being provided for different groups."
But Kevin Curley, chief executive of umbrella body the NACVS, said: "We are concerned that this could lead to a reduction in grants for BME groups.
The recommendation is ill-advised and not evidence-based. The argument is that local authority funding aggravates segregation, but the experience of councils for voluntary service is that funded BME groups are given the strength and confidence to take part in city-wide activities. It empowers the groups to get involved in the mainstreams of social and political life."
He warned: "A lot of the groups will close. How does that add to the progress of race relations?"
Maneer Afsar, director of the North West BME Network for the Voluntary and Community Sector, said the recommendation sought social cohesion but "without reference to equality ... how does this proposal seek to enforce sharing, especially when mainstream organisations have traditionally claimed to provide equality of access when there has been none?
"We need to stop local authorities using this as permission to remove funding from the very much-needed services to disadvantaged communities."
A spokeswoman from the ODPM said the department would respond to the Select Committee's report in due course.