The poll, a survey of 300 finance professionals from Hays Accountancy and Finance, found that commercial accountants and finance managers associate voluntary organisations with poor salaries, bureaucracy, low status and limited career opportunities.
In contrast, the private sector was felt to offer good remuneration, variety, security and opportunity.
Eighty per cent said they wanted to stay in the commercial sector. The charity sector was preferred by just one respondent in 20, which was still more than the 2 per cent who highlighted the health service as a desirable working environment and the 3 per cent who named local government.
However, the survey offered slightly more reassurance for charity finance departments from public sector workers. Fifteen per cent would switch to the not-for-profit sector, it revealed.
Andy Robling, national director for Hays Accountancy and Finance Public Services, said: "Organisations must act to ensure would-be recruits don't rule out voluntary sector roles because of misconceptions."
Shirley Scott, chief executive of the Charity Finance Directors' Group, agreed that the sector was suffering from misrepresentation.
She said: "If the respondents have confused bureaucracy with accountability and risk management, then the sector may plead guilty. But there are career opportunities for those who are able, as in every other sector. People who work in the voluntary sector are never going to receive the highest salaries, but they do receive fair salaries and benefit from the satisfaction of knowing that they are making a difference to people's lives."
However, David Senior, managing director of recruitment consultancy Action Planning, claimed there was no shortage of well-qualified senior financial staff wanting to make the transition to the voluntary sector.
He said: "As much as half of the response for voluntary sector finance roles can come from private sector applicants. Far more finance people than, say, marketers, successfully cross the sector divide, since their roles are so readily transferable."
In the survey, 29 per cent of private sector finance personnel said they would be tempted to move to jobs that offered a better balance between work and home life.
"The voluntary sector must build on its unique selling points in order to attract and retain the cream of commercial finance expertise," said Robling.