A survey of 1,000 people by nfpSynergy asked respondents to rate how wasteful or economical different activities were.
When asked what salary would be appropriate for a chief executive of a charity with a turnover of more than £10m, on average the respondents felt £46,000 was about right.
In fact, chief executives heading charities with incomes of £10m-£25m earn an average of £71,000.
Only 16 per cent of respondents said they believed it acceptable for chief executives to be paid £60,000 or more, while 83 per cent thought that figure was wasteful.
Joe Saxton, co founder of nfpSynergy, said: "We need to explain that these salaries are justified because they are necessary to attract the right calibre of people."
Stephen Bubb, chief executive of the chief executives' body Acevo, criticised the findings. "If you ask silly questions, you will get silly answers," he said. "If you talked to a room full of people about what charities do, they wouldn't say this."
Bubb agreed, however, that the public needed to be better informed about the nature of charities. "There are too many negative stereotypes about charities, but a public education campaign would not really be something for us," he said.
The survey also showed that the public think rebranding and glossy reports are wasteful, but that lobbying government and holding celebrity events are good uses of money.
A new organisation, the ImpACT coalition, was set up nearly a year ago to address public concerns about charities, including the level of chief executives' pay, and to improve transparency and public confidence in the sector.
More than 70 charities and trade bodies, including the top 20 fundraising charities in the UK, have joined ImpACT so far.