The institute returned £4m to the Office of the Third Sector when the programme ended in March because it could not spend all the money.
The organisation blamed "employer apathy", but Bill Lane, joint chief executive of payroll giving agency South West Charitable Giving, criticised the scheme.
Lane, an institute member, was particularly critical of the decision not to place a lower limit on the number of employees required to qualify for a grant or the length of time they had to donate for.
This meant sole traders could claim £300 grants in return for giving £1 for one month. "Some employers took advantage to improve their cash flows," Lane writes in Third Sector this week.
The programme, which ran between April 2004 and March this year, allowed enterprises with up to 500 staff to claim grants of £300 to £500 in order to promote payroll giving.
Tina Steele, payroll giving project manager at the institute, said 7 per cent of companies that claimed grants had only one employee.
She said: "If you have too many rules, you run the risk of people saying 'we don't want to do this'. A percentage of people will take the wrong approach, but the vast proportion won't.
"This grant payroll scheme was set in place to incentivise small to medium-sized enterprises, but a lot start at one employee. So where do you draw the line?
"It's possible to say that a small employer with one employee wouldn't have put the scheme in place unless they had taken the grant."
A Cabinet Office spokesman said: "Grants were set at appropriate levels for participants. To assess funding requirements of individuals would have led to a potentially inaccessible system."