"The fact of the matter is most people in Britain are against fox hunting, and that includes most Conservative voters," said a spokeswoman for the International Fund for Animal Welfare.
David Maclean, Conservative MP for Penrith and Border, sent a letter to all Conservative MPs on Cameron's behalf last week, saying he would repeal the act if he won the next General Election.
But campaigners are confident any attempt to reverse the ban would be met with strong opposition.
Ifaw, the RSPCA and the League Against Cruel Sports commissioned an Ipsos Mori poll to mark the second anniversary of the ban this month. It shows that nearly 45 per cent of Tory voters - the group that was traditionally most likely to oppose it - support a continuation of the ban. This compares with 30 per cent who favoured scrapping the ban.