Mencap and the RNID are among six disability charities backing Remploy's decision to close roughly half its factories. Remploy, which was set up in 1944 and provides specialist employment services for disabled people, plans to divert resources from the factories into finding mainstream employment for disabled people.
The GMB is campaigning against the proposal and wrote to all six charities last month requesting a meeting to discuss the closures.
Mencap and the RNID rejected the invitation. Jo Williams, chief executive of Mencap, said a meeting would "waste precious time". The RNID referred the GMB to Remploy.
A GMB spokesman said: "It's cowardly that they are refusing to justify themselves. They are saying the factories should be shut down and the money used to subsidise shelf-filling jobs at Tesco."
But Sarah Moss, deputy director of communications at the RNID, said: "We support plans to end an antiquated, inefficient system - currently subsidised to the tune of £111m a year - and help larger numbers of disabled people find more fulfilling or progressive mainstream employment."
Disability campaigning charity Radar has agreed to meet the GMB, but a date has yet to be set. Leonard Cheshire, Mind and Scope have not yet replied.
The GMB has organised a series of protests at the factories and a rally at the Labour Party conference on 24 September. It has also tabled a motion condemning the closures at next week's TUC conference.
Remploy said that about 2,270 disabled people would be affected by the changes but there would be no compulsory redundancies.