Farmer this week resigned as the charity representative on the scrutiny panel set up to monitor the Work Capability Assessment, which determines whether people on incapacity benefit are fit to work.
He said his position had become untenable because the assessments were damaging the people Mind helps and ministers had failed to address his concerns.
Chris Grayling, the employment minister, wrote to Farmer two days after he resigned saying the latter’s position had become untenable because of "Mind’s potential legal action against the department in relation to the Work Capability Assessment".
Grayling’s letter, seen by Third Sector, says that, in the light of this, "it would not be appropriate for you to continue as a member of the group".
Farmer told Third Sector Mind was not planning to take any legal action against the DWP. He said the charity had merely signalled to the DWP in November last year that it wanted to see changes to the Work Capability Assessment process and, in a separate issue, had spoken favourably about a judicial review brought against the department by two individuals.
He said that this did not amount to Mind taking legal action and had not affected his decision to stand down from the scrutiny panel.
"We do not feel it is accurate to describe Mind as bringing a claim against the DWP," said Farmer.
"When the DWP raised concerns about my position on the scrutiny panel being untenable we felt that was challengeable, but decided for the reasons outlined in my resignation letter that it was time to go."
A Mind spokeswoman said some Work Capability Assessments were being rushed and the process should be slowed down or postponed.
The DWP has refused to do this, saying it would delay the assessments and leave some people on benefits for longer.
A DWP spokeswoman said: "It is an extremely complicated set of recommendations and would involve essentially redoing the whole of the Work Capability Assessment for both mental health and physical health issues. We are going to test this approach in a paper exercise to assess its impact, but it is not right to put the whole process on hold."