Gay rights organisation Stonewall questioned if Phillips would act in the best interests of all of the communities covered by the body, which takes over from the Commission for Racial Equality, the Disability Rights Commission and the Equal Opportunities Commission.
"He has not so far delivered for lesbian and gay people, who were rarely mentioned in the Equalities Review he chaired," said Alan Wardle, director of parliamentary and public affairs at Stonewall.
Help the Aged urged Phillips to look beyond combating racism alone and make sure he worked to reduce discrimination right across the board.
Phillips was chosen from a short list of candidates, which is thought to have included Shami Chakrabarti, the director of Liberty.
A spokeswoman for Liberty said the organisation was excited about the new commission, but declined to comment on the appointment of Phillips.
The spokeswoman also refused to confirm whether Chakrabarti had applied for the position.