A small minority of Pakistani men see white women as "third-class citizens" and "fair game", according to Baroness Warsi, the Coalition's only Muslim minister.
The Conservative Party co-chairman spoke frankly about the problem after nine Muslim men were last week found guilty of grooming young white girls for sex.
Police played down any racial element in the case, but Lady Warsi made it clear that she thinks it is a factor.
"There is a small minority of Pakistani men who believe that white girls are fair game," she said in a newspaper interview. "And we have to be prepared to say that. You can only start solving a problem if you acknowledge it first.
"This small minority who see women as second class citizens, and white women probably as third class citizens, are to be spoken out against."
Lady Warsi urged Muslim leaders to get to grips with the issue and make sure any men who do not respect women are not tolerated.
"In mosque after mosque, this should be raised as an issue so that anybody remotely involved should start to feel that the community is turning on them," she said. "Communities have a responsibility to stand up and say, 'This is wrong, this will not be tolerated'.
"Cultural sensitivity should never be a bar to applying the law."
Lady Warsi is Britain's most senior Muslim politician. She had decided to speak out on the controversial issue after her father told her she should be "out there condemning it as loudly as you could".
Her remarks closely echo those of Jack Straw, the Labour MP and former Home Secretary, who was criticised for saying some UK Pakistani men see white girls as "easy meat".
The Blackburn MP spoke out last year after two Asian men who abused girls in Derby were given indeterminate jail terms.
He said there was a "specific problem" in some areas where a minority of Pakistani men "target vulnerable white girls".
Following his comments, Keith Vaz, the Labour MP and chairman of the Home Affairs Committee, said it was wrong to stereotype a whole community.