Often people are being discriminated against on the basis of some ethnic, racial or gender characteristic because it serves as a class signal. Thus in the United States the slaves were black. And in many jurisdictions being black meant that you were ipso facto a slave. Thus being black was a class signal. Being a slave is a class reality. Slaves were members of society. This is often forgotten. Being a slave in the South meant that you held an identifiable place in Southern society. And even within slave society there were class divisions between the house slaves and the field hands.
When the slaves were emancipated, their social class did not change. They remained exploited labour. In the much more efficient capitalist labour market, the master did not have to buy the whole slave, he could just buy their labour. And if they starved that wasn't his fault.
Freedom opened up the American Dream. Any boy born in America could grow up to be President. And in due course a black man was elected President of the United States. But as President he did not consider it his responsibility to free the slaves. The American prison system is a slave labour system. There are more slaves in America today than there were at the time of emancipation. Because a black man was elected President that proved the virtue of the class system. People at the bottom were there because that was where they deserved to be. The slaves were slaves because they deserved to be slaves. It's a meritocracy.
The problem in discrimination is that people are perceived and treated as members of the under class, when they are not. The misperception is a problem, the mistreatment is a problem, but the real problem is that there is an under class to be identified with, and that membership justifies the bad treatment because the meritocracy has proven they are inferior.