It was impossible to imagine the aloof, dignified, powerful High Lord living as, of all things, a slave.
One of the great things about movies is that it's just that short period of time. It's a bubble. The last thing you want to know is that Elizabeth and Darcy had a fight over how to treat the servants!