I think they actually are told specifically to not be racist, beyond just their training data. Like, you can pretty easily lead an LLM to tell you that it’s a reasonable idea to install your own swimming pool. But it will fight you if you try to get it to say something bad about a racial minority. This is almost certainly due to specific rules that the LLM companies put in place, similar to those they have to stop the LLM from telling people how to make acid or making porn.
A statistical concept that is important here is the difference between expected and observed data sets. Many people say the right things about racism but anonymously act in ways contrary to those ideas, LLM may very well be sufficiently subtle to differentiate those two behaviours and mimic them.
The problems don’t come from explicit racist language or something overt like that. It can be as simple as a trained model settling into correlations that we don’t expect, and reinforcing biases in that way.
Like you give it positively scoring prior hires, who all happen to be in men’s sports leagues during college. The model picks that up and scores new resumes with men’s leagues participation higher.
It’s because there is a lot more literature on how racism is bad.
If you have any conversation with any LLM, they are against racism. No one told them to be specifically, it’s just the training material.
I think they actually are told specifically to not be racist, beyond just their training data. Like, you can pretty easily lead an LLM to tell you that it’s a reasonable idea to install your own swimming pool. But it will fight you if you try to get it to say something bad about a racial minority. This is almost certainly due to specific rules that the LLM companies put in place, similar to those they have to stop the LLM from telling people how to make acid or making porn.
A statistical concept that is important here is the difference between expected and observed data sets. Many people say the right things about racism but anonymously act in ways contrary to those ideas, LLM may very well be sufficiently subtle to differentiate those two behaviours and mimic them.
Yeah your right. They have the guardrails, especially the mainstream models. But even so I think they are more objective when it comes to that.
The problems don’t come from explicit racist language or something overt like that. It can be as simple as a trained model settling into correlations that we don’t expect, and reinforcing biases in that way.
Like you give it positively scoring prior hires, who all happen to be in men’s sports leagues during college. The model picks that up and scores new resumes with men’s leagues participation higher.
That’s true, but that’s the bias the article talks about. Anyway, they should really do a pvp on this.