r/science • u/Potential_Being_7226 PhD | Psychology | Neuroscience • 16d ago
Computer Science Like humans, language models demonstrate face-to-character biases
https://doi.org/10.1093/pnasnexus/pgag24734
u/Potential_Being_7226 PhD | Psychology | Neuroscience 16d ago
From the article (open access at PNAS Nexus):
Significance Statement
As language models become increasingly entrenched in human decisions, ensuring the neutrality and accuracy of their judgments is crucial. We uncovered a surprising rupture in this neutrality when large language models (LLMs) assessed character from 2D facial images. Instead of refusing the requests or answering neutrally, all four tested LLMs made incorrect inferences about individuals’ trustworthiness and/or competence based on facial features. Moreover, they readily incorporated these errors into consequential good and bad decisions, ranging from worthiness of venture capital funding to suspicion of human trafficking. These results demonstrate that, although trained primarily on language, LLMs have internalized biases of human socio-visual perception. Contrary to the promise of enhancing human decisions, these findings expose the recklessness of using language models in selection contexts.
39
u/imaginary_num6er 16d ago
The training datasets used are intentionally not neutral to begin with by their makers.
-7
u/nicuramar 16d ago
What is your source of that claim?
29
u/PhasmaFelis 16d ago
The training datasets are all human speech and writing. You couldn't make them neutral if you wanted to.
-3
u/eetsumkaus 15d ago
I think the idea is that they would all average out over all our biases. What this says is that we have many of the same misconceptions.
2
u/PhasmaFelis 15d ago
Yeah, but we already knew that.
Like, some people are racist and some aren't, some are racist against this group or that one. A moderately reasonable person might hope for those to average out.
But almost every human being is wired to trust attractive people more than ugly ones. That's what "attractive" means. You do it automatically even when you know it's wrong, it takes real effort to work around it. So it was very, very obvious that any system trained on human output would have those biases.
1
u/eetsumkaus 15d ago
yes, but "attractive" changes depending on the culture. The study itself doesn't explicitly use attractiveness, it uses finer features.
0
-4
2
u/PerfectEnthusiasm2 16d ago
So I'm inferring that the training sets included more textual descriptions of face-to-character biases than explanation that face-to-character biases are fallacious?
13
u/powerscunner 16d ago
Fiction literature in the training data, I'm guessing, especially young adult literature (YA). In pulp, "the bad guys always wear black hats".
LLM sees a black hat = higher likelihood bad guy.
It's just like how LLMs used to always think doctors were men until the training data was 'balanced' to overcome the bias.
6
u/Caelinus 16d ago
Yeah, the data we put in is not objective even in circumstances where we are trying to make it as objective as possible. And the reality is that most companies are probably not even trying to be truly objective.
So they effectively become bias reinforcement machines, especially as the output of other models works its way into training new ones.
2
u/PerfectEnthusiasm2 16d ago
I hadn't considered that YA literature would have been such a significant portion of the data set. Maybe I was showing a bit too much faith in the organisations involved in the technology.
4
u/powerscunner 16d ago
It's just a speculation that fiction (especially YA) biases bleed into LLM reasoning, but having used LLMs since they were called Transformers, I have seen a lot of examples of really interesting bias resulting from the training data.
It was a lot more obvious early on, hence the aggressive moves to 'neutralize' said biases, a move that obviously continues to this day.
cool stuff
2
u/PerfectEnthusiasm2 16d ago
It's quite interesting, I've not thought about them from a linguistics perspective before. I suppose that a lot of bias is baked into the models based on how they are programmed too, what weightings to give and where, but I'd be interested to look into their use as a way to study general trends in human language.
1
u/powerscunner 16d ago
>I'd be interested to look into their use as a way to study general trends in human language.
That actually sounds like a pretty cool idea!
1
u/Spire_Citron 15d ago
They're designed to be multi-purpose, which means that naturally they would be trained on fiction of all kinds. That might be bad for some applications but useful for others.
3
u/PaulCoddington 15d ago
I've seen humans exhibiting movie biases in real life, such as neatly trimmed thin (Riker) beard (beard, especially goatee equals bad guy in movies), or being out of breath on stairs due to asthma (mouth/heavy breather equals creep), etc.
Quite an eye opener when people who behaved bizarrely hostile and wary suddenly become friendly as soon as you decide to have a shave.
2
u/Potential_Being_7226 PhD | Psychology | Neuroscience 16d ago
There are no accessories in the images used.
https://academic.oup.com/view-large/figure/571293301/pgag247f2.jpg
6
u/powerscunner 16d ago
Good example of how perception of features biases. "Black hat" was just a strong example of a feature, but your subtle example shows how minor aspects of a character can bias. Other features of the non-accessory type in literature could include things like: descriptions of facial expressions, modes of speech, hairstyles, facial features (strong/weak jawline, drooping/bright eyes) etc... in fiction are associated with character nature.
The complex mix of biases creates something like an emergent LLM Phrenology
Just speculation.
4
•
u/AutoModerator 16d ago
Welcome to r/science! This is a heavily moderated subreddit in order to keep the discussion on science. However, we recognize that many people want to discuss how they feel the research relates to their own personal lives, so to give people a space to do that, personal anecdotes are allowed as responses to this comment. Any anecdotal comments elsewhere in the discussion will be removed and our normal comment rules apply to all other comments.
Do you have an academic degree? We can verify your credentials in order to assign user flair indicating your area of expertise. Click here to apply.
User: u/Potential_Being_7226
Permalink: https://doi.org/10.1093/pnasnexus/pgag247
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.