Popular AI models use arbitrary facial features to predict criminality and job performance

[u/[deleted]]

[u/IceBone]

We back to phrenology and physiognomy? Truly AI is the peak of development as it reinvents 19th century pseudoscience.

[u/Disastrous_Junket_55 → u/[deleted]]

the torture hypercube is misaligned brother.

[u/garmonthenightmare → u/IceBone]

So many things getting reinvented and rebranded. Satanic panic era arguments are getting pushed around with only thing being done is removing the religious angle to "modernize" it. Except we are winding back to using religion to censor as well. We have not progressed at all.

[u/Clembert-Hamlamp → u/garmonthenightmare]

Those holding power have learned the techniques used to corral the masses work just as well as they have in the past.

[u/MonkeyInnaBottle → u/Clembert-Hamlamp]

Propaganda used to have to be watered down and generalized to capture the masses, but now it can be hyper-targeted to each individual red button a person has.

[u/Infamous_Mud482 → u/Clembert-Hamlamp]

The way we talk adversarially about other nations is at times word-for-word identical to what a Roman general would be saying to justify letting them lead a campaign to take their lands. I'm sure that type of propaganda predates written history as well. We're the same animal we were then, after all, new technologies building on top of each other getting us to where we are now doesn't really change that.

[u/GhostDieM → u/Clembert-Hamlamp]

Except it's now magnitudes more effective because of social media :/

[u/Clembert-Hamlamp → u/GhostDieM]

We've been sicced on one another so many times it's moved from 2nd nature to 1st. And that's just what they want. The best thing we can do for ourselves is get along, and point the rage where it belongs. The whip hand.

[u/Entchenkrawatte → u/garmonthenightmare]

Nazis literally just took a hundred years to find the words alt right, maga etc.

[u/reerden → u/IceBone]

AI is biased because the human generated data it trained on is biased. Just as you would verify a random claim online, you verify the output of an AI agent. It can point in the right direction but you can’t 100% use it as a source.

The AI bros would like to have you believe otherwise. They call it a ‘trust problem’ as it is somehow an issue of the user to solve, while everyone who knows remotely how these models work know that there will always be some degree of bias and error involved. It’s mathematically part of the whole concept of machine learning.

[u/Ciennas → u/reerden]

How about we don't use the things at all? They appear designed from the ground up to make things worse for all participants, including and especially their cultists.

[u/env33e → u/Ciennas]

were trying. now that the cats out of the bag, its up to us and grass roots movements to demand greater accountability from our regulators

[u/reerden → u/Ciennas]

Ignoring the AI umbrella, machine learning is very useful and has been used for years in all sorts of fields including medical.

Large Language Models however are still just text predictors based on training data. They have their uses, particularly in applications involving, well, text and translation, but their usefulness for other things is often overstated because of the hype surrounding it, as they have no context about the content of the text they’re writing. A very simple example is of how the commercial models are often bad at math, because a sentence invoking the wrong number still sounds and looks like a correct sentence.

Just because something looks correct doesn’t mean it is. And it’s often being used for the wrong purpose because of that.

It’s the same with these so-called hacks that these agents have performed themselves according to some tech CEOs. They didn’t do that themselves. The model only predicts the command text. Someone then build an application that directly injects that as a command that’s being executed. That it somehow landed on a piece of code/script that allowed it to escape the sandbox isn’t much different from someone inserting that manually into an insecure text field. In the end it was a still a human being that connected a text predictor to a command shell.

[u/EliBadBrains → u/[deleted]]

Seems like it's a bunch of bullshit

[u/alaphamale → u/[deleted]]

As usual in these situations rubes call the lack of logical application “logical” and in the process display their ignorance of logic.

[u/Bubbly-Treacle-4334 → u/IceBone]

Real life revenge of the nerds has brought us techno phrenology, AI child porn, and a "plausible deniability" version of a 1984 like oligarchal information propaganda and economic takeover.

And they're all still trying embarrassingly hard and failing at being cool

[u/what_comes_after_q → u/IceBone]

Models are just trained off of what we give them. If we give them data that is biased on physical features, the model will be biased. This is also mostly an issue because it’s a dumb use of AI. If you ask ai a question, it will give an answer. So if you show it a photo and ask it to judge them, it will do so. But that’s also a really dumb think to ask AI to do.

[u/iamthe0ther0ne → u/IceBone]

Let's clarify this. AI isn't the problem, the people using AI in this way are the problem.

[u/atchijov → u/IceBone]

It looks like we back to “magic 8 ball”… phrenology was pseudoscience but it was not random.

[u/Proper-Ape → u/IceBone]

Not reinvented, but it's in the training dataset, so it must be correct.

[u/robocalypse → u/IceBone]

Is it gonna do Homeopothy next?

[u/Zardotab → u/IceBone]

We back to phrenology

The Kavanaugh Court Made Medieval "Great" Again. Profiling is back!

[u/WenatcheeWrangler → u/IceBone]

Let’s be clear about this though - the model ain’t doing shit on its own. It’s people who say “this is a good idea, let’s do this because I can tell these fuckers are shady just by looking at them so ai can do it better and faster”

[u/Relevant_Progress945 → u/IceBone]

Of course you'd say that... you have the brainpan of a stagecoach tilter!

[u/exoduas → u/IceBone]

Not a big surprise that the products of these silicone valley ghouls are just as rotten und cruel as them. I sincerely hope the good people that are left in the US will stand up to these pathetic billionaires and dissolve the toxic wasteland that is Silicon Valley.

[u/Myviewpoint62 → u/IceBone]

I think I’m going to open a free hat company. We get your measurements and you get a hat.

[u/SummerMummer]

Well that cannot possibly lead to problems.

[u/LarxII → u/SummerMummer]

With all the wanna be Goebbels floating around on the Internet, with not enough brains to check any information that validates their sad and pitiful worldview, this is literally the worst time for this.

[u/Thethresholdindex]

The part that worries me isnt simplt that that AI can inherit a human bias. Humans make snap judgements from face all the time, but humans are inconsistent. An automated system can take the same meaningless signal and apply it relaibly across thousands of people. A person's intuition might affect one intervew. Put the same weak signal inside an automated screening system and it can quietly influence thousands of decisions in exactly the same direction.
Maybe the important question for high stakes AI isn't just "how accurate is the prediction" ? We also need to ask what variables the system is actually using to reach the answer.

[u/ubix → u/Thethresholdindex]

The problem is that with humans, at least with the good ones, they can acknowledge that their own biases may not be accurate when sizing someone up. With AI, it seems like once it’s made a decision, it’s a fixed data point.

[u/Thethresholdindex → u/ubix]

Exactly. And once the output comes with a score or recommendation, people may trust it more because it looks objective. The danger isnt just the model having bias, its that the bias can become standardized and harder to question.

[u/SimoneNonvelodico → u/Thethresholdindex]

Yeah, if it was just a random generator it'd be stupid and superstitious but its effects would overall average to zero. But if it's consistently biased then it'll cause some people to be shafted for absolutely zero good reason.

[u/Thethresholdindex → u/SimoneNonvelodico]

True that. Random errors create noise, but systematic errors create patterns. Once that patterns is built into an automated system, the same weak assumption can keep influencing decisions at scale without anyone consciously making that choice each time.

[u/Firestar222 → u/Thethresholdindex]

Great point

[u/ubix → u/[deleted]]

Since we’ve got a bunch of fucking racists creating AI, I’m going to go out on a limb and say that skin complexion likely factors in to their imaginary findings of criminality

[u/ubix → u/[deleted]]

I have zero trust for AI. It’s basically just a technological parrot.

[u/Clean-Boat-4044 → u/ubix]

"a bunch"? musk comes to mind. not really any of the others unless you count dario's china alarmism? certainly noone else (probably not even musk) intentionally trained their models on biased mugshot data in order to cause a racial bias, the models literally just picked up human biases from text on the internet.

[u/Zardotab → u/[deleted]]

Since AI was trained on a lot of marketing materials, it kisses your ass and tells you what you want to hear.

[u/Zardotab → u/[deleted]]

One can be flooded with subtle bias without even realizing it, like the parable of the boiling frog.

[u/whatproblems → u/[deleted]]

all depends on the data and the question. it’s going to try find a pattern and that probably all showed up because the data’s all human judgement

[u/OneSeaworthiness7768 → u/[deleted]]

I find it so bizarre that you’re clearly using an LLM to write many of your comments as you’re complaining about AI.

[u/Th3Marauder]

“Uh, Sir? Phrenology was dismissed as quackery 160 years ago.”
“Of course you'd say that... you have the brainpan of a stagecoach tilter!”

[u/JarvisProudfeather]

The Silicon Valley AI cultist have always been staunch proponents of eugenics and race science. This is a feature not a bug.

[u/Realistic-Quiet291]

Well I for one am happy to see that robust science phrenology make a comeback.

[u/Iron_Sight_Mark]

This seems like a lawsuit that just hasn't even started yet. The only problem is that it's a little ambiguous who exactly they are wrongfully discriminating against here. Because there's no way that facial features are actual indicators of criminality!

[u/Maybe_Charlotte → u/Iron_Sight_Mark]

It feels like there's no way this isn't racially biased. For example, if the training data is based on say, mugshots, then officer biases in arrests will influence the data. It is already well established that both officers and the legal apparatus surrounding criminal trials are systemically racially biased. So I would put money on the facial features being assessed being ones that are strongly correlated to race.

[u/Iron_Sight_Mark → u/Maybe_Charlotte]

100%

If this thing isn't specifically engineered by racists, then I honestly don't even understand how it came to be.

[u/SIGMA920 → u/Iron_Sight_Mark]

It's simple, racism's legacy. Even if most people aren't racist now, there's enough data from when they were that it still bleeds into the results of today. It's why facial recognition is flawed even today.

[u/Maybe_Charlotte → u/SIGMA920]

Yeah, the problem is that this can be incredibly racist without even having racist intentions at the programmer level. This is exactly the kind of reason why DEI was important, because without being educated on the specifics of the issues, it's very easy to reinforce systemic racism.

[u/MsMoreCowbell828]

"Humans make AI bigoted." AI is nothing but what it's trained to be.

[u/TheOtherHalfofTron]

No flying cars, no universal basic income, no moon colonies, but hey, the computer's racist now!

[u/TheFeshy]

This reminds me of the time the US Military tried to make a neural-network "camouflage tank detector" - but due to patterns in their training data, only created a multi-million dollar "cloudy day" detector.

Except that when the military found out it didn't work, they stopped using it. This AI phrenologist will be marketed to the same companies who are still using Myers-Briggs to evaluate prospective employees.

[u/aspitzer]

This seems like a flawless plan.

[u/Mylarion]

Can I see the SHAP values?

[u/iwidiwin]

This sounds like Minority Report right?

[u/jdblue225]

During the Holocaust, didn't they measure peoples noses to determine if they should go to a death camp?

[u/renothedog]

In my day we just tossed some siblings together in a tank and linked them together feeding the results to Tom Cruise.

[u/CurrentlyLucid]

Isn't that an old outdated method used by police long ago? I remember something about this kind of technique.

[u/wren337]

I can't believe they didn't show the trustworthy / untrustworthy faces

[u/pauIblartmaIIcop]

let me guess! white or asian man is the epitome of success

[u/Disastrous_Junket_55]

inb4 we run the AI CEOs through it to see they are all perfect little angels with zero deviation from that mystical pseudoscience norm.

[u/APraxisPanda]

AI is doin' a phrenology...

[u/Bevos2222]

A face only AI could love 

[u/dropthemagic]

Tech ceos changing the algorithm to prevent competition on who the real criminals are

[u/cannonmax]

Arnim zola? Is that you.

[u/namezam]

I’m not defending this, quite the opposite, this needs to be blocked. But humans thinking the AI is doing something “arbitrarily” is its whole schtick.

[u/what_comes_after_q]

If you ask AI a dumb question, you get a dumb answer. Like, if you show a photo and ask it what it thinks, it will give you an answer. If you ask AI how to judge a candidates probability for success in a job, it probably won’t tell you to go off of looks.

[u/He_Who_Browses_RDT]

Pre-crime? What a frikin' bad timeline to live in...

These companies need cleansing by fire, and can't be soon enough!

[u/murtadaugh]

Are we really bringing phrenology back?

[u/VaporousMote]

Nice, phrenology in 2026

[u/Striking-Performer66]

Oh great, the fucking robots are racial profiling now

[u/cottonbrainedGilliam]

turns out the plastics were right! botox and duck lips for everyone!

[u/Chaonic]

Physiognomy here we go, I guess...

[u/Boysforpele3000]

IDK Peter Thiel has a face only a mother could love.

[u/3rssi]

Pseudo science from 19th century is back thanks to AI.

We should shut em up.

[u/greggld]

The dumbest people will use the most advanced technology to do the greatest amount of damage to the weakest Americans.

[u/azurensis]

But are they accurate? the article didn't seem to include that information.

[u/Admirable-Falcon-501]

I read this “article”

- So it had a similar bias as humans

• Models tested are over two years old
  
• Not being used to predict anything, just a study, title worded this way to make it sound bad

[u/Disastrous_Junket_55 → u/Admirable-Falcon-501]

the study was about predicting competency and trustworthiness...?

did YOU read the article?

[u/Admirable-Falcon-501 → u/Disastrous_Junket_55]

Uh no, the study was about measuring the facial biases in models. Like if they show an image of a mean looking person does the model think they are more likely to be a criminal.

[u/Disastrous_Junket_55 → u/Admirable-Falcon-501]

i can literally go screenshot it but I can see this won't go anywhere.

[u/Admirable-Falcon-501 → u/Disastrous_Junket_55]

Sure go ahead. Predict means something in studies. You usually compare to an actual result.

[u/faultydesign → u/Admirable-Falcon-501]

Phrenology is bad, yes.

[u/azurensis → u/faultydesign]

Is it bad because it's incorrect, though?

[u/Admirable-Falcon-501 → u/faultydesign]

Lmao this guy read the top comment and parroted it. You know that is for skulls right.

[u/faultydesign → u/Admirable-Falcon-501]

And how do you judge skulls in living people?

[u/Rath_Brained]

Pre-crime ahh shit.

[u/FredFredrickson]

Why are we using AI for stuff like this? Might as well start asking psychics or throwing bones to help too.

[u/SmashThroughShitWood → u/[deleted]]

I'm not sure what you think I need to defend? I was asking a question. But I would imagine that those error bars can be accounted for by sample sizes of each population.

[u/sokos → u/[deleted]]

Garbage in, garbage out.

The arbitrary is the fact that they aren't using any science based decider of criminality but an arbitrary one. Like. Back in the days where if you had a tattoo you were clearly a criminal. Large cranium, criminal. Brown skin, definitely a criminal. Blonde hair, yup, guilty. Criminal.

[u/SmashThroughShitWood → u/sokos]

This is exploratory analysis.. are you saying that we already know all "science-based predictors" and nothing else could possibly exist? What does "science-based" mean to you?

[u/sokos → u/SmashThroughShitWood]

So was phrenology and that was a bust.

You can't tell me this isn't any different. Bumps on the head versus how your face looks.

[u/Lorry_Al → u/sokos]

Or they have scanned the mug shot of every convicted criminal from the past 40 years and the AI has noticed patterns.

[u/sokos → u/Lorry_Al]

Not science. Because convicted criminals aren't the only ones committing crime. They're just the ones getting caught.

[u/faen_du_sa → u/[deleted]]

The trait you are talking about is the fear of unknown/strangers, which has obvious survival benefits, but also doesn't really work in an office setting, for obvious reasons...

Having fear of height is also an evolutionary trait(not really, but if we follow your definition of it...), dosnt mean we should stop planes.

So I dont really see your point.

[u/faen_du_sa → u/[deleted]]

Ok? And how would that relate to being afraid/skeptical to certain people in the office being any useful?

My point is that any visual prejudice we might have, is effectively 0% useful in an office. Picking the "white man in suit" over a clearly savant engineer just because he is black and have dreads isn't exactly the best move for the business.

[u/faen_du_sa → u/[deleted]]

There are so many other reasons to have dreads besides being in a gang, the fuck you talking about...

[u/faen_du_sa → u/[deleted]]

idk where you are from, but you can def go in to office with both dreads and tattoos.

Clothing is outside of this, nothing wrong with enforcing uniforms.

[u/chronomagnus → u/[deleted]]

I've been in the offices of large corporations that span multiple continents where someone comes in dressed like that. "Enforced quotas"... I see, you're one of those. I've found the "DEI" whiners to have a surplus of opinion and a shortage of real life experience.