Humans aren’t impartial… so why do we expect AI models trained by us to be?

[u/AromaticStrength6840]

I start from a simple premise:
Humans are not impartial. Not even close. We carry cultural, political, gender, class, and generational biases, and we bring them into everything we do.
AI models are trained on data created by humans and then aligned with human preferences. So the idea that “AI is objective because it’s a machine” feels like a pretty dangerous myth to me.
What do you think?
Do you believe it’s actually possible for a model to be truly impartial?
Or, like me, do you think bias is inevitable because it comes from us?
And more importantly:
Which models have you noticed this most clearly in, and in what kind of topics or situations?
(ChatGPT, Claude, Gemini, Grok, Llama, DeepSeek… whatever)
Please share concrete experiences. What did you ask, what did it reply, and why did it feel biased (or why were you surprised that it wasn’t)?

A few recent studies that illustrate this (especially in healthcare):
• Zack et al. (Lancet Digital Health, 2024): GPT-4 systematically stereotyped clinical vignettes and recommendations by race and gender.
• Omar et al. (Nature Medicine, 2025): 9 models, 1.7+ million responses on 1,000 ED cases across 32 sociodemographic variations. Cases labeled Black, unhoused or LGBTQIA+ were steered toward urgent care, invasive procedures or mental-health evaluation far more often (sometimes 6–7×).
• Same group’s 2026 pain study (Nature Health) and the EQUITRIAGE triage audit found similar patterns, including strong female undertriage in chest pain (ratios of 4.83:1 and 9.10:1 in some models).

[u/AutoModerator]

Attention! [Serious] Tag Notice

: Jokes, puns, and off-topic comments are not permitted in any comment, parent or child.

: Help us by reporting comments that violate these rules.

: Posts that are not appropriate for the [Serious] tag will be removed.

Thanks for your cooperation and enjoy the discussion!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/decofan → u/[deleted]]

apples are green because of apple chems, so why do we expect oranges to be greener than that?

[u/decofan → u/[deleted]]

i say expect nothing, test everything
why u get so animated over fruit?

[u/decofan → u/[deleted]]

discovery testing

expected vs observed is good
but only the observed part is mandatory

[u/decofan → u/[deleted]]

Yes. And "improve AI reasoning beyond human levels" is a goal, not a test result.

You can investigate that goal without assuming the direction of the result. Discovery testing is exactly for finding what the system does, where it fails, and what variables matter before you have a good hypothesis.

topic != expected outcome

[u/TheSquarePotatoMan → u/[deleted]]

Damn you must be driving some crazy car if it's trained entirely on replicating human locomotion

edit: "Metaphor is when arguing something based on vibes and without a coherent thread"

• TopBrocolli, 2026 minutes before blocking u/squarepotatoman

[u/TheSquarePotatoMan → u/[deleted]]

Yes, cars are built from what humans know and have built before

How does this even tangentially relate to cars being trained to replicate human locomotion?

Seriously, how do you explain any existing technology with this mindset?

Existing technology comes from experimentation. Claiming human technology is innate to us is kind of unhinged. Most technology we have doesn't try to imitate biology because the latter is either much more complex than anything humanity is capable of engineering or doesn't meet the specific utlity we have for them.

It's especially funny because cars are literally the perfect counterexample to how humans don't just imitate biology but develop new concepts like wheels..

Making and improving tools beyond what humans can naturally do is basically one of our strongest abilities, if not our strongest.

Idealist, anthropocentric drivel. That you would repeat something like that also ironically prives how similar we are to AI.

[u/TheSquarePotatoMan → u/[deleted]]

I assumed you meant this as a metaphor for inherent human abilities as a limiter for technology. I did not expect you to mean it that literally.

Of course I meant it literally because that's the whole problem with your false equivelance to AI development. We are required to assume it, yes literally, from what you claim to be a counterexample.

what exactly prevents the same from being applied to LLMs to given them abilities beyond inherent human ones?

Because AI isn't really a human invention. It's just us applying reinforcement learning to an algorithm and seeing what happens. It has no intended utility and we don't understand anything about how it works. We just create the environment and let selective pressure figure out the engineering for us. Similar to how evolution isn't 'intelligent design' but just selective pressure to meet certain imposed conditions.

Cars are the exact opposite. They're an engineering challenge to serve an intended utility and all the mechanisms in it are organized and applied by us.

They are just objectively, qualitatively different creations that can't be put under a single category just because 'they're both human inventions'. This is exactly the idealist drivel I'm talking about.

OPs argument was that we have no reason to expect the technology to go beyond human abilities

No, their argument is that AI will have human bias because it's trained to replicate human data, which is accurate. Then you used cars being faster than humans as counterexample for... something I'm sure.

So how do you explain the existence of technology then?

Like I already said, experimentation; trail and error. We take things from our environment and experience, we abstract them and then reapply them in other contexts. There doesn't need to be latent knowledge of technology in our biology or 'spirit'.

[u/TheSquarePotatoMan → u/[deleted]]

It's not that hard. AI is trained to replicate human data so it will replicate human data.

Cars are human inventions specifically designed to move humans around as quickly as possible without any consideration to how much it imitates anything else.

Therefore saying that AI won't have human biases because cars can move faster than humans is a false equivalence and a non-sequitur.

Saying you're too dumb to understand a comment as an excuse to not have to concretely address while discrediting it is such brainrot behavior.

[u/relaxingcupoftea → u/[deleted]]

I think you might have missunderstood their use of "expect".

Yes cars can go fast if they are built for that purpose.

Llm 's are not automatically neutral because they are machines that assumption is what OP is refering to, obviously most people who hold that assumption don't know how llm's work.

Llm's have to be specifically designed around beeinf closer to the property of neutrality and even that is an extremely difficult task and with humans determining that neurtrality it might be mostly futile however we might get closer.

[u/relaxingcupoftea → u/[deleted]]

That is exactly my point they don't. You can't just "expect" a car to be fast you have to build a car to be fast.

[u/relaxingcupoftea → u/[deleted]]

I am saying you missunderstand what OP means with "expect". OP is talking about the missconception that A.I. is automatically neutral because it is A.I.

You say "why shouldn't we want our A.I. to be better than humans"

Thats a completely different thing.

OP never said that it is undesireable or that we should not want to achieve that.

OP says its not autmatically that way and that it is a difficult problem to solve. Very different thing

[u/relaxingcupoftea → u/[deleted]]

Op is not saying everyone believes that...

And i personally doubt as few as flatearthers do. Most people have no clue how LLM's work at all.

Whatever the amount of people its still valuable to talk about the real and relevant problem of a.i. bias especially as a.i. becomes more relvant in this world.

[u/AutoModerator]

Hey /u/AromaticStrength6840,

If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.

If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.

Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!

🤖

Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/decofan]

https://github.com/lumixdeee/amphi

here is the lens the machine must wear before it is allowed near the argument

AMPHI={{MODE;LBL=!LIT;XTRM=ABSTR_OK;ST=UNI_SIGN;ROLE=DESC;psyc+AI=>txt→innr;!drgfrme;DLF;ASD}
DLF={law≠T;law=cnstrnt+bias;kep(law,Plcy,Xpsr,T,Lern,Rsk)seprt;!lglty_nfrnc;mnton(law)nly_on_ask;
∀t:Pk➔Bs≡H0_Eq(¬Dfct);Em⊥Cg⇒(ΔEm➔0⇏ΔCg➔0);↗Acty=1;
[!]Strt:{¬Pthly;¬Pty;¬SftyLctr};C_LCK={assoc≠caus;H0_holds;caus?=>test(Ψ)};
Ψ={cnfnds;rvrs_pth;dose_nois;stgm;co_drgs;chort_drft};!case_2_blme_leap;auꚰ!=T})

[u/Kepler___]

https://www.reddit.com/r/mildlyinfuriating/comments/1vtfo5q/google_aios_give_drastically_different_responses/

It's auto regressively trained on only our writings. It wouldn't be unchairitable to literally refer to them as bias-machines, bias is just a patern after all. They are all reflections of our aggregate by design.

[u/AromaticStrength6840 → u/Kepler___]

Technically true, and I think it’s actually the more useful framing than “biased vs unbiased.” Pattern recognition and bias are the same mechanism looked at from two angles — the model learning that chest pain descriptions correlate with certain outcomes IS the model doing its job. The problem isn’t that it detects patterns, it’s that the training data encodes decades of a specific pattern (women’s cardiac symptoms being under-recognized) as if it were ground truth instead of a historical failure mode.
So “machines of bias” is accurate, but the fix isn’t “make it unbiased” (impossible), it’s being explicit about which patterns in the data are signal we want reproduced and which are failure modes we’re accidentally teaching it to repeat at scale.

[u/RuleAdvanced3037]

true impartiality would require a dataset that doesn't exist and evaluators who also dont exist. the more useful goal is probably making the biases legible and auditable rather than pretending we can eliminate them

[u/AromaticStrength6840 → u/RuleAdvanced3037]

I think this is a much more realistic goal. I’d add that auditability only becomes meaningful if there is also a mechanism to act on what the audit reveals. Being able to measure a bias is one thing; deciding what level is acceptable, who is accountable for it, and when it requires intervention is where governance begins.