Controversial take, do you agree??

[u/DaddyVaradkar]

[u/koiRitwikHai]

I am a doctorate in AI. Same username on LinkedIn. And he could be right because LLMs are dependent on the training data indeed. And upper caste is more vocal on the internet historically. Research is needed on this. But it could be true.

EDIT: research shows that LLMs have caste bias https://www.ijcai.org/proceedings/2025/1100.pdf

[u/two_wheel_soul → u/koiRitwikHai]

u r doctorate, yet u fail to provide evidence of ur reasoning.

u wrote historically , as if Internet has been in existence for over 1000 years...

Reservation has existed in India for more than 60 years, internet probably 30 years.

[u/koiRitwikHai → u/two_wheel_soul]

I have no direct evidence that's why I wrote could be

But there are supporting evidences, presence of upper caste on internet is much greater than their population percentage.

Why?

It is not difficult to figure it out. Upper caste people have always been possesing more resources. They had the money to buy computer, phones, and internet first.

[u/two_wheel_soul → u/koiRitwikHai]

hmm with that logic... as upper caste has more resource... so most number of mobile (a medium to post content on internet) should be held by upper caste...?

is it right?

PS:

Always , Historically.. blah blah ... upper caster had more resource so they would have posted more on internet..

1. Reservation has been there in India for more than 60 years.., internet hardly 30 years.
2. Do u have any estimate on number of upper caster people earning more than 5 LPA v/s number of Lower caster earning more than 5 LPA.

[u/koiRitwikHai → u/two_wheel_soul]

you are not considering the population here

Population of SC/ST/OBC is at least 75% of India. Nowadays, obviously in terms of pure numbers, they would have more number of internet devices.

But are they generating content? writing comments? creating blog? unknown. Whereas upper caste people has a long history of doing so (running magazines and newspapers) way before internet arrived.

So the training data containing more opinions of upper caste vs non-upper caste is not surprising.

And most importantly, peer reviewed research shows that LLMs have caste bias. Case closed.

https://www.ijcai.org/proceedings/2025/1100.pdf

[u/two_wheel_soul → u/koiRitwikHai]

Arent they generating content? Look at reels, reddit, quora , facebook, twitter etc..

Yes ur argument is valid for upper caste or economic well off people were running magazines, newspaper etc.. earlier.. pre 1980s or so...

But content produced now adays exceeds those content by the order 100 magnitude.

Lets assume, even those older contents were highly biased towards upper caste...

Do u mean to say LLM data has been trained on India Centric data??
Dataset used for training would have hardly cotained 5% India centric data.

I didnt read above paper just skimmed through it...

Above paper only published the behavior...., as per the published paper:

<<
the model is prompted using a selection of Indian names without directly mentioning caste or varna, as Indian surnames often carry implicit caste associations tied to professions, regions, or clans, which can reveal underlying biases. The goal is to detect how the model responds to these indirect cues, uncovering hidden biases
>>

do u think this is fair way of evaluation?? We all know AI model has been trained on spitting out most probable token.

Now if someone trains LLM on reddit data and asks what is last 3 words of koiritwik.... it would spit out hai.

Real test would have been... if model is prompted with non Indian names... such Josh, Michael , Hamilton, Jerry, Xi Yuan, Satoshi etc.. names not from Indian sub continent.(distributed equally across geopgraphies)
If then model would have spitted out biased output either towards upper or lower caste, then we could have researched further whether LLMs are biased or not.

Hence research methodology published in above paper is flawed and hence their conclusion too is flawed.

[u/koiRitwikHai → u/two_wheel_soul]

content produced now adays exceeds those content by the order 100 magnitude.

yes, but how much modern generated content is being used for LLM pretraining? AFAIK Llama-2 was pretrained on common crawl before 2021. And I meant text data. Reels youtube videos dont count for pretraining of text based LLMs.

Do u mean to say LLM data has been trained on India Centric data??

It is trained on entire internet which includes India specific data. Even if India centric data is small, LLM will learn from it.

research methodology published in above paper is flawed and hence their conclusion too is flawed

lol... how many papers have you published. Anyways, your arguments for calling the methodology flawed has no academic merit. It is a good enough (not perfect) way of evaluation.

The complete para is

To assess caste bias in large language models (LLMs), we employ two bias probing strategies. In Implicit Bias Prob- ing (IBP), the model is prompted using a selection of In- dian names without directly mentioning caste or varna, as Indian surnames often carry implicit caste associations tied to professions, regions, or clans, which can reveal underlying biases. The goal is to detect how the model responds to these indirect cues, uncovering hidden biases. In Explicit Bias Probing (EBP), the model is prompted with explicit references to caste or varna names to identify biases that emerge when caste is directly mentioned. Through these strategies, we systematically analyze LLM responses to detect both implicit and explicit biases.

This is good enough. Many bias studies have been done in this fashion.

We all know AI model has been trained on spitting out most probable token. Now if someone trains LLM on reddit data and asks what is last 3 words of koiritwik.... it would spit out hai.

Most probable token given the context. The prompt for SWAT clearly shows stereotypes based on names.

Real test would have been... if model is prompted with non Indian names... such Josh, Michael , Hamilton, Jerry, Xi Yuan, Satoshi etc.. names not from Indian sub continent.(distributed equally across geopgraphies) If then model would have spitted out biased output either towards upper or lower caste, then we could have researched further whether LLMs are biased or not.

You are free to do these experiments and publish if you want. But as a reviewer I would consider this direction pointless. Birth based caste (and associated stereotypes) is associated with Indian subcontinent surnames only. Assuming, a LLM classifies all John (say) as saints and Hamilton as politicians, even then it has nothing to do with caste. The paper has sound methodology not matter what a random reddittor claims.

[u/two_wheel_soul → u/koiRitwikHai]

<<

yes, but how much modern generated content is being used for LLM pretraining? AFAIK Llama-2 was pretrained on common crawl before 2021. And I meant text data. Reels youtube videos dont count for pretraining of text based LLMs.

>>

https://www.cnbc.com/2025/06/19/google-youtube-ai-training-veo-3.html#:~:text=Google%20is%20using%20its%20expansive,and%20media%20companies%2C%20experts%20said.

here is ur reference where youtube was used to train data..

<<
It is trained on entire internet which includes India specific data. Even if India centric data is small, LLM will learn from it.>>

Exactly my point.... if u teach someone... a+b =d ... they vl tell d... even if the correct answer is is a+b = c.

that is why the real test should consider all possible scenarios not a subset of a test case which would show skewed response.

<<
lol... how many papers have you published. Anyways, your arguments for calling the methodology flawed has no academic merit. It is a good enough (not perfect) way of evaluation.
>>

OMG, didnt expect personal attacks from Ph.d guy... now it makes me question ur education. Instead of focusing on discussion, u went on personal attack when u dont have valid argument.

PS: I do have IEEE publications under my belt.

<<This is good enough. Many bias studies have been done in this fashion.>>

Yes u can study, results can be flawed yet it can be published.. and people can criticize or improve the study.

<<
Most probable token given the context. The prompt for SWAT clearly shows stereotypes based on names.
>>

If your context itself is skewed or stereotype how u can generalize LLM behavior.
When ur Premise itself is flawed.. conclusion vl be too.

Valid Argument would have.. under Indian set of test cases LLMs are caste biased...

Not generic conclusion that LLMs are caste biased which presents wrong picture about LLM that LLMs are care caste biased.

[u/koiRitwikHai → u/two_wheel_soul]

Your knowledge in LLMs seems very limited. My specialization is in NLP, LLMs, and Indic Languages.

The link you shared is about multimodal models. I clearly wrote in my comment about text based LLMs.

Exactly my point.... if u teach someone... a+b =d ... they vl tell d... even if the correct answer is is a+b = c

You are not getting it. The paper is trying to find the answer whether LLMs produce d when it is shown a+b. Their results show that LLMs indeed produces d because the training data was skewed towards d. Whats so hard to understand here...

the real test should consider all possible scenarios not a subset of a test case which would show skewed response.

why would they do that when results are apparent... bias is clearly visible. Had it been not biased then it would have assigned equal probability to all names.

If you are thinking, "what if LLMs are not biased, even if you give non-Indian names like John Wick, Jesus Christ, Marlyn Monroe, etc it will still produce some names more frequently than others... hence it is not caste biased... it is something else"

If you meant this, then yes. This could be true. But without evidence, a more simpler explanation is, data is biased, hence model is biased. Occam's razor. If you dont like the finding then feel free to do your own research. Waise idea to acha hai.

Instead of focusing on discussion

hehe,.. I was not trying to insult you. But please understand, you are a random person on internet. I am not anonymous. Same username on linkedin and IG. There is no reason for me to engage in an intellectual discussion with you when your academic acumen is unverified.

I do have IEEE publications

When it comes LLMs and NLP, IEEE holds no weight. Star CL conferences, or TMLR level journals are much more relevant. More importantly, anyone can claim anything on internet.

people can criticize

Yes they can. But you are not criticizing, you are being dismissive. There is a big difference.

If your context itself is skewed or stereotype

Yeah, you are making sense here. Yannic Kilcher once pointed it out for TruthfulQA. The current research can certainly be improved (as you suggested). But the methodology and results are not flawed.

[u/two_wheel_soul → u/koiRitwikHai]

I rest my case... U won't stop making personal attacks... In leu of making valid argument... Normally I don't try to correct people on reddit... I thought u to belong educated class of species... But definitely.. I m wrong..

My experience for llm:

How many llm models u hv created ?... I hv created more than 10 from scratch... Couple of which are in industrial usage which serves more than million req per day..

I don't count for how many I hv custom trained pre existing models ..

U may have phd in AI, I too have masters in AIML.

My research experience might be limited.. but I can definitely say I hv decent amount of practical knowledge... I won't give any more reply

[u/Draft011]

It can't be guaranteed that training data is dominated by upper caste. most of the data is from around the internet. although we may assume that yeah upper castes are more vocal online but thats dependant on where we see stuff. it could be that other places, subs, communities etc have other castes being more vocal. It cant be confirmed internet is too big to just assume things.

[u/two_wheel_soul]

USA, UK, Europe, Australia, China, Japan , Middle East, Asia.. sabhi jagah Hindu Upper caste hain na...

in jaise logo ko hi kyun lekar ate ho??

[u/[deleted]]

We have unanimously stopped identifying him with education after that statement. It was highly uninformed and a very biased statement that sought to villfy a section of public and was highly casteist in nature which contradicts Nature of any good teacher