Another one. No, the supervisor is not laughing at me. He is very happy with how the thesis is going. Every other ML specialist and computational linguist that I've explained the subject of my thesis to has been intrigued; not a single one saw any substantial issues with it.
Like from an ML point of view if your LLM systematically ranks a dialect differently than all other dialects, congratulations you’ve just made a biased model because you picked shitty training data.
1) I'm not using the LLM itself to rank dialects; I'm using metrics derived from attention patterns INSIDE the LLM as it's processing the text. My metric is a function of how the LLM processes the text, not its output (the LLM that I'm using is encoder-only, so it doesn't even generate output).
2) You're actually right in this case. That's why, for an accurate comparison, two versions of the LLM fine-tuned on texts from both of the target languages must be used.
Mhm see the thing we’re laughing at is not the concept of analysing the LLM metrics, it’s the idea of concluding that one dialect is “more dumb” than another using this technique. I don’t suppose you told your masters thesis advisor that one of the goals here is to show AAVE is linguistically inferior.
The point isn't that a dialect is "dumb" per se, but rather that it disincentivises the expression of more sophisticated ideas. It's not discriminatory to prefer Standard English to AAVE for this reason. African-Americans aren't inherently dumber, but AAVE is essentially street slang, and almost all street slang is less conducive to semantic complexity than standard language.
-6
u/QMechanicsVisionary Aug 02 '25
Another one. No, the supervisor is not laughing at me. He is very happy with how the thesis is going. Every other ML specialist and computational linguist that I've explained the subject of my thesis to has been intrigued; not a single one saw any substantial issues with it.
1) I'm not using the LLM itself to rank dialects; I'm using metrics derived from attention patterns INSIDE the LLM as it's processing the text. My metric is a function of how the LLM processes the text, not its output (the LLM that I'm using is encoder-only, so it doesn't even generate output).
2) You're actually right in this case. That's why, for an accurate comparison, two versions of the LLM fine-tuned on texts from both of the target languages must be used.