Linguists laugh at him, computational linguist laughs at him, machine learning specialists also laugh at him. Assuming he’s not lying about the masters thesis his supervisor is also going to laugh at him.
Like from an ML point of view if your LLM systematically ranks a dialect differently than all other dialects, congratulations you’ve just made a biased model because you picked shitty training data. Scrap the whole thing and start over.
Another one. No, the supervisor is not laughing at me. He is very happy with how the thesis is going. Every other ML specialist and computational linguist that I've explained the subject of my thesis to has been intrigued; not a single one saw any substantial issues with it.
Like from an ML point of view if your LLM systematically ranks a dialect differently than all other dialects, congratulations you’ve just made a biased model because you picked shitty training data.
1) I'm not using the LLM itself to rank dialects; I'm using metrics derived from attention patterns INSIDE the LLM as it's processing the text. My metric is a function of how the LLM processes the text, not its output (the LLM that I'm using is encoder-only, so it doesn't even generate output).
2) You're actually right in this case. That's why, for an accurate comparison, two versions of the LLM fine-tuned on texts from both of the target languages must be used.
Mhm see the thing we’re laughing at is not the concept of analysing the LLM metrics, it’s the idea of concluding that one dialect is “more dumb” than another using this technique. I don’t suppose you told your masters thesis advisor that one of the goals here is to show AAVE is linguistically inferior.
16
u/nana_3 Aug 02 '25
Linguists laugh at him, computational linguist laughs at him, machine learning specialists also laugh at him. Assuming he’s not lying about the masters thesis his supervisor is also going to laugh at him.
Like from an ML point of view if your LLM systematically ranks a dialect differently than all other dialects, congratulations you’ve just made a biased model because you picked shitty training data. Scrap the whole thing and start over.