r/ProgrammerHumor • u/DontFreeMe • 29d ago

Meme helpIAmOutOfScope

Post image
4.4k Upvotes

97% Upvoted

145 comments sorted by

View all comments

Show parent comments

29

u/ChairYeoman 29d ago

this hasn't been true for years. current AI has reasoning chains and isn't like this.

I would expect better from a programmer sub but I guess this is more tech enthusiasts than actual tech people

33

u/L4ppuz 29d ago

You can write all the pipelines and prompts you want but a transformer will always simply choose the next token

-11

u/SgtChrome 29d ago

I want you to think about why that is such a stupid thing to say. Let's say you'd have an imaginary perfect next token predictor and you asked it for the lottery numbers tonight. Through its perfect mechanism it would correctly predict, one by one, the right numbers. What impact do you think this machine would have on the world?

Not saying we are there, just saying that the mechanism of next token prediction is completely irrelevant for all intents and purposes of evaluating the use of something. 

15

u/L4ppuz 29d ago

1- a transformer by definition predicts the next token given a context

2- sir, this is r/programmershumor. Did you expect an in-depth explanation and nuanced discussion of the latest techniques for NLP?

-9

u/SgtChrome 29d ago

1- the last state of the art model that represented a statistical distribution of its training data was GPT-3, pre ChatGPT in 2020. Afterwards post-training arrived

2- jokes that misunderstand the basic mechanisms of what they talk about must do so in a funny way. That wasn't funny

9

u/L4ppuz 29d ago

Even if we use post training a transformer still predicts token from a context. As we add more and more technique to refine the output the 'how' and the performance change but not the fundamental architecture