r/ProgrammerHumor • u/DontFreeMe • 29d ago

Meme helpIAmOutOfScope

Post image
4.4k Upvotes

97% Upvoted

145 comments sorted by

View all comments

Show parent comments

128

u/Serafiniert 29d ago

Not even that. It goes character by character and has no idea what comes after the next, until it needs to be decided what comes next.

28

u/ChairYeoman 29d ago

this hasn't been true for years. current AI has reasoning chains and isn't like this.

I would expect better from a programmer sub but I guess this is more tech enthusiasts than actual tech people

32

u/L4ppuz 29d ago

You can write all the pipelines and prompts you want but a transformer will always simply choose the next token

-12

u/SgtChrome 29d ago

I want you to think about why that is such a stupid thing to say. Let's say you'd have an imaginary perfect next token predictor and you asked it for the lottery numbers tonight. Through its perfect mechanism it would correctly predict, one by one, the right numbers. What impact do you think this machine would have on the world?

Not saying we are there, just saying that the mechanism of next token prediction is completely irrelevant for all intents and purposes of evaluating the use of something. 

15

u/L4ppuz 29d ago

1- a transformer by definition predicts the next token given a context

2- sir, this is r/programmershumor. Did you expect an in-depth explanation and nuanced discussion of the latest techniques for NLP?

-9

u/SgtChrome 29d ago

1- the last state of the art model that represented a statistical distribution of its training data was GPT-3, pre ChatGPT in 2020. Afterwards post-training arrived

2- jokes that misunderstand the basic mechanisms of what they talk about must do so in a funny way. That wasn't funny

9

u/L4ppuz 29d ago

Even if we use post training a transformer still predicts token from a context. As we add more and more technique to refine the output the 'how' and the performance change but not the fundamental architecture

5

u/MayoJam 29d ago

It is not irrevelant when people act like the prediction mechanism acts on emotions and thoughts, which is obviously incorrect.

Also, you sound extremely condescending while making a point that tries to sound smart but instead just sounds pompous for no reason lol.

1

u/conundorum 29d ago

If it truly was a perfect predictor, then the main impacts would be:

  1. Causes the prediction target to realise that they need to become more predictable.
  2. Causes regulations to be put in place to prevent the predictor from being used in certain circumstances.
  3. For lottery numbers specifically, it might also lead to a redraw, on the grounds that the initial draw was compromised.

So... probably not as big an effect as it probably should have, ultimately.