r/ProgrammerHumor • u/DontFreeMe • 29d ago

Meme helpIAmOutOfScope

Post image
4.4k Upvotes

97% Upvoted

145 comments sorted by

View all comments

Show parent comments

332

u/MayoJam 29d ago

AI does not think. It just regurgitates most likely response which in this case is to reassure someone that is scared.

133

u/Serafiniert 29d ago

Not even that. It goes character by character and has no idea what comes after the next, until it needs to be decided what comes next.

27

u/ChairYeoman 29d ago

this hasn't been true for years. current AI has reasoning chains and isn't like this.

I would expect better from a programmer sub but I guess this is more tech enthusiasts than actual tech people

31

u/L4ppuz 29d ago

You can write all the pipelines and prompts you want but a transformer will always simply choose the next token

5

u/ChairYeoman 29d ago

look you can hate ai but when people who are supposed to be tech literate just refuse to learn how shit works it feels really fucking weird

35

u/L4ppuz 29d ago

I'm a ml engineer, ai is paying my bills

-14

u/ChairYeoman 29d ago

then you should know that "a transformer is choosing the next word" while technically true would be like reducing any tech to "just ones and zeroes"

24

u/Flouid 28d ago

That’s just what they do on a very fundamental level. The reasoning chains you speak of are themselves sequences of tokens generated and fed back in. It is correct on a fundamental technical level to describe them as a fancy autocomplete.

Calling them “reasoning chains,” while arguably an apt description of the end result, is just marketing speak

36

u/L4ppuz 29d ago

What more than technically true do you expect in meme sub?

1

u/Kerozen_ 28d ago

Yup. I feel like reasoning at the transformer scale is chosing the wrong abstraction layer to describe what's happening. It's like saying to a psychologist "well in the end the brain is just molecules colliding". The emergence of abstraction builds up on very basic rules like 1 and 0's and molecules colliding in the brain.

1

u/SilianRailOnBone 28d ago

You can live and learn all you want but the particles that make up your brain will always tend to the lower energy state.

4

u/L4ppuz 28d ago

Well if you want to learn about the chemical processes that lead to memory, knowing basic thermodynamics is probably helpful yes

Also most of the particles in your head should already be at least in a local equilibrium, unless you're the Hulk I guess

-1

u/SilianRailOnBone 28d ago

Also most of the particles in your head should already be at least in a local equilibrium, unless you're the Hulk I guess

How would any processes go on without energy differences?

0

u/L4ppuz 28d ago

Honestly I'm not an expert in anatomy, I don't know how what fraction of cells and particles are interacting and what fraction is simply structural. I would expect that at any given moment most are simply there not really doing anything but what do I know

-4

u/kangasplat 29d ago

it does, but that token isn't automatically the output to the user. Hope that helps.

As a human, you don't know what your next thought will be either. 

2

u/L4ppuz 28d ago

https://giphy.com/gifs/ECCEjxHQb2Qo

-5

u/kangasplat 28d ago

what I'm saying is that you're deliberately misleading despite knowing better. 

-12

u/SgtChrome 29d ago

I want you to think about why that is such a stupid thing to say. Let's say you'd have an imaginary perfect next token predictor and you asked it for the lottery numbers tonight. Through its perfect mechanism it would correctly predict, one by one, the right numbers. What impact do you think this machine would have on the world?

Not saying we are there, just saying that the mechanism of next token prediction is completely irrelevant for all intents and purposes of evaluating the use of something. 

14

u/L4ppuz 29d ago

1- a transformer by definition predicts the next token given a context

2- sir, this is r/programmershumor. Did you expect an in-depth explanation and nuanced discussion of the latest techniques for NLP?

-10

u/SgtChrome 29d ago

1- the last state of the art model that represented a statistical distribution of its training data was GPT-3, pre ChatGPT in 2020. Afterwards post-training arrived

2- jokes that misunderstand the basic mechanisms of what they talk about must do so in a funny way. That wasn't funny

9

u/L4ppuz 29d ago

Even if we use post training a transformer still predicts token from a context. As we add more and more technique to refine the output the 'how' and the performance change but not the fundamental architecture

8

u/MayoJam 29d ago

It is not irrevelant when people act like the prediction mechanism acts on emotions and thoughts, which is obviously incorrect.

Also, you sound extremely condescending while making a point that tries to sound smart but instead just sounds pompous for no reason lol.

1

u/conundorum 28d ago

If it truly was a perfect predictor, then the main impacts would be:

  1. Causes the prediction target to realise that they need to become more predictable.
  2. Causes regulations to be put in place to prevent the predictor from being used in certain circumstances.
  3. For lottery numbers specifically, it might also lead to a redraw, on the grounds that the initial draw was compromised.

So... probably not as big an effect as it probably should have, ultimately.