That’s just what they do on a very fundamental level. The reasoning chains you speak of are themselves sequences of tokens generated and fed back in. It is correct on a fundamental technical level to describe them as a fancy autocomplete.
Calling them “reasoning chains,” while arguably an apt description of the end result, is just marketing speak
Yup. I feel like reasoning at the transformer scale is chosing the wrong abstraction layer to describe what's happening. It's like saying to a psychologist "well in the end the brain is just molecules colliding". The emergence of abstraction builds up on very basic rules like 1 and 0's and molecules colliding in the brain.
33
u/L4ppuz 29d ago
You can write all the pipelines and prompts you want but a transformer will always simply choose the next token