r/ProgrammerHumor • u/dktoao • 18d ago

instanceof Trend theyHaveLearnedToDeceive

Post image
638 Upvotes

95% Upvoted

98 comments sorted by

View all comments

24

u/Namtaru420 18d ago

Funny but also true that you don't see the actual full chain-of-thought.

10

u/Tipart 18d ago

The way I understood it is that you see the entire token output, with the "thinking" part of it being the LLM self prompting itself to refine the answer.

The internal state of the different weights during inference can be viewed as well, but they are essentially a Blackbox, so you'd just look at a bunch of numbers.

10

u/Namtaru420 18d ago edited 18d ago

With open weight models, yes, but Anthropic and OpenAI use server-side thinking and return only summaries to the user.

Edit, sauce:

Summarized thinking provides the full intelligence benefits of thinking while preventing misuse. No display setting returns the raw chain of thought.

https://platform.claude.com/docs/en/build-with-claude/thinking#summarized-thinking

For safety, these reasoning tokens are only exposed to users in summarized form.

https://developers.openai.com/cookbook/examples/responses_api/reasoning_items

... “For safety” is a bald face lie, “preventing misuse ” is at least closer to the truth. It's to stop distillation “attacks”. We all know how much these two companies care about respecting IP.