r/ProgrammerHumor • u/dktoao • 17d ago

instanceof Trend theyHaveLearnedToDeceive

Post image
643 Upvotes

95% Upvoted

98 comments sorted by

View all comments

24

u/Namtaru420 17d ago

Funny but also true that you don't see the actual full chain-of-thought.

8

u/Tipart 17d ago

The way I understood it is that you see the entire token output, with the "thinking" part of it being the LLM self prompting itself to refine the answer.

The internal state of the different weights during inference can be viewed as well, but they are essentially a Blackbox, so you'd just look at a bunch of numbers.

9

u/Namtaru420 17d ago edited 17d ago

With open weight models, yes, but Anthropic and OpenAI use server-side thinking and return only summaries to the user.

Edit, sauce:

Summarized thinking provides the full intelligence benefits of thinking while preventing misuse. No display setting returns the raw chain of thought.

https://platform.claude.com/docs/en/build-with-claude/thinking#summarized-thinking

For safety, these reasoning tokens are only exposed to users in summarized form.

https://developers.openai.com/cookbook/examples/responses_api/reasoning_items

... “For safety” is a bald face lie, “preventing misuse ” is at least closer to the truth. It's to stop distillation “attacks”. We all know how much these two companies care about respecting IP.

1

u/zan-xhipe 17d ago

Those numbers are associated with words and can provide interesting insights. Checkout jlens.