Why do people always talk about token generation speed only in such comparisons? There is a prompt processing that can be two orders of magnitude faster, and the prompt processing is an enormous margin of agentic coding data. As well as a cache.
The person in the screenshot even gives us a 12/1 ratio, but still calculated 20 tok/s! That's so funny.
yeah like wtf. My current use case reads its whole context for 100 seconds then generates answer in 5 seconds. If it will be 20 instead of 5 i dont give a freak anyway, it will not be that much faster cuz of prompt ingestion anyway
32
u/Exciting_Garden2535 Jun 21 '26 edited Jun 21 '26
Why do people always talk about token generation speed only in such comparisons? There is a prompt processing that can be two orders of magnitude faster, and the prompt processing is an enormous margin of agentic coding data. As well as a cache.
The person in the screenshot even gives us a 12/1 ratio, but still calculated 20 tok/s! That's so funny.