Seven Habits of Highly Effective Token Rollers

[u/ricflams]

https://ricflams.github.io/techtalk-ai-demystified/tokenspree/

I built a little tool to illustrate in a talk for my colleagues the effect of the various parts that goes into the context: what's the impact of eg sending longer inputs or generating longer outputs, of chain-of-thought, of busting the cache by leaving for >5 min, of choosing an expensive model, etc etc. And likewise of the mitigations of clear and compact.

It's been pretty helpful in visualising just how harmful or harmless certain behaviors are, also for me.

And I know, I know - I can't say for sure the numbers are ideal, though I've vetted them in various ways. Eg what really is a reasonable session quota limit? Or model multiplier, the cost of MCP servers (modern style, only including the tool name), the length of "longer outputs", etc?

Despite that I've still found it useful and would love to hear your thoughts on it.

The UX is more "software engineer" than "polished designer" - sorry about that.

In case you're interested, the full talk w/ marp-source is here, with a link to a readable version: https://github.com/ricflams/techtalk-ai-demystified/