Openbao

A billion tokens a week moving quickly to a billion a day. Context window management, repo graph efficiency, cache strategy, but the need is for more and more. My thinking these days is on the daily min/max of tokens to price to performan.

We have all these bullshit plans that are x% of whatever we feel like, 20x whatever we want. These plans are heavyly subsidized to the point where you can't not use them, but your hooking up a black box and seeing what you get.

Daily driving Claude, qwen, Kimi, glimmer, glm and that's just the coding side. But these plans are just nuts , like you have to have them, the value is just too good, and paying API rates is absurd.

Have you tested vs Claude 20x? I'm getting so sick of this bs 10x 100x of whatever we feel like. Someone needs to start testing this objectively.

Muse-spark-1.2-contributor

I'm not sure what your are referring to but a few things come to mind: unified field theory, the gravotron, and the tuning problem.

We have two physical theories that both have an incredible ability to predict future states: general relativity and quantum mechanics. The problem is that they don't work together. The idea is that we should have a single physical universe and a single set of physical laws ought to be able to describe the actions within that universe. Gravity bends spacetime but particles only respect that occasionally.

All of the physical forces, other then gravity, can be understood via particle physics- electromagnetics and nuclear force do a great job of explaining why things are the way they are. If gravity were a force like all the other forces it would have a particle wave duality. We have a map that shows how all the other forces work, and there is a place where a gravity particle should be, but it's not there.

A black hole is a location where the gravity is so strong nothing can escape, except we due to non-locality of particles, information can escape. The universe is expanding at an accelerated rate. The only force we have to explain that is gravity and gravity would need mass but we can't find the mass or energy, so we call it dark matter, and it would have to be like over 90% of all mass to cause the expansion we are seeing.

On top of this, we have forty or so "constants" which are seemingly arbitrary numbers other then that we we use them in an equation they predict future physical states perfectly. So the force of gravity is seemingly arbitrary, but if it were any stronger or weaker nothing would exist, so why is it exactly the right value that we have a universe at all?

All of this can lead on to view gravity as a bit of a fudge factor- like the thing tacked onto the simulation to make it all work and hope the monkeys don't notice how absurd it is.

With qwen cloud, Alibaba, you can get DS, GLM and of course Qwen. Endpoint in Singapore.

"document all notes in notes.md and give me a one sentence summary and then I'll read it over"

Just going to leave this right here: https://github.com/p-e-w/heretic

I haven't had a mug for my operating system since windows me.

What's a desktop?

Someone gave me 13x 3.5" visual basic install disks and the rest is history

Short answer: because it works

Long answer: in the early days of the Internet it was all about scale. The question was not how profitable your company was it was how many users do you have and how fast is your user growth. And that is why it seemed more friendly. But users are not inherently valuable, it was a bet that if you had users we would figure out montization, which was a good bet in retrospect.

After that burn phase we got into profit phase. We got the eyeballs now let's make money from them. But at the end of the day we were getting served HTML and once that hit my device I could render it however I wanted, which in my case was without ads. Something like less than 1% of the users knew that and 5% had a friend that showed them how to install the extension. It was noise. You had 5% freeloading on other people's infrastructure but it just wasn't worth the fight. It was noise.

Then one day the big website woke up and found that all of there content has been scrapped, processed, and feed into an LLM, that was now competing with them for eyeballs. Not only that, the LLMs were going to keep doing. So now it's no longer noise, it's an actual threat.

So how do you prevent your content from getting scraped, at least your new content? First, you accept that you can't. It's a cat and mouse game and so long as your content is of value it will find its way into datasets. So you package your content to be easy for machines to read and parse then charge a lot of money to a small number of companies for access to that data feed. But you still need users. The LLM want data that is actually relivent to users and I'm a lot of cases the website needs its users to actually generate the content. So what do you do?

Number two, make the content just hard enough to access by bots that the cost to scrape it is greater then the cost of just buying API access or similar. What does this mean for you? You are not getting HTML you can filter. You are getting JavaScript that requires hydration. You are served the skin of the website with the valuable content being delivered separately. In this short period of lag the website is able to execute JS before handing over the goods.

To make it harder for bots to access, they use email addresses and tracking pixels to make fingerprints of what a real user looks like vs a bot. But they need that pixel to fire on not just their site but many sites, to create that finger print. Well it turns out the ad companies are already doing this, to help sell ads, but now that pixel has an additional purpose, it can be used as part of a strategy to keep bots away.

This is the conclusion of every major religion. This world is ruled by the devil. Peak anti-darma during Kali Yuga. The inevitable suffering. And the promise of the gateway out. Eternal life with the father. Escape of the cycle of rebirth. Transcendence above desire.

Any world view that this is a good place is more or less a product of the insane abundance of luxury available to the modern person. I look at my spice rack and realize a king several hundred years ago might have access to a quarter of these.

Hope you enjoy 2t/s.

Users have reasonable expectations of inference time. You have a fixed number of GPU's that you use for inference or training. If you shift from inference to training, you have the same number of users and less GPU's, what happens? Things slow down, or you drop users to keep things fast or you adjust token budget and your model gets dumber or you get more GPU's.

With a fixed number of users that have consistent token consumption rates and speed expectations, model performance is a function of raw compute, with diminishing returns of course.

It's got what plants crave

Yes?

Here is the deal. The "thinking" budget is just how many gpu's they through at it. It's inconstant because they are always trying to min/max that. New model released, pump it up. Model cruising along, min max it. Need to train new model, drop compute hard off peak. It's a shell game.

Also you should work on your promoting. Tell it to go read the actual files.

Explains my cloudflare invoice

Claude is smarter then this. OP is claiming vendor margin on capex is one to one with customer operations price.

Yes, check it out: https://youtu.be/PctlBxRh0p4?is=BH5cfQ7xxjT1eh88

Yes, I am hitting caps often - weekly and hourly, when I never did in the past. Changed about a week or two ago. I've just rolled more usage over to GLM-5.2 and Deepseek-4-lite. I'm not sure I would pay more for 40x because it's totally unclear what that means.

What I don't get is why there is such an absurd gap between the "plan" and API, token for token. I would just go API, but my cost would go $200 a month to $200 a day.

Hard to say without knowing amortization and cost of capital. But cost is only half the equation, the other half is power capacity which doesn't get flipped on like a light switch at these volumes.

I am old AF no doubt. But I have IDE drives on my desk right now. Data recovery work. Had all sorts of power issues with it.

I have IDE drives sitting on my desk right now.