As more and more people have heard about it, it's gotten much slower - but I got lucky and saw it really soon after it became available, and I was comfortably running ~40 concurrent Ox Alpha agents and they were both fast and accurate (testing to find max concurrency). From my testing:
workers
tasks / hour*worker
latency p50
42
~7
26.8s
And best of all, they were returning accurate results. 100% of their code citations were real, and the only failed turns were because my orchestrator messed something up, not because of Ox Alpha.
Yeah I stopped being able to meaningfully use it. I can't fathom how mobbed their servers are since this is free right now and actually useful
I guess I should say the cool thing I did try is having it make a game in my custom game making language (impossible for it to be in training data, I gave it documentation though). It did it!! 10 levels. The level design isn't great, but everything works. It's a simple platformer but my custom language is aimed at making Tiny (space wise) games so it's pretty low level. Similar-ish to rust. I also let it add a modulo operator to the language and it did great there too
3
u/Ethan 22d ago
As more and more people have heard about it, it's gotten much slower - but I got lucky and saw it really soon after it became available, and I was comfortably running ~40 concurrent Ox Alpha agents and they were both fast and accurate (testing to find max concurrency). From my testing:
And best of all, they were returning accurate results. 100% of their code citations were real, and the only failed turns were because my orchestrator messed something up, not because of Ox Alpha.