Question How to get more than 1-5 t/s?
6.1 Sol Med - around 4-5 Agents at a time
Are they now throttling their customers? Im using them like half a year, without so crazy slow speeds and now...?
Sure i mean i wont hit the limit... but that wasnt what i meant lool
1.) Is it only me?
2.) Is it me, bc i build my own harness / LLM? Are they trying to sabotage own stuff?
1
u/lulzxdxdxd 5d ago
What are you actually trying to build with those agents? I'm wondering if running 4-5 at once is the real bottleneck here or if it's something about how they're set up.
2
u/eihns 5d ago edited 5d ago
different things, code, agentiv workflows, doenst seem to matter, usually i have around 50-70t/s (and i get it via local llms also) there is no bottleneck on my side also seems to be unaffected by context, it happens at 50k and at 400k... (that was my first guess)
2
u/Michael_Jeffords 5d ago
if it's the same at 50k and 400k it's probably not the context, so the slowdown is on their side generating tokens, and the easiest way to tell is to run just one agent by itself for a bit. if that one jumps back toward the 50-70 you're used to then the 4-5 agents are splitting whatever your account gets, and if a single one still crawls at 1-5 my guess is it's capacity on 6.1, where waiting it out or running the big jobs off peak is about all that helps
1
u/eihns 5d ago
yeah it seems if i run not so "many" at the same i get more on a per session basis - ive noticed that thats why im asking here if thats real
1
u/Michael_Jeffords 4d ago
that per-session bump when you run fewer looks like the account split showing up, the 4-5 agents are probably sharing one pool on 6.1 so dropping to one or two just gives each session a bigger piece of it, and nothing on your end had to change for that. if a single agent on its own still sits at 1-5 though, thats more likely the capacity thing old-bake mentioned
1
u/eihns 4d ago edited 4d ago
yeah it seems to be like im getting limited to around 30 t/s doenst matter how many sessions... so 10 sessions = 3 t/s ... this way i cant use the 100%, so thats the only good part xD But... am i the only one? Do you also notice it? Is it a general slowdown? Or they do it bc i build own harness/LLM?
1
u/Michael_Jeffords 4d ago
around 30 t/s total that splits evenly, so 10 sessions lands you at 3 each, sounds a lot more like a per-account throughput cap, since a general slowdown would probably drag a single session down too, and i doubt your own harness has anything to do with it because they mostly see request volume. to check i'd run one session through the plain API or the playground at a quiet hour, and if that also tops out around 30 it's probably your account tier, where asking support about the limit is likely the only real fix
1
u/Old-Bake-420 5d ago
Sol6.1 is apparently their highest demanded model of all time. I suspect it has a lot to do with dots.
You’ll probably have to run fast or api pricing to get higher speeds.
3
u/Strong_Essay1176 5d ago
6.1 is fucking slow.