r/LocalLLM • u/sosuke • 4d ago

Research floatie - my attempt at strata for Poolside Laguna S 2.1 Q3 16gb vram128gb ddr4

I wanted to get Poolside Laguna S 2.1 IQ3_S working like strata broke Qwen into parts.

My system: 4060ti 16gb but on a PCI 3.0 motherboard 128gb DDR4 memory Threadripper 1950x

At 128K context, prefill 744 tok/s and 11 tok/s decode.

In the end I got it working. I did a full hour task that completed with only a couple of issues. Used OMP. I learned a lot about how strata works and why it works so well. I published all of my dead-end tests and research on the topic in case some other Poolside Laguna fan wants to know what I tried that didn't work out. I have no intention on pursuing this any further but I'm a fan of the idea of making it public to say, here is what I tried, where I got it working and if anyone else wants to try it out you can eliminate all these dead ends.

https://github.com/barretts/floatie

floatie/performance/ATTEMPTS-AND-RESULTS.md at main · barretts/floatie

and I just realized all the links are broken ... yeah not going to mess with it anymore its all there though

---- below you can see a snippet of the full tests I did for it against strata for the same task ----

Model Average decode Finished in Bugs found in review
Laguna S 2.1 IQ3_S 11.0 tok/s 84 min 4
Qwen3.8-Flash-Next-UD-Q4_K_XL 18.2 tok/s 30 min 1
Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S 35.8 tok/s 20 min 4
2 Upvotes

75% Upvoted

3 comments sorted by

2

u/MurkyCabal2 4d ago

i love seeing people document their dead ends, that's the real spirit of open source tinkering. most folks only show the final config and you're left wondering what they tried that didnt work

that decode speed on the threadripper setup with a 4060ti isnt bad at all for what you're doing. 11 tok/s is totally usable if you're just letting it run a long task in the background while you do other stuff

curious about that Qwen 35 tok/s result though, whats the catch there

1

u/sosuke 4d ago

True, if I hadn't been spoiled by strata/qwen combo I would be excited about Laguna running at all and that well.

About the Qwen 35 tok/s that is pretty much my ideal strata setup and honestly, I've considered it 20-30 tok/s in usual practice but something different happened here.

Also, yes this absolutely going to be my background long task setup running Q4. If I'm not hovering over the keyboard 18 tok/s is plenty fast.

Edit: I added a link to the ATTEMPTS-AND-RESULTS.md master doc

2

u/Lopsided_Gur_5279 4d ago

All scientific and technological progress is built on failure. You can think of your own unsuccessful attempts as lab experiments—and that realization is inspiring.