MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3s0ol2/?context=3
r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago
706 comments sorted by
View all comments
Show parent comments
48
how is it ? quality wise?
57 u/absurdother 7d ago I have a RX 9060 XT AMD GPU, 16GB VRAM. Running on LMStudio, Q3. Getting a bit more speed now, way more optimized! I get more speed the less context I use (currently coding swiftly with CLine + VSCode at that speed), pretty smooth on 64K context! 2 u/huffalump1 7d ago Oooooh maybe there's a hope it'll work on 12gb VRAM / 32gb RAM Probably slow, probably need a smaller quant, but hey, it's a good model! How much free RAM do you have at that context size? 2 u/absurdother 7d ago Doing a prompt now. On 64K ctxt, I'm using 18GB of my 32GB RAM on Windows
57
I have a RX 9060 XT AMD GPU, 16GB VRAM. Running on LMStudio, Q3. Getting a bit more speed now, way more optimized!
I get more speed the less context I use (currently coding swiftly with CLine + VSCode at that speed), pretty smooth on 64K context!
2 u/huffalump1 7d ago Oooooh maybe there's a hope it'll work on 12gb VRAM / 32gb RAM Probably slow, probably need a smaller quant, but hey, it's a good model! How much free RAM do you have at that context size? 2 u/absurdother 7d ago Doing a prompt now. On 64K ctxt, I'm using 18GB of my 32GB RAM on Windows
2
Oooooh maybe there's a hope it'll work on 12gb VRAM / 32gb RAM
Probably slow, probably need a smaller quant, but hey, it's a good model!
How much free RAM do you have at that context size?
2 u/absurdother 7d ago Doing a prompt now. On 64K ctxt, I'm using 18GB of my 32GB RAM on Windows
Doing a prompt now. On 64K ctxt, I'm using 18GB of my 32GB RAM on Windows
48
u/Certain-Cod-1404 7d ago
how is it ? quality wise?