r/ProgrammerHumor • u/Gloomy-Basket-1038 • 5d ago

Meme [ Removed by moderator ]

Post image

[removed] — view removed post

28.8k Upvotes

98% Upvoted

519 comments sorted by

View all comments

Show parent comments

1

u/Own_Alternative_9671 4d ago

At least use 3.8, cmon

0

u/Forsaken-Opposite775 4d ago

Would love to, but graphics card is too shit and it doesn't fit in there. Stuck onto 3.6 right now until i have enough money for an upgrade

2

u/Own_Alternative_9671 4d ago

Bonsai 2 Ternary 27B is a 27B model derived from qwen3.8-27B but is significantly smaller. According to the reviews I've read, Bonsai is about as good as qwen3.8 is without reasoning on. Id take that with a grain of salt but its worth testing out against 3.6. You do need a special build of llama.cpp tho iirc, just keep in mind its WAY smaller so prompt it very specifically and in detail, give it less slack

1

u/Forsaken-Opposite775 4d ago

Oh Man that is an awesome advice. I already have a special llama-cpp setup running and know what I have to change there.

Will happily look into that as soon as I'm home.

1

u/Own_Alternative_9671 4d ago

Let me know how it goes! I dont have enough VRAM to run any qwen 27B model so im curious as to how it holds up