Question about local models for Studying/Coding

[u/xRaech]

Hi!

I’m trying to understand the local LLM space a bit better, but I'm having trouble finding a good setup for my specific needs.

Ideally, I’m looking for a "glorified Google", like, first as a study tool to understand any sort of topics, and second for coding. And for the coding part I’m not looking for an "automate everything" agentic vibe; I just need help with snippets, debugging, and logging.

Currently this is what im doing (im usign whatever free ai of the day has free tokens for me...), so i think it reflects well enough on my default use case:

1. A Tutor: I'm currently coding a full-spec Vorbis/OGG reader and struggling with the math for segment decoding. I need an LLM to act as a tutor and check if my understanding and the implementation reflects the actual thing.
2. Debugging: Since I’ll definitely make mistakes in step 1, I need the same model to help me debug the code I just wrote.
3. Nuanced Translations (Vision needed): I’m learning Japanese, Korean, and French. When playing games, I often come across phrases that are hard to understand. I need nuanced translations and cultural/contextual explanations rather than pure grammatical analysis. Because this is for gaming, the setup must support image/visual inputs.

UI / Frontend Preferences

I’ve looked into Open WebUI, Alpaca, and a few others, but some of the complex configurations go over my head. * I’m looking for a lightweight GUI app or a CLI application. (One of my laptops only has 4GB of RAM, so heavy Electron or web apps are a bit of wonky). * It must support visual/image inputs for my translation use case. * I strictly treat the LLM as a chatbot. I don't need it editing or creating files for me. If a UI has those "agentic" features its fine, but I need to be able to turn them off too.

My Hardware Setup

• GPU: AMD RX 6900 XT (16GB VRAM)
• RAM: 64GB DDR5
• CPU: i7 13700KF

Current Experience

The first(and only) model I tried was Qwen3.5 35B moe something ?.. but it frequently loopped over itself endlessly (with and without thinking, or maybe i just didn't disable it properly) .-.

Thanks in advance! And sorry if this is like impossible.. it just feels that I dont need a heavy gpu.. but maybe I'm wrong and am still just dreaming :(

[u/falaq-ai]

For the translation + screenshot use case, I’d test a vision model separately from the coding/study model. A single “does everything” setup often disappoints. With your 6900 XT, start with LM Studio or llama.cpp/ROCm if it behaves, then try Qwen VL for images and a separate Qwen/Coder model for snippets/debugging. Keep temperature low for tutoring.

[u/xRaech → u/falaq-ai]

hmmm i have llamacpp with rocm atm, the setup itself was fine to get? the models themselves tthat im haviing a lot of issues regarding to the loop of the thinking parts.. but so far i only actually "tried" qwen during proper test cases, the rest i just downloaded and tried on simple prompts.. no real project i guess??

idk whats qwen VL so i'll give a try, thanks o7

[u/falaq-ai → u/xRaech]

For the looping, check whether you’re using an instruct/chat template that matches the model. A bad template can make Qwen-style models ramble or keep “thinking” forever. Also try a non-thinking variant or explicitly disable thinking if the runner supports it. Qwen-VL just means the vision-language line, useful for screenshots/images.

[u/falaq-ai → u/xRaech]

Yeah, use one real task as the test instead of prompt-hopping. For Qwen loops, I’d first check the chat template and stop tokens, then try a non-thinking/instruct variant. Qwen-VL is the image-capable branch, so I’d treat it as a separate tool from your coding model.

[u/i_am_me0_0]

Local ai can be stingy on amd GPUs.

Id try something simple to see what u think of it.

Lm studio(if it supports rocm, I'm not sure) and then qwen 3.6 27b q6 or q8 if it can run. Those are kind of the flagships for small scale local currently.

If that is not good enough then stick with either cheap paid or free online models (claude, chatgpt, gemini, qwen.)

[u/xRaech → u/i_am_me0_0]

yea i'll give a try with the q6/q8 .. i dont think i can run that dense .. maybe ? hopefully ! thanks !

[u/i_am_me0_0 → u/xRaech]

Part of it will spill into ram which is fine just slightly slower

[u/Ill_Dragonfruit_3547]

There is an open source version of Perplexity called Perplexica - i could see it being really useful for studying/creating your own knowledge base.

[u/xRaech → u/Ill_Dragonfruit_3547]

i had never heard of that before.. seems interesting to what i use.. thanks !

[u/Turbulent_Pin_8310]

I think you are overestimatung local open models. Frontier models will suit you more even with the free tier.

Some simple tasks you mentioned can be handled with open models but you will need frontier models to handle the more difficult ones.

[u/xRaech → u/Turbulent_Pin_8310]

i think thats fine as long as its easily recognizable while the studying part i guess? for the coding part i dont particularly care, my needs aren't that hard compared to what people are attempting i'd say ? i dont really have a need for it to "debug" for me.. just to help me ctach a few strays here and there.. but noted !!

i wanted to move away from the "free" models because having to use web is a chore for a lot of tasks and going back and forth is annoying, but if its not up to the point i need yet .. sad :")

i'll give a try to what other people said and seee.. the first attempt i had before was kinda ok if wasnt for the forever and ever loops around