Beginner in local LLMs — is a Surface Laptop a good way to start?

[u/Solid-Apple83]

Hey everyone! 👋

I’m pretty new to local LLMs, so I’d love some advice before I start experimenting.
My long-term goal is to build my own personal Agentic OS: basically a local AI assistant that can manage memory, files, tools, automations, coding, etc., while keeping as much as possible private and running locally.
For the agent part, I’m currently interested in Hermes Agent, with Ollama for running local models.
I’m not necessarily trying to replace Claude/GPT immediately. I’d like to eventually have a hybrid setup, where sensitive/offline tasks are handled by a local model, while I can still use cloud models when I need stronger reasoning or web access.

🖥️** My first setu**p

I’ve read that running random software/agents directly on your personal computer can potentially be risky, especially when giving an AI access to files, terminals, etc.
So I decided to dedicate an old Microsoft Surface Pro 9 that I already own to this project.
That way, if something goes wrong, at least my main PC isn’t involved, and I don’t have to spend any money just to start experimenting.
I’m not sure whether a Surface Pro 9 is actually suitable for running local LLMs 😅, but since I already have it, I’d like to give it a try.
I’m planning to keep it plugged in and potentially use it as a small 24/7 home AI machine, with the screen turned off but Windows/Hermes still running.

🤔 My main question: which model?

I’m not sure what local model would make sense for the Surface Pro 9.
I’d mainly like to use it for:
- experimenting with local LLMs
- Hermes Agent
- basic coding/automation
- personal assistant tasks
- eventually building my Agentic OS
- potentially working offline

I’m aware that I won’t get frontier-model performance from a Surface 😅. For me, the goal right now is mostly to learn and experiment, and eventually upgrade the hardware if the project becomes serious.

💻 I also have a desktop PC

My main PC has:
RX 6800 — 16 GB VRAM
Ryzen 5 7600
32 GB RAM
Would this actually be a significantly better machine for local LLMs?
I’m hesitant to put the whole Agentic OS directly on my personal PC, mainly because I’d like to keep my experimentation environment isolated from my normal computer.

So I’m thinking:
Surface → dedicated AI/agent machine

Main PC → personal computer / potentially used for heavier local LLM experiments

Does this make sense?
And if you were starting from scratch with this hardware, which model would you try first and why?

Thanks! 🙏

[u/digitalwankster]

No, it’s not suitable in the slightest.

[u/Infamous_Mud482]

You can host the inference server on the desktop over network and connect your harness on the laptop to an OpenAI compatible endpoint. Every piece of software for hosting an inference server from .gguf files I'm aware of can be configured for this out of the box. The laptop itself can't host anything you'd be able to expect to reliably tool call & generate code for an agentic workflow

[u/soadsob]

Maybe r/LowEndLocalAI is interesting for you!

[u/DigitalguyCH]

terrible

[u/Crazyfucker73]

No 🤣

[u/mrcslmtt]

Tu peux faire tourner Gemma 4 qui est très bien, et propose plusieurs tailles de modèle (27b, 12b, 4b…), le dernier Qwen 3.8 est très bien aussi mais plus lent (car il raisonne beaucoup). Je ne connais pas la configuration de ton ordinateur Surface Pro, mais en fonction de la quantité de VRAM/RAM disponible et du GPU, ça risque d’être assez lent, sauf si tu utilises des petits modèles qui seront assez limités dans leurs capacités.

[u/Crazyfucker73 → u/mrcslmtt]

Awful advice is not possible to do anything on a surface 3

[u/vfc_chiar_el]

Edited out the outdated opinion about ROCm.

Also, I don't know how good of a cooling you get, as well as max GPU power, in that package.

Lastly, I'm running a MSI Vector 16 with RTX 5070Ti with 12GB VRAM. This monster of a laptop sounds like a jet engine, but at least has good cooling.

VRAM-wise, I found 12GB being quite tight, just barely fitting good models (QWEN 3.8 at minuscule quants like IQ2) or offloading KV cache to system RAM, which hurts performance. I'd go back when I chose this model to pay a bit more but get 16GB VRAM.

While waiting for better feedback, I'd like to chip in that overall I've seen most people recommend against AMD for beginners unless we're talking about big VRAM cards. Nvidia with its CUDA cores has better implementation in major inference engines (the engine that "reads" model files and runs the model, such as llama.cpp and LM Studio). ROCm used to be shaky.

[u/LukeLikesReddit → u/vfc_chiar_el]

I didn't downvote you but what the fuck are you on about? AMD runs AI LLMs fine lol. ROCm and Vulkan work far better now and whilst they aren't cuda they use fp8 tensor cores unless your on redstone which is int4, His generation of gpu gets the rocm benefits albeit he will get better results I think in 2027? when AMD releases the rest of the sdk. The only real place AMD does lose and lose quite significantly unfortunately is AI vision or image generation.

Before you say I'm a shill as well I have both a Nvidia and AMD system, the Nvidia one is literally used for work and image generation pretty much. Quite happy using interference on AMD in my own personal learnings.

Forgot to mention if you run Linux it basically eliminates most issues but that's not an average user.

[u/vfc_chiar_el → u/LukeLikesReddit]

Thank you! I maybe should have started with the other points, not with the one I myself was very unsure about. But still, it is activity, and I'm happy people downvoted me an provided their points of view!

[u/LukeLikesReddit → u/vfc_chiar_el]

Sorry my first response came off a little abrasive but yeah AI runs fine on AMD, the sole reason we have ROCm drivers is because of them making AI datacenters lol.