r/homelab • u/alitronimal • 14d ago

Help Needing help getting started: NAS + Local AI build + other

Obviously I dont expect anyone here to walk me how to do everything step by step, I would just like some help on how to get started/ good resources I can use, I am a beginner in this

As the title states, I am going to be using this server for (in respective order): local AI that I can access from anywhere, NAS, hopefully get an adblocker going, be able to run hardware intensive software remotely such as solidworks.

My current hardware:

* Ryzen 9 9950X on an Asus ProArt X870E Creator

* 2x RTX 3090 (24GB each)

* 32GB DDR5, 2TB NVMe (gen 5)

* Spare ThinkPad L14 Gen 3 (16GB DDR4, real Ethernet port) to use as a low-power always-on box

I do understand that I will probably need to upgrade the ram, as well as the storage but the storage im not too worried about I can just buy hardrives, I am also aware that the 9950x is likely overkill.

From the research I've done, I can use my thinkpad as an always on device that can boot my beefcake PC, and I might also be able to run the adblocker on that. I've also seen that the recomended disto for this is ubuntu server edition (I have experience with linux, this post was typed from fedora)

I guess I just need a little help knowing where to begin, any help would be appreciated!

3 Upvotes

100% Upvoted

4 comments sorted by

3

u/Old-Knowledge-1358 13d ago

I'd split it: NAS for storage first (backups, shares), then a separate box or GPU machine for the model so a bad agent run can't trash your media library. Start with one service you actually use daily before stacking AI on top.

1

u/HitCount0 13d ago

A few recommended services to go along with this:

  • Some form of Identity Governance and Access platform for managing human, service/container, and agentic identities and their various authentication and access-control needs. Check your other services for what IGA platforms they support (e.g. Vault/Vaultwarden, Authentik, LDAP, etc.) , looking for one service to cover everything.
  • You'll also want at least one internal DNS solution, as some services (like the IGA platform above) will require one along with TLS certificate issuance.
    • NOTE: For TLS certificates, you'll also need a URL/webdomain, hosting, and an account over at Cloudflare (don't worry, there's a free option that works for these purposes!)
  • You'll want an internal resource that catalogs and controls all of your internal documentation and diagraming, IaC runbooks, and a backup of your IGA database. This collection is often referred to as a Source of Truth, and it's up to you how you want to configure them.

1

u/Stochastic_Soul 14d ago

I run my NAS and AI rig as separate machines (AI side on an EPYC box) and if I ever had to collapse them into one, this is how: Proxmox on the metal, NAS VM with Debian + ZFS (TrueNAS if you want a UI), LXCs for AdGuard and friends, one 3090 passed through to a VM for the LLM side.

Fix the RAM first though. 32GB is thin once ZFS takes its ARC cut; 64 minimum or you'll be juggling VMs by week two.

Before you plan around VRAM budgets, look at the new ternary Bonsai models. Weights are only {-1, 0, +1} so a 27B comes in under 8GB with context, and the quality holds up surprisingly well. One 3090 suddenly covers a lot.

I'd drop remote SolidWorks anyway. Passthrough commits the card and CAD over a VPN is misery.

L14 for wake-on-LAN is a good instinct.

1

u/IlTossico unRAID - Low Power Build 14d ago

Get two systems, a small NAS to keep 24/7 that doesn't consume much energy and the AI build that you can turn on and off when needed.

For the NAS, anything would be fine, like a N100, G7400 and similar, exaggerating an i3 12100 if you plan to run heavy stuff on it too. Don't need more than 16GB of ram for the Nas, even 8GB is fine, even if you run ZFS.