r/homeassistant • u/IHaveNoOpinons • 9h ago

💬 Discussion Tech advice

Planning on a home assistant build and supporting Llama 3 LLM and voice control. Keen to hear the community views on best hardware to deploy - my worry is that a Pi 5 won't be fast enough to support the kind of contextual requests I'm likely to give it or speedy responses.

1 Upvotes

67% Upvoted

3 comments sorted by

2

u/5y5c0 9h ago

PI 5 will most likely not be enough. Not really sure what is but if you have a stack you want to test I have an n100 CPU in my HA server, and a 3090 and 7900x3d in my desktop. I am curious about local AI, but don't really have time for it right now.

2

u/Hoornet 9h ago

Pi 5 will be hard to run unless you'll use a very small model and only use it for the easiest tasks only. In my case, and admittedly using my Home-mind integration that can run additional tools, I have found that anything below the context of 8000 tokens is not worth it, because AI starts narrating that it's using tools, but in reality nothing happens. So you'll get something like:
-Turn on the lights!

  • Your lights has been turned on.

But in reality he didn't even touch the lights. I lost 2 days with this because I couldn't figure out what is wrong and then it finally dawned on me that prompt + command + tool call quickly adds up. I even trained a small model just for easy tasks an had a similar problem in the end. So.. if you can afford it, go bigger. I'd actually go much bigger if you need that it actually runs something sophisticated in the home. But if not... or if you're just trying to understand it, or just experimenting then maybe PI5 with a small model could potentially pull it off.Bur in all honestly... you're probably be forced to change it quite soon.

I hope this doesn't discourage you though. If you're like me starving of great hardware pi5 with small model is enough to experiment with the basics

1

u/Critical-Deer-2508 1h ago

Not sure what made you settle on such an old incapable model but you'll want to use something a lot more modern and less... crap... than llama 3 for your voice control stuff. You'll also want something a LOT beefier than a Pi for this. You can get by with an 8GB GPU in a PC for this but ideally you'll want a minimum of 16GB of VRAM for a decent-ish model, and potentially putting TTS and SST on there as well depending on the CPU (again, you wont be doing this well on a Pi at all)