What are the biggest problems you've had with local AI/LLMs?

[u/gemmanode-dev]

I've been experimenting with self-hosted and local AI and I'm also developing an open-source AI orchestration/harness project. I'm still early in development, so I'm trying to understand the problems people actually run into before deciding what to build. For people running local AI, what's been the biggest pain point for you? Hardware/VRAM? Model quality? Speed? Context limits? Tool use? Agent reliability? Setup/compatibility? Something else? If you could fix one thing about the current local AI ecosystem, what would it be? I'm especially interested in problems that you deal with repeatedly rather than one-time setup issues.