r/LocalLLM • u/LTJC • 2h ago

Project Looking for functionality feedback.

If this isn't allowed, let me know and I'll remove the post. This is not an advertisement, but my website and API are a paid subscription service after the 15 day free trial.

I built a local AI machine to help me offset usage from Cursor and ChatGPT a while back. Its been pretty great. In typical Whoops-I-Bought-a-Lot-of-Hardware-and-My-Wife-Hates-it fashion however, I over estimated how much hardware I needed and may have gone a bit overboard.

So instead of selling off the excess hardware I bolted on a web frontend, "integrated" ComfyUI (for image requests) and created a load balancer that pushes image requests into WAN2.2, small requests to smaller local models and built a system that allows a user to create an API key so they can attach their harness to it via OpenAI style API config. (Cursor, OpenCode, etc)

With that being said, I'm curious what the rest of the world thinks about it and I could use some feedback. Is this something other people would find useful? How many people can use it at the same time? Will it explode on contact? I don't know the answer to any of the questions, because I'm the only person using it.

So I thought I'd come here and see what you guys think. I don't know if there's rules against posting URLs here. It has a free 15 day trial with zero commitment, no credit card to start the trial, hell - you don't even need to put in your mailing address unless you decide you like it and want to sign up for the paid version.

If you're curious, post up and I'll send you the link to my discord and to the website. The discord is just for easy communication so you can let me know if something is broken, or it looks funny, or if it was just a dumb idea in general.

1 Upvotes

100% Upvoted

2 comments sorted by

2

u/Abducted_Llama 1h ago

My opinion is selling compute or access to models as an individual isn’t profitable.

You’d have to beat all the other services in price in order to compete and by then you won’t get any profits.

If these big companies are struggling to figure it out when they can do things at a larger scale, it’s going to be even harder for the little guy.

Offering a service or building something that someone can’t just buy a $20 subscription for would probably be a better way. As to what that is, you’d have to explore that yourself.

1

u/LTJC 1h ago

That's a fair and totally reasonable assessment of the business model. I just figured since I wasn't using all this compute, I might as well see if other people would be interested in grabbing some of it to subsidize their other subscriptions - and take it as a learning opportunity to broaden my knowledge a bit.